Inference and calibration of automatic gain compensation

Abstract

The present invention describes methods, systems and computer program products for inferring and correcting automatic gain compensation (AGC) values ​​that vary over time. A device emits a series of inaudible reference audio signals during recording. The reference audio signal has a constant amplitude. A microphone of the device records the reference audio signal while recording audio content. The device may apply AGC during the recording. An AGC inference module receives the recorded signal and extracts a representation of the reference audio signal that has been subjected to the AGC. The AGC inference module determines changes in the level of the extracted representation of the reference audio signal over time. Based on the changes, the AGC inference module infers the AGC applied to the recording over time. The AGC inference module may then provide the AGC for reading, or undo the effect of the AGC applied to the audio content.

Description

[0001] CROSS REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the following priority applications: ES patent application P201731153 filed on September 27, 2017 (reference: D17024ES) and US provisional application 62 / 591,258 filed on November 28, 2017 (reference: D17024US) and EP application 17204135.2 filed on November 28, 2017 (reference: D17024EP), which are hereby incorporated by reference. Technical Field

[0003] The present invention generally relates to audio signal processing. Background Art

[0004] A system implementing automatic gain compensation (AGC) (also referred to as automatic gain control) receives a variable signal level as input and provides an adjusted (e.g., flat) signal level as output. Typically, AGC has the effect of reducing strong signals and enhancing weak signals. AGC may be desirable in some applications (e.g., voice recording), but undesirable in some applications (e.g., music recording). Some modern mobile devices (e.g., some smartphones) are configured to apply AGC to all audio recordings. In such devices, AGC may be enabled or not enabled at the operating system level. These devices may lack an interface for applications running on the device to adjust the system-level AGC. In addition, these devices may not provide information about how much AGC is applied to the application. In various situations, for example, during live music recording, it is desirable to be able to control or at least read the AGC gain. In particular, in music recording, it may be desirable to record all signals with AGC disabled or any AGC applied revoked so that complex dynamic processing specific to music can be applied to each recorded signal or to its mix.

[0005] US 1,137,448 discloses a system in which an ultrasonic pilot tone is used to control the amplification of an audio signal emitted by a loudspeaker. To this end, the loudspeaker is provided with a transducer for the ultrasonic pilot tone and the amplification is controlled so that the amplified pilot tone has a constant level, regardless of the position of the loudspeaker relative to the microphone. Thus, the volume of the loudspeaker's speech is also maintained constant. Summary of the invention

[0006] A technique for inferring an AGC value that varies over time and undoing an AGC effect that varies over time is described. A device (e.g., a smartphone or tablet computer) emits a series of reference audio signals during recording. The reference audio signals have frequencies that are inaudible to humans (e.g., beyond the range of human hearing). A microphone of the device records the reference audio signals while recording audio content. The device may apply AGC during the recording. An AGC inference module receives the recorded signal and extracts a representation of the reference audio signal that has been subjected to the AGC. The AGC inference module determines changes in the level of the extracted representation of the reference audio signal over time. Based on the changes, the AGC inference module infers the AGC applied to the recording over time. The AGC inference module may then provide the AGC to a consuming device. The consuming device may undo the effect of the AGC applied to the audio content.

[0007] A method for determining a time-varying gain applied to audio content during recording is disclosed. The method includes: generating a reference audio signal on a device that captures the audio content that is inaudible to humans in the presence of the audio content; driving a speaker coupled to the device to emit the reference audio signal; and simultaneously capturing the audio content and the reference audio signal as a captured signal. In an embodiment, the reference audio signal is emitted at the same level (i.e., at the same signal power). The capturing may be by a microphone coupled to the device and an associated analog-to-digital converter. The capturing includes applying a common time-varying gain to a signal including the audio content and the reference audio signal. The method further includes extracting a representation of the reference audio signal from the captured signal (which has been subjected to the time-varying gain); determining a change in level of the representation of the extracted reference audio signal over time; and inferring the time-varying gain that has been applied to the audio content during the capturing based on the change in level. All previous steps may be performed on the device mentioned above. The representation of the extracted reference audio signal may include a series of impulse responses corresponding to the reference audio signal. The determined level of the representation of the reference audio signal (e.g., the amplitude of the impulse response) includes information about the amount of the time-varying gain applied to the reference audio signal and the audio content at each time. For example, the level of the representation may be inversely proportional to the applied gain at the corresponding time instant. The method may further include providing information about the inferred time-varying gain (e.g., a signal or value) to another device, such as a consumption device that may also receive the audio content. The consumption device may then undo the effect of the applied time-varying gain on the audio content by applying a corresponding inverse gain to the audio content.

[0008] In addition, a device for determining a time-varying gain applied to audio content during recording is disclosed. The device includes: a reference signal generator configured to generate a reference audio signal that is inaudible to humans in the presence of the audio content for emission through a speaker coupled to the device; a recording interface configured to receive a captured signal including the audio content and the reference audio signal after the time-varying gain has been applied to both the audio content and the reference audio signal; a reference signal extractor configured to determine a representation of the reference audio signal based on the captured signal; and a gain analyzer configured to infer the time-varying gain that has been applied to the audio content during the recording based on a level of the representation of the reference audio signal. The captured signal may be received by a microphone and an associated analog-to-digital converter coupled to the device. The device may further include a gain output configured to output a signal representing the time-varying gain, for example, to a consumption device that also receives the audio content. Alternatively, a gain interface may be provided on the device to provide information about the gain applied to the audio content to other components of the device, which further process the audio content, for example, by undoing the effect of the applied time-varying gain on the audio content. In a similar manner, the recording interface may be a software interface within the device. The device may further include a speaker driver that receives the reference audio signal and is configured to drive the speaker coupled to the device to emit the reference audio signal. The speaker, the microphone, and the analog-to-digital converter may be part of the device, for example, carried by a housing of the device, the housing also carrying a processor configured to implement the device components mentioned above. In an embodiment, the speaker and / or microphone may be connected to the device by a wire (for example, via a USB cable) or wirelessly (for example, via a short-range communication means such as Bluetooth). TM ) is coupled to the device.

[0009] The features described in this specification can achieve one or more advantages over conventional audio recording techniques. The features improve conventional audio recording techniques, where system-level AGC is forced by the system to the audio content and cannot be managed by the application. By revoking the effect of the system-level AGC, the disclosed technology improves the fidelity of the recording. For example, when recording live music with a high dynamic range, the system-level AGC can flatten the recorded level, so that the quiet channel is enhanced and the loud channel is reduced, thereby modifying the original expected dynamics of the performance. The disclosed technology can revoke the effect of the system-level AGC and restore the original expected dynamics. The disclosed technology eliminates the need to design different inference algorithms for different device and operating system combinations. The disclosed technology does not rely on factory calibration of each device or individual terminal to remove the AGC. The disclosed technology does not require special hardware to remove the AGC, because (for example) the disclosed technology can be implemented on consumer smartphones, and most consumer smartphones are capable of emitting and recording signals exceeding 20kHz. The disclosed technology does not need to run a dedicated calibration process that can prevent the device from operating normally or continuing to record. The disclosed technology gives users or audio processing applications more control over audio recording.

[0010] The details of one or more implementations of the disclosed subject matter are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the disclosed subject matter will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a diagram illustrating an example device that implements AGC inference and correction.

[0012] Figure 2 is a diagram illustrating an example series of reference audio signals.

[0013] Figure 3 is a diagram illustrating an example recorded audio signal and an extracted reference audio signal.

[0014] Figure 4 is a graph illustrating example AGC gains inferred by a device.

[0015] Figure 5 is a block diagram illustrating an example AGC inference module.

[0016] Figure 6 is a flow chart of an example process for inferring AGC based on a reference audio signal.

[0017] Figure 7 It is a graphic illustration implementation reference Figures 1 to 6 Block diagram of an example device architecture for a mobile device that describes features and operations.

[0018] Figure 8 yes Figures 1 to 6 Block diagram of an example network operating environment for devices in FIG.

[0019] In the various drawings, like reference symbols indicate like elements. DETAILED DESCRIPTION

[0020] Example device

[0021] Figure 1 1 is a diagram illustrating an example device 102 that implements AGC inference and correction. Device 102 is a computing device that includes one or more processors. For example, device 102 may be a mobile device, such as a tablet computer or a smart phone. Device 102 may be coupled to (e.g., include or be connected to) a microphone 104 and a speaker 106. In various embodiments, device 102 may be coupled to multiple microphones and speakers. For example, device 102 may be plugged into a stereo microphone and speaker, or wirelessly connected to a stereo microphone and speaker. Device 102 is configured to record the audio content of event 108. Event 108 may be any event in which audio content (e.g., speech, vocals, or instrumental music) is produced. For example, event 108 may be a live performance event, such as a concert, in which one or more musical instruments and / or one or more vocalists perform.

[0022] During recording, device 102 emits a series of reference audio signals from speaker 106. Each of the reference audio signals may be a signal having a frequency range beyond the range of human hearing (e.g., above 20kHz). Each of the reference audio signals may have a duration that is preconfigured or determined in real time, for example, one second. Device 102 may emit the reference audio signals at a configurable recurrence rate (e.g., once every two seconds). The recurrence rate corresponds to the desired AGC inference resolution. The level of the emitted reference audio signals is uniform across the series. The level is low compared to the level of the audio content of event 100, but is above the noise floor. Each reference audio signal may be a linear or exponential sweep. The reference audio signal may also be a narrowband noise emitted continuously or in bursts, for example, between 20KHz and 22KHz. Other inaudible reference signals known in the current state of the art to be suitable for loudness estimation may be used.

[0023] Microphone 104 picks up both audio content and reference audio signal. The signal arriving at microphone (including both audio content and reference audio signal) may have a first dynamic range 110. Device 102 includes AGC module 112. AGC module 112 applies AGC to the arrival signal, thereby generating an AGC-processed signal. AGC module 112 may include a submodule that analyzes the power level of the arrival sound and continuously determines a time-varying gain in response. Another AGC submodule receives the currently determined AGC gain value and modifies the arrival signal according to the gain value, thereby changing the overall gain of the signal. For example, AGC module 112 may reduce the loud section of the arrival signal and / or enhance the quiet section of the arrival signal. After being processed by AGC module 112, the AGC-processed signal may have a second dynamic range 114 that may be different from the arrival signal. The arrival audio signal may be flattened by AGC module 112, wherein the second dynamic range 114 is smaller than the first dynamic range 110. Device 102 may prevent a user or application from accessing or manipulating AGC module 112 and reading the amount of AGC applied to a signal. In some implementations, microphone 104 itself has hardware, software, or firmware components that apply AGC to an incoming signal and output an AGC-processed signal. In such a case, AGC module 112 may not be necessary.

[0024] Device 102 may be coupled to an AGC inference module 116, which is configured to determine and optionally cancel the AGC applied to the signal by the AGC module 112. The AGC inference module 116 is a component included in or connected to the device 102. In some implementations, the functions of the AGC inference module 116 may be performed by a cloud-based service connected to the device 102 via the Internet. The AGC inference module 116 receives an AGC-processed signal from the AGC module 112. For example, in the case where the reference signal is an exponential scan, the AGC inference module 116 processes the AGC-processed signal by convolving the AGC-processed signal with the inverse scan. The convolution makes each reference audio signal into an easily identifiable impulse response (IR) and filters out the audible audio picked up by the microphone. The AGC applied by the AGC module 112 may cause the level of the IR in the time series to change. Assuming that the original reference audio signal is emitted at the same level, the AGC inference module 116 can derive the gain applied to the microphone 104 by the AGC module 112 according to the corresponding level of each IR in the sequence. The derived gain can be a time series corresponding to the sequence of reference audio signals. The derived gain is relative, which describes the evolution of the initial unknown absolute gain over time. The absolute gain applied to the recorded signal transmitted through different domains (such as digital signals, analog voltages, electroacoustic transducers, air, etc.) depends on the number of unknown parameters (many parameters other than AGC). In order to achieve compensation of the AGC effect, it is not necessary to know the absolute gain. What must be known is whether this gain changes over time (due to AGC) and how much it changes. This allows the AGC effect to be undone and the signal with the original dynamics to be restored (possibly at different average signal levels).

[0025] AGC inference module 116 may provide the derived sequence of gains to a consuming device, which may be a server or component of device 102. Optionally, the consuming device or AGC inference module 116 may apply the inverse of the sequence of derived gains to the audio content, thereby undoing the effect of the AGC applied to the audio content by AGC module 112. Undoing the effect of the AGC may result in restoring the original dynamic range 110 of the arrival signal, which may be desirable in various scenarios (e.g., in recording music at event 108).

[0026] Instance Signal

[0027] Figure 2 is a diagram illustrating a reference audio signal of an example series. For simplicity and convenience, five reference audio signals of a time series are shown. The horizontal axis represents time in seconds. Figure 2In the top portion of FIG. , the vertical axis represents the level in decibels (dB). Waveforms 202, 204, 206, 208, and 210 representing five reference audio signals are shown. Figure 2 In the bottom portion of FIG. 2 , the vertical axis represents frequency in Hertz (Hz). Spectrograms 212, 214, 216, 218, and 220 represent the frequencies of five reference audio signals. Figure 1 The speaker 106 of the device 102 emits a reference audio signal.

[0028] For the frequency of the reference audio signal, the device may select any frequency that is outside the audible frequency range of humans, can be played by a speaker, and can be effectively picked up by a nearby microphone. In some implementations, the frequency may be above the human hearing range and below one-half the sampling rate of the speaker. For example, for a device with an internal speaker operating at a sampling rate of 48kHz, the device may select a sine wave sweep of the reference audio signal that covers a frequency range between the upper limit of audible frequencies (e.g., 20kHz) and one-half the sampling rate (e.g., 24kHz). The frequency response of the internal speaker is sufficient to be effectively picked up by a nearby microphone. In some implementations, the sweep is a linear sweep. In some implementations, the sweep is an exponential sweep.

[0029] For the level of the reference audio signal, the device may select a level that is higher than the ambient noise floor in the same frequency range. As shown in waveforms 202, 204, 206, 208, and 210, the level is constant across the transmission of copies of the reference audio signal. For each reference audio signal, the device may implement a fade-in and fade-out. As shown in the beginning and ending edges of each of waveforms 202, 204, 206, 208, and 210, the fade-in and fade-out may avoid abrupt transitions in sound at the beginning and end of each reference audio signal.

[0030] For duration and reoccurrence rate, the device may determine how long each reference audio signal lasts and how long the reference audio signals are spaced apart based on a pre-specified and user-configurable desired AGC resolution. Figure 2 , the device may determine that the duration of each reference audio signal is one second, and that the speaker should emit a reference audio signal every two seconds. Shorter or longer durations may be used. Higher or lower recurrence rates may be used. The device may cause the speaker to begin emitting reference audio signals at the frequency, level, duration, and recurrence rate in response to determining that recording has begun.

[0031] Other reference signals may be used. Another example is narrowband noise, for example between 20 KHz and 24 KHz, which may be emitted continuously or in short repetitive bursts similar to the reference signals mentioned above.

[0032] Figure 3 FIG. 1 is a diagram illustrating an example of a recorded audio signal and an extracted reference audio signal. Figure 3 The waveform 302 at the top portion of shows a recorded audio signal including audio content (e.g., a musical performance, speech, ambient noise plus a reference audio signal) emitted from a speaker. The level of the reference audio signal is relatively low compared to the level of the audio content.

[0033] In the example shown, waveform 302 indicates a steady signal being played in a room with a sound source (e.g., one or more speakers) playing audio content. The level of the signal increases and decreases in discrete steps, as determined by a device implementing AGC (e.g., Figure 1 As can be seen in waveform 302, during the loudest portion, the device applies AGC to keep the level constant. While the audio content is being recorded, the device's internal speaker emits a reference audio signal independent of the level of the external sound source.

[0034] The device processes the recorded audio signal as shown in waveform 302. The processing includes convolving the recorded audio signal with an inverse scan corresponding to a reference audio signal. The convolution filters out spectral components outside the frequency range of the reference audio signal and transforms each reference audio signal into a corresponding impulse response (IR) that can be easily identified.

[0035] exist Figure 3 A series 304 of IRs extracted from the recorded audio signal is shown in the bottom portion of FIG. Assuming the reference audio signal is emitted at the same level, the level of the IR will be constant without applying AGC. As shown in the series 304, the level of the IR changes at various times according to the corresponding AGC amount. The level of the IR is inversely proportional to the amount of AGC gain change applied at each time. The device can determine the change and derive the gain change applied to the microphone based on the level of each IR in the series 304.

[0036] Figure 4 is a diagram illustrating example AGC gains inferred by the device. Figure 4 , the horizontal axis represents time in seconds. The vertical axis represents the amount of AGC applied in dB. The device may determine the AGC applied to the audio content over time 402. The device may be based on the above reference Figure 3The device may output the value of AGC 402 to a storage device or to another device. The device may apply a series of gains to the audio signal that compensate for AGC 402. Compensating AGC 402 may include increasing the level of the recorded audio signal at a particular time by the amount of AGC applied to the recorded audio signal at that time. Compensation may undo the effect of the AGC and restore the recorded audio signal to its original original level.

[0037] Figure 5 1 is a block diagram illustrating an example AGC inference module 116. The AGC inference module 116 includes various components, as described in additional detail below. Each component of the AGC inference module 116 may be implemented by one or more computer processors. Figure 1 The AGC inference module 116 is implemented on the device 102 or one or more servers (e.g., one or more computers in a cloud computing environment).

[0038] The AGC inference module 116 includes a reference signal generator 402. The reference signal generator 402 is a component of the AGC inference module 116 configured to generate a time series of reference audio signals. The reference audio signal may be a stable signal, such as a pure tone or bandpass noise above the upper limit of the frequency range of human hearing (20kHz). The reference audio signal may be a sweep above or mostly above the upper limit of the frequency range of human hearing, such as an exponential sweep. The reference signal generator 402 provides the reference audio signal to one or more speakers.

[0039] The AGC inference module 116 may include a recording interface 404. The recording interface 404 is a component of the AGC inference module 116 that is configured to receive an audio signal. The recording interface 404 may receive an audio signal that has been subjected to AGC applied by system software (e.g., by an operating system) or subjected to AGC applied by an AGC-capable microphone or by an application. Typically, the effect of AGC is to attenuate the louder portions of the audio content rather than boosting the soft portions. However, in some cases, AGC may also provide signal amplification. The present invention addresses both cases and provides compensation for the AGC effect in both cases.

[0040] The recording interface 404 provides the received audio signal to a reference signal extractor 406. The reference signal extractor 406 is a component of the AGC inference module 116 that is configured to process the received audio signal (including convolving the received audio signal with an inverse scan) to determine a representation of a reference audio signal (e.g., an IR corresponding to a time series). The reference signal extractor 406 provides the IR series to an AGC analyzer 408.

[0041] The AGC analyzer 408 is a component of the AGC inference module 116 that is configured to infer AGC values ​​applied to the time series of the audio signal based on the level of the representation of the reference signal received from the reference signal extractor 406. The AGC analyzer 408 may provide the inferred AGC values ​​as output to a consuming device, or provide the inferred AGC values ​​to the AGC corrector 410.

[0042] AGC corrector 410 is a component of AGC inference module 116 that is configured to receive an audio signal from recording interface 404 and an AGC value from AGC analyzer 408. AGC corrector 410 can undo the effect of AGC in an audio signal by applying the inverse of the received AGC value to the audio signal. AGC corrector 410 can provide the audio signal as an output to a consuming device with the AGC effect undoed. AGC corrector 410 can be designated as a consuming device of the AGC value.

[0043] The AGC inference module 116 may include a verification module 414. The verification module 414 is a component of the AGC inference module 116 configured to verify the robustness of the AGC inference and correction. The verification module 414 compares the inferred AGC and level of the recorded audio signal to a predicted measured curve of gain change as a function of signal level. The verification may reveal anomalies in the inference. Detecting such anomalies may be useful in determining that the inference is interrupted (e.g., when an emitting speaker is accidentally covered by a user).

[0044] In some implementations, the verification module 414 may analyze the level of the deconvolved IR. If the IR level is below a threshold corresponding to the IR value in normal conditions, even when the AGC is functioning at maximum effect, the verification module 414 may discard this particular IR. The verification module 414 may compare the IR level to the recorded audio signal. If the level of the recorded audio signal at a given time is in a range where the AGC will not function, but the IR level indicates a gain change above a given threshold, the verification module 414 may discard this particular IR.

[0045] Example process

[0046] Figure 6 is a flow chart of an example process 600 for inferring AGC based on a reference audio signal. Figure 1 The system executes process 600 (device 102, server system, or any combination of the two).

[0047] The system for capturing audio content emits (602) a reference audio signal that is inaudible to humans when the audio content is present. The reference audio signal may be in a frequency range that is inaudible to humans. The system may emit the reference audio signal through a speaker coupled to the system (e.g., a built-in speaker on the device, an external speaker plugged into the device, or a speaker wirelessly connected to the device). The reference audio signal may be emitted repeatedly at a recurrence rate corresponding to a given time resolution. The reference audio signal may be above an upper limit of the human hearing frequency range or below a lower limit of the human hearing frequency range. The reference audio signal may be emitted at the same level across the signal. Each of the reference audio signals includes a corresponding scan. The scan may be a linear scan or an exponential scan covering a frequency range between an upper limit of human audible frequencies and one-half of a speaker sampling rate (e.g., between 20kHz and 24kHz), respectively. In some embodiments, the reference audio signal includes bursts of band-limited noise. In some embodiments, the system may continuously emit the reference audio signal, thereby allowing the finest time resolution. The reference audio signal may include continuous narrowband noise.

[0048] The system records (604) the audio content and the reference audio signal using a microphone. The recording is referred to as a recorded signal. The microphone may be a microphone coupled to the system, such as a built-in microphone on the device, an external microphone plugged into the device, or a microphone wirelessly connected to the device.

[0049] The system extracts (606) a representation of a reference audio signal from the recorded signal. The representation of the reference audio signal includes a series of IRs. Extracting the representation may include the following operations. The system convolves the recorded signal with an inverse scan. The convolution filters out spectral components outside the frequency range of the scan. The system converts each scan into a corresponding IR. In an implementation where the reference audio signal includes noise bursts, the system may extract the reference audio signal by filtering the input signal.

[0050] The system determines (608) a change in the level of the representation of the extracted reference audio signal over time.

[0051] The system infers (610) a time-varying gain to be applied to the audio content based on the level changes. The system determines the inferred gain based on the amplitude of the generated IR. Inferring gain can occur in response to determining that each IR is above a threshold amplitude to avoid mistaking accidentally covered speakers for high amounts of gain.

[0052] The system provides (612) the time-varying gain and the audio content to a consuming device. The consuming device may be a device coupled to a speaker and a microphone, or another device. The consuming device may be a storage device, such as a flash memory, a streaming service device, or a playback device.

[0053] In some embodiments, the system may undo the effect of the applied gain on the audio content. The system then provides the audio content to a storage device, a streaming media server, or a playback device with the effect of the applied gain removed. Removing the gain may occur in response to determining that the level of the audio content is in a range where AGC should be applied.

[0054] Exemplary Recording Device Architecture

[0055] Figure 7 It is a graphic illustration implementation reference Figures 1 to 6 700 is a block diagram of an exemplary device architecture 700 of a device that implements the described features and operations. The device may be, for example, Figure 1 The device 102 of the present invention can include a memory interface 702, one or more data processors, an image processor, and / or a processor 704, and a peripheral device interface 706. The memory interface 702, one or more processors 704, and / or a peripheral device interface 706 can be separate components or can be integrated into one or more integrated circuits. The processor 704 can include an application processor, a baseband processor, and a wireless processor. For example, the various components in the mobile device can be coupled by one or more communication buses or signal lines.

[0056] Sensors, devices and subsystems may be coupled to the peripheral device interface 706 to facilitate multiple functionalities. For example, a motion sensor 710, a light sensor 712 and a proximity sensor 714 may be coupled to the peripheral device interface 706 to facilitate the orientation, illumination and proximity functions of the mobile device. A positioning processor 715 may be connected to the peripheral device interface 706 to provide geographic positioning. In some implementations, the positioning processor 715 may be programmed to perform the operation of a GNSS receiver. An electronic magnetometer 716 (e.g., an integrated circuit chip) may also be connected to the peripheral device interface 706 to provide data that can be used to determine the direction of magnetic north. Therefore, the electronic magnetometer 716 may be used as an electronic compass. The motion sensor 710 may include one or more accelerometers configured to determine the change in the speed and direction of movement of the mobile device. A barometer 717 may include one or more devices connected to the peripheral device interface 706 and configured to measure the air pressure around the mobile device.

[0057] Camera subsystem 720 and optical sensor 722 (eg, a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) optical sensor) may be utilized to facilitate camera functions, such as recording photographs and video clips.

[0058] Communication functions may be facilitated by one or more wireless communication subsystems 724, which may include radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystems 724 may depend on the communication network over which the mobile device is intended to operate. For example, the mobile device may include a network designed to communicate via a GSM network, a GPRS network, an EDGE network, a Wi-Fi network, or a wireless network. TM or WiMax TM Network and Bluetooth TM Network-operated communication subsystem 724. In particular, the wireless communication subsystem 724 may include a hosting protocol so that the mobile device can be configured as a base station for other wireless devices.

[0059] The audio subsystem 726 may be coupled to a speaker 728 and a microphone 730 to facilitate supporting voice functions, such as voice recognition, voice replication, digital recording, and telephony functions. The audio subsystem 726 may be configured to receive voice commands from a user.

[0060] The I / O subsystem 740 may include a touch surface controller 742 and / or other input controllers 744. The touch surface controller 742 may be coupled to a touch surface 746 or pad. The touch surface 746 and touch surface controller 742 may, for example, detect contact and movement or the breaking thereof using any of a plurality of touch sensitivity technologies including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface 746. The touch surface 746 may include, for example, a touch screen.

[0061] Other input controllers 744 may be coupled to other input / control devices 748, such as one or more buttons, a rocker switch, a thumb wheel, an infrared port, a USB port, and / or a pointer device (e.g., a stylus). One or more buttons (not shown) may include up / down buttons for volume control of the speaker 728 and / or microphone 730.

[0062] In one embodiment, pressing a button for a first duration may unlock the touch surface 746; and pressing a button for a second duration longer than the first duration may turn power on or off to the mobile device. The user may be able to customize the functionality of one or more of the buttons. For example, the touch surface 746 may also be used to implement virtual or soft buttons and / or a keyboard.

[0063] In some implementations, the mobile device can present recorded audio and / or video files, such as MP3, AAC, and MPEG files. In some implementations, the mobile device can include the functionality of an MP3 player. Other input / output and control devices can also be used.

[0064] The memory interface 702 may be coupled to a memory 750. The memory 750 may include high-speed random access memory and / or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and / or flash memory (e.g., NAND, NOR). The memory 750 may store an operating system 752, such as iOS, Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system (e.g., VxWorks). The operating system 752 may include instructions for handling basic system services and for performing hardware-dependent tasks. In some implementations, the operating system 752 may include a kernel (e.g., a UNIX kernel).

[0065] The memory 750 may also store communication instructions 754 to facilitate communication with one or more additional devices, one or more computers, and / or one or more servers. The memory 750 may include: graphical user interface instructions 756 to facilitate graphical user interface processing; sensor processing instructions 758 to facilitate sensor-related processes and functions; phone instructions 760 to facilitate phone-related processes and functions; electronic messaging instructions 762 to facilitate electronic messaging-related processes and functions; web browsing instructions 764 to facilitate web browsing-related processes and functions; media processing instructions 766 to facilitate media processing-related processes and functions; GNSS / positioning instructions 768 to facilitate general GNSS and positioning-related processes and instructions; camera instructions 770 to facilitate camera-related processes and functions; magnetometer data 772 and calibration instructions 774 to facilitate magnetometer calibration. The memory 750 may also store other software instructions (not shown), such as security instructions, network video instructions to facilitate network video related processes and functions, and / or network shopping instructions to facilitate network shopping related processes and functions. In some embodiments, the media processing instructions 766 are divided into audio processing instructions and video processing instructions to facilitate audio processing related processes and functions and video processing related processes and functions, respectively. Boot records and International Mobile Equipment Identity (IMEI) or similar hardware identifiers may also be stored in the memory 750. The memory 750 may store audio processing instructions 776, which, when executed by the processor 704, may cause the processor 704 to perform various operations, including (for example) Figure 1 and Figure 4 The operation of the AGC inference module 116.

[0066] Each of the instructions and applications identified above may correspond to a set of instructions for performing one or more functions described above. These instructions need not be implemented as separate software programs, program steps, or modules. Memory 750 may include additional instructions or fewer instructions. Furthermore, various functions of the mobile device may be implemented in hardware and / or in software (including in one or more signal processing and / or application specific integrated circuits).

[0067] Figure 8 yes Figures 1 to 6 800. For example, devices 802a and 802b can communicate in data communications via one or more wired and / or wireless networks 810. For example, a wireless network 812 (e.g., a cellular network) can communicate with a wide area network (WAN) 814 (e.g., the Internet) using a gateway 816. Similarly, an access device 818 (e.g., an 802.11g wireless access point) can provide communication access to the wide area network 814. Each of the devices 802a and 802b can be Figure 1 Device 102.

[0068] In some implementations, both voice and data communications may be established via the wireless network 812 and the access device 818. For example, the device 802a may make and receive telephone calls (e.g., using Voice over Internet Protocol (VoIP) protocol), send and receive email messages (e.g., using Post Office Protocol 3 (POP3)), and retrieve electronic documents and / or streams (e.g., web pages, photos, and videos) (e.g., using Transmission Control Protocol / Internet Protocol (TCP / IP) or User Datagram Protocol (UDP)) via the wireless network 812, gateway 816, and wide area network 814. Likewise, in some implementations, the device 802b may make and receive telephone calls, send and receive email messages, and retrieve electronic documents via the access device 818 and wide area network 814. In some implementations, the device 802a or 802b may be physically connected to the access device 818 using one or more cables and the access device 818 may be a personal computer. In this configuration, the device 802a or 802b may be referred to as a "tethered" device.

[0069] Devices 802a and 802b may also establish communication through other means. For example, wireless device 802a may communicate with other wireless devices (e.g., other mobile devices, phones, etc.) via wireless network 812. Similarly, devices 802a and 802b may communicate with each other using one or more communication subsystems (e.g., Bluetooth TM The communication device) establishes peer-to-peer communication 820, for example, a personal area network. Other communication protocols and topologies may also be implemented.

[0070] Device 802a or 802b may communicate with one or more services 830, 840, and 850, for example, via one or more wired and / or wireless networks. For example, one or more audio and video processing services 830 may provide services for audio processing, including revoking the effects of AGC from audio content based on the inferred AGC, as described above. Mixing service 840 may provide a user interface that allows a mixing professional to log in through a remote console to perform post-recording mixing operations on audio content. Streaming service 850 may provide a user interface that allows a user device to download or stream mixed audio content, in which the AGC effects forced on the audio content have been revoked.

[0071] Device 802a or 802b may also access other data and content via one or more wired and / or wireless networks. For example, content publishers (e.g., news sites, Really Simple Syndication (RSS) feeds, websites, blogs, social networking sites, developer networks, etc.) may be accessed by device 802a or 802b. Such access may be provided in response to a user touching, for example, a web page object through invocation of a web browsing function or application (e.g., a browser).

[0072] Several embodiments of the invention have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

Claims

1. A method for determining a time-varying gain applied to audio content during recording, comprising: generating, on a device capturing audio content, a reference audio signal that is inaudible to humans in the presence of the audio content; driving a speaker coupled to the device to emit the reference audio signal; capturing the audio content and the reference audio signal simultaneously as captured signals via a microphone coupled to the device and an associated analog-to-digital converter; wherein capturing comprises applying a common time-varying gain to the audio content and the reference audio signal; extracting a representation of the reference audio signal from the captured signal; determining a change in the level of the extracted representation of the reference audio signal over time; inferring the time-varying gain that has been applied to the audio content during the capture based on the level changes; and Information about the inferred time-varying gain is provided to another device.

2. The method of claim 1, wherein the reference audio signal is re-emitted at a rate corresponding to a given time resolution.

3. The method according to any of the preceding claims, wherein emitting the reference audio signal occurs continuously. 4 . The method according to claim 1 , wherein the reference audio signal is higher than an upper limit of a human hearing frequency range or lower than a lower limit of a human hearing frequency range. The method according to claim 1 , wherein the reference audio signals are at the same level.

6. The method of claim 1 or 2, wherein each of the reference audio signals comprises a scan.

7. The method of claim 6, wherein each scan is an exponential scan that is ultrasonic and is less than one-half the sampling rate of the speaker.

8. The method according to claim 6, wherein: The representation of the reference audio signal comprises a series of impulse responses, and Extracting the representation of the reference audio signal includes convolving the recorded signal with an inverse scan, thereby producing an impulse response for each scan.

9. The method of claim 8, wherein the extrapolated time-varying gain is determined as a function of an amplitude of the generated impulse response.

10. The method of claim 8, wherein the extrapolated time-varying gain is determined in accordance with convolving the generated impulse response with pink noise or white noise and measuring the corresponding rms.

11. The method according to claim 1 or 2, comprising: undoing, by a consuming device, the effect of the applied time-varying gain on the audio content; and The audio content is provided to a storage device, a streaming server, or a playback device with the effect of the applied time-varying gain removed.

12. The method of claim 11, wherein removing the effect of the applied time-varying gain is performed in response to determining that the level of the audio content is in a range in which automatic gain compensation should be applied.

13. A system for determining a time-varying gain applied to audio content during recording, comprising: speaker; microphone; one or more processors; and A non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 12.

14. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of an audio system, cause the one or more processors to perform the method of any one of claims 1 to 12.

15. An apparatus for determining a time-varying gain to be applied to audio content during recording, the apparatus comprising: a reference signal generator configured to generate a reference audio signal that is inaudible to humans in the presence of the audio content for emission through a speaker coupled to the device, a recording interface configured to receive a captured signal after the time-varying gain has been simultaneously applied to the audio content and the reference audio signal, wherein the captured signal includes the audio content and the reference audio signal after applying the time-varying gain; a reference signal extractor configured to extract a representation of the reference audio signal from the captured signal and to determine a change in a level of the extracted representation of the reference audio signal over time; and A gain analyzer is configured to infer the time-varying gain that has been applied to the audio content during the recording from the changes in the level of the representation of the reference audio signal.

16. The device of claim 15, further comprising a speaker driver that receives the reference audio signal and is configured to drive the speaker coupled to the device to emit the reference audio signal.

17. The device of claim 15 or 16, further comprising a gain output configured to output a signal representative of the time-varying gain.

Images