Event-based adaptive equalizer

Abstract

Techniques are described for equalizing audio, e.g., to prevent or mitigate fatigue in response to an identified audio event. In some embodiments, the techniques include processing a moving time window of audio input to identify an audio event, determining an audio effect based on the audio event, generating an audio output by applying the audio effect to the audio input, and producing a sound field based on the audio output.

Description

Technical Field

[0001] This application relates to techniques for audio processing, and more specifically, to using event detection to adaptively equalize and otherwise modify audio effects, for example, to prevent or reduce fatigue. Background Technology

[0002] Audio systems utilize various techniques to achieve post-processing effects to enhance the end-user experience. These effects can create an audio atmosphere or a desired audio effect. However, effects that provide a good first impression of an audio system can lead to listener fatigue over time. For example, bright highs may provide a clear and detailed impression, but over time, fatigue may make the sound harsh or shrill. Similarly, strong mids may initially provide impressive vocals, but over time, the same strong mids may become jarring. A strong bass experience may initially provide a good impression of sound quality, but over time, it can also cause physical fatigue.

[0003] While some products offer limits to prevent listening at high volume levels, existing products cannot adapt to constantly changing audio signals. Instead, post-processing effects represent a trade-off in the reproduced audio within a given audio system, as the limits or effects are applied statically to each track. Therefore, despite volume limits, the resulting audio may still cause fatigue. The resulting audio may also be too low to provide a good impression or the desired audio effect.

[0004] As described above, there is a need in the art for improved techniques for audio effects processing. Summary of the Invention

[0005] One embodiment of this disclosure describes a method comprising: processing an audio signal to identify an audio event based on a value such as a running average generated based on a moving time window of the audio signal; determining one or more audio effects based on the audio event; generating an audio output by applying the one or more audio effects to an audio input; and generating a sound field based on the audio output.

[0006] At least one technical advantage of the disclosed technology over existing technologies is that it enables adaptive modification of the audio field that changes over time to provide a safe and healthy listening experience while maintaining short-term impact, thus offering greater accessibility. The disclosed technology allows audio systems to automatically vary equalization, dynamics, and other audio processing effects over time in response to different types of events, such as those causing fatigue. Therefore, the disclosed technology does not involve trade-offs with static applications but rather applies adaptive effects based on real-time audio analysis. These technical advantages represent one or more technical improvements over existing methods. Attached Figure Description

[0007] By referring to various embodiments, one can understand in detail the aforementioned features of various embodiments in the more specific description of the inventive concept briefly described above, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only illustrate typical embodiments of the inventive concept and should therefore not be construed as limiting the scope in any way, and other equally effective embodiments exist.

[0008] Figure 1 This is a schematic diagram illustrating an audio system according to various implementation schemes;

[0009] Figure 2 This illustrates various implementation schemes. Figure 1 A diagram illustrating the operation of an audio processing application;

[0010] Figure 3 This illustrates various implementation schemes. Figure 1 A diagram illustrating the operation of an audio processing application;

[0011] Figure 4 This illustrates various implementation schemes. Figure 1 A diagram illustrating the operation of an audio processing application;

[0012] Figure 5 This illustrates various implementation schemes. Figure 1 A diagram illustrating the operation of audio processing applications; and

[0013] Figure 6 It is a flowchart of method steps for generating audio output based on audio events such as fatigue events, according to various implementation schemes. Detailed Implementation

[0014] Numerous specific details are set forth in the following description to provide a more thorough understanding of various embodiments. However, it will be apparent to those skilled in the art that the inventive concept can be practiced without one or more of these specific details.

[0015] Figure 1This is a schematic diagram illustrating an audio system 100 according to various embodiments. As shown, the audio system 100 includes, but is not limited to, one or more computing devices 110, one or more microphones 150, and one or more loudspeakers 160. The computing device 110 includes, but is not limited to, one or more processing units 112 and one or more memories 114. In various embodiments, an interconnect bus (not shown) connects one or more processing units 112, one or more memories 114, microphones 150, loudspeakers 160, and any other components of the computing device 110. The one or more memories 114 store, but are not limited to, an audio processing application 118, an event module 120, a profiling module 122, audio inputs 136, audio outputs 138, control data 140, microphone audio 142, audio profiles 144, audio events 146, and audio effects 148. Although shown as a submodule of the audio processing application 118, the event module 120 and the profiling module 122 may include executable instructions that work in conjunction with the audio processing application 118 as submodules and / or separate software modules.

[0016] In various embodiments, one or more computing devices 110 are included in any feasible audio system, such as a vehicle audio system, a home theater system, a soundbar, etc. In some embodiments, one or more computing devices 110 are included in one or more devices, such as consumer products (…). For example, Portable speakers, games, and other products), vehicles ( For example, Main unit or amplifier for cars, trucks, vans, etc.), smart home devices ( For example, Intelligent lighting systems, security systems, digital assistants, etc.), communication systems ( For example, Telephone conferencing systems, video conferencing systems, speaker amplification systems, etc. In various embodiments, one or more computing devices 110 are located in various environments, including but not limited to indoor environments. For example, Living room, meeting room, conference room, home office, etc.) and / or outdoor environment ( For example, Terrace, roof, garden, etc.). The computing device 110 can also provide output audio signals to the speaker 160 or other audio devices. example like, The audio output 138 generated by the audio processing application 118 is used to generate a sound field that provides various audio effects.

[0017] One or more processing units 112 may be any suitable processor, such as a central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), and / or any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU and / or a DSP. Generally, processing unit 112 may be any technically feasible hardware unit capable of processing data and / or executing software applications.

[0018] Memory 114 may include random access memory (RAM) modules, flash memory cells, or any other type of memory cell or combinations thereof. Processing unit 112 is configured to read data from memory 114 and write data to memory 114. In various embodiments, memory 114 includes non-volatile memory, such as optical drives, magnetic drives, flash drives, or other storage. In some embodiments, a separate data storage area, such as an external data storage area (“cloud storage”) included in a network, may supplement memory 114.

[0019] Microphone 150 includes any feasible audio recording device capable of detecting and recording audio. Speaker 160 includes various loudspeakers and other audio devices for outputting audio to create a sound field or various audio effects 148 in the vicinity of the user. In some embodiments, speaker 160 is associated with a speaker configuration that includes identified speaker locations stored in memory 114. The speaker configuration indicates the position and / or orientation of speaker 160 in three-dimensional space and / or relative to each other and / or relative to microphone 150, vehicle, vehicle seat, gaming chair, a particular speaker of speaker 160, computing device 110, user position, etc.

[0020] Audio processing application 118 retrieves or otherwise identifies the speaker configuration of speaker 160 to apply certain audio effects 148 to the audio produced using speaker 160. Audio processing application 118 identifies audio input 136. For example, audio processing application 118 retrieves audio input 136 from memory 114 and / or receives audio input 136 via a network (such as a local area network or wide area network). In some embodiments, audio processing application 118 processes audio input 136 according to a time range or discrete time period, according to a running average or other shifted or running time window audio process.

[0021] Audio input 136 includes any feasible signal or data that includes an audio signal. Audio input 136 includes audio, video, multimedia, or other data files, streams, etc. In some embodiments, audio system 100 receives audio input 136 via a network (such as a local area network or a wide area network). The network may include a public network and / or a private network. Audio system 100 persistently and / or temporarily stores audio input 136 in memory 114. In some embodiments, audio processing application 118 processes audio input 136 in discrete time blocks or time segments. Audio processing application 118 divides audio input 136 into discrete and uniformly spaced time segments according to time units for processing. Audio processing application 118 also divides audio input 136 into frequency ranges, as discussed above.

[0022] Audio output 138 is a processed version of audio input 136, which is processed to include one or more audio effects 148. Audio effects 148 include any feasible modifications to audio input 136 to produce any desired audio output 138. For example, audio effects 148 include equalization, dynamic processing such as compression, expansion, reducing (and / or increasing) transients, reducing (and / or increasing) signal saturation, volume adjustment, reverb, delay, echo, spatial effects, etc. Audio processing application 118 applies one or more audio effects 148 by modifying at least one equalizer value or parameter to perform equalization, perform dynamic processing, or perform volume processing, etc. Therefore, audio processing application 118 dynamically applies one or more of equalization, dynamic equalization, compression, expansion, transient shaping, saturation, volume / gain control, clipping, or excitation (e.g., harmonic excitation) based on audio input 136.

[0023] Event module 120 includes any feasible executable instructions for identifying one or more audio events 146 based on audio signals. In various embodiments, the audio signals include audio input 136 and / or microphone audio 142. Audio events 146 include a plurality of trigger events mapped to one or more audio effects 148 or modifications to one or more audio effects 148. Event module 120 stores the mapping of audio events 146 to audio effects 148 and / or modifications to audio effects 148 in memory 114. Event module 120 stores audio events 146 in memory 114, including fatigue events, brightness events, clarity events, impact events, etc. Fatigue events include high fatigue events (…). For example, Fatigue scores or levels above the threshold and low fatigue events ( For example, (Fatigue score or level below a threshold). For example, a high fatigue event could include an indication that the user may be experiencing fatigue due to an audio system 100 ( For example,An event that causes fatigue with recently generated audio or sound (within a threshold time period) is considered a fatigue event. In contrast, low-fatigue events may include indicators of user fatigue based on the audio system 100 (…). For example, Within a threshold time period, an event that has recently generated audio or sound may be attenuating. In some implementations, high-fatigue events are mapped to audio effects 148 modifications such as reducing transients, reducing signal saturation, reducing one or more equalizer bands, and / or reducing volume. In some implementations, low-fatigue events are mapped to audio effects 148 modifications such as increasing transients, increasing signal saturation, increasing one or more equalizer bands, and / or increasing volume.

[0024] In various implementations, the event module 120 identifies one or more audio events 146 and corresponding one or more audio effects 148 based on a time window of a threshold time value. The event module identifies parameters corresponding to audio input 136 within the time window, microphone audio 142 within the time window, current control data 140, audio profile 144, previous audio events 146, the time elapsed between the current time and the previous audio event 146, and previously and currently applied audio effects 148. In some implementations, the event module 120 identifies one or more audio events 146 and corresponding one or more audio effects 148 and / or modifications or updates to the audio effects 148. The event module 120 bases its identification on the overall peak level within the moving time window (…). For example, For all frequencies), one or more specific peak levels at frequencies ( For example, One or more audio events 146 are identified for one or more frequency ranges, overall and / or frequency-specific root mean square levels, overall and / or frequency-specific signal-to-noise ratios, and other measures of the audio input 136 (or other audio signal). In some embodiments, the event module 120 identifies one or more audio events 146 and corresponding one or more audio effects 148 based on the decibel level of the microphone audio 142 (or other audio signal). In some examples, the audio event 146 corresponds to a frequency range ( For example, Low / mid / high frequency (BLT / QLT) balance events, clarity events, detail, harshness events, impact or shock events, warmth events, muddiness events, soundstage width and immersion events, volume-based equalization events, dynamic equalization events, fatigue events, etc. In some implementations, various different audio events are identified over a range of time ranges or durations, from instantaneous threshold events to longer time range events.146

[0025] The profiling module 122 includes any feasible executable instructions for identifying one or more audio profiles 144 based on audio signals. In some embodiments, the profiling module 122 includes one or more machine learning models, such as a classification machine learning model that selects one or more categories corresponding to at least a subset of the one or more audio profiles 144 based on inputs or input parameters including audio input 136 and / or microphone audio 142 and other parameters such as control data and / or audio effects 148. The machine learning model output indicates one or more values ​​or categories of at least a subset of the one or more audio profiles 144. In some embodiments, the profiling module 122 includes one or more rule-based models that select one or more categories corresponding to at least a subset of the one or more audio profiles 144 based on inputs or input parameters including audio input 136 and / or microphone audio 142 and other parameters such as control data and / or audio effects 148. The rule-based model output indicates one or more values ​​or categories of at least a subset of the one or more audio profiles 144.

[0026] In various implementations, audio profile 144 indicates one or more of the following: a specific genre within a set of genres, beats per minute, dominant frequency or other characteristic frequency, dynamic range, signal-to-noise ratio, other audio characteristics, etc. In some implementations, profiling module 122 stores a mapping of audio profile 144 to audio effect 148 (or modifications to audio effect 148) in memory 114. For example, audio profile 144 may correspond to and / or be identified based on audio characteristics, including but not limited to frequency range (…). For example, The analysis module 122 identifies one or more audio profiles 144 corresponding to the audio signal and provides one or more audio profiles 144 to the event module 120. In some embodiments, various audio profiles 144 are identified over various time ranges or durations, ranging from instantaneous threshold profiles 144 to longer time range audio profiles 144.

[0027] Control data 140 includes any feasible user-specified or user-controlled signals for controlling the audio generated by audio system 100. In some embodiments, control data 140 includes user-specified volume levels, one or more user-specified equalizer parameters, one or more user-specified equalizer parameters, and one or more user-specified spatial parameters. For example, Left-right balance, front-back attenuation wait Control data 140 may also include one or more user-specified audio effects 148.

[0028] Microphone audio 142 includes any feasible audio signals detected using microphone 150. In some embodiments, microphone audio 142 is temporarily stored in access memory (such as RAM of one or more memories 114). In some embodiments, microphone audio 142 is additionally or alternatively stored in non-volatile memory of one or more memories 114.

[0029] Each of the one or more microphones 150 can be any technically feasible type of audio input device, such as any type of moving, condenser, ribbon, microelectromechanical system (MEMS), or other type of microphone. The one or more microphones 150 capture audio including audio signals generated by one or more speakers 160 of the audio system 100. The captured audio is provided to an audio processing application 118.

[0030] Each of the one or more speakers 160 can be any technically feasible type of audio output device. Each of the one or more speakers 160 outputs audio modified by an audio processing application 118. The one or more speakers 160 generate a sound field based on the audio output 138. The sound field includes one or more audio effects 148, which the audio processing application 118 applies to the audio input 136 to generate the audio output 138.

[0031] In one operational example, audio system 100 uses audio processing application 118 to perform audio processing to include or update audio effects 148, thereby reducing fatigue in response to a high fatigue level or increasing impact and / or clarity in response to a low fatigue level. Audio processing application 118 receives audio signals such as audio input 136 and / or microphone audio 142. In some embodiments, audio processing application 118 uses profiling module 122 to process the audio signal to identify one or more audio profiles 144 applicable to the audio signal. Audio processing application 118 also uses event module 120 to process the audio signal to identify one or more audio events 146. Audio processing application 118 identifies audio events 146 based on the overall peak level within a moving time window or time block, one or more frequency-specific peak levels, overall and / or frequency-specific root mean square levels, overall and / or frequency-specific signal-to-noise ratio, moving average peak and / or root mean square levels, and / or other metrics of audio input 136. Audio processing application 118 maps one or more audio events 146 to one or more audio effects 148. Audio processing application 118 applies and / or modifies audio input 136 to include one or more audio effects 148. Therefore, audio processing application 118 generates audio output 138 based on audio input 136 and the most recently identified or updated audio effect 148. In some embodiments, generating audio output 138 involves progressively modifying one or more effect parameters of a previous one or more audio effects 148 to the new audio effect 148 over a specified transition time. Audio processing application 118 provides audio output 138 to one or more speakers 160 to produce a sound field including the audio effect 148.

[0032] Audio processing application 118 responds to different types of audio events 146, including events that cause fatigue, and / or audio profiles 144, by applying one or more audio effects 148 to automatically vary equalization, dynamics, and other effect parameters over time. Audio processing application 118 iteratively performs real-time adaptive processing of the audio signal by applying or adjusting one or more audio effects 148 over time. Audio processing application 118 generates audio output 138 based on control data 140 and the current audio effect 148. In some embodiments, generating audio output 138 involves progressively modifying one or more effect parameters of a previous one or more audio effects 148 to the new audio effect 148 over a specified transition time. The transition time can be instantaneous or any configured time period.

[0033] Figure 2 This illustrates various implementation schemes. Figure 1A diagram illustrating the operation of the audio processing application 118. As shown, the audio processing application 118 and / or associated modules include or perform, but are not limited to, moving time window operation 202, audio event recognition 204, and audio effect adjustment 206. Input data for the audio processing application 118 includes, but is not limited to, audio input 136, audio effects 148, and control data 140. Output data for the audio processing application 118 includes, but is not limited to, audio output 138.

[0034] In the illustrated operational example, audio processing application 118 uses audio input 136 to identify audio event 146. Audio processing application 118 multiplies, mixes, or otherwise modifies audio input 136 based on control data 140 and previous audio effects 148 to generate modified audio input 136. Audio processing application 118 performs one or more shift-time window operations 202 on the modified audio input 136. Shift-time window operation 202 includes processing the modified audio input 136 according to a time range or discrete time intervals to identify running averages or other shift or running time window parameters. In various embodiments, audio processing application 118 performs shift-time window operation 202 to identify the overall peak level of all frequencies within a shift-time window extending backward by a specified time window length from the current time, one or more frequency-specific peak levels of one or more frequency intervals or ranges, overall and / or frequency-specific root mean square levels, overall and / or frequency-specific signal-to-noise ratios, and other metrics of audio input 136.

[0035] Audio processing application 118 uses event module 120, for example, to perform audio event recognition 204. Audio processing application 118 identifies the presence of any audio event 146 within a moving time window or time block, for example, based on overall peak level, one or more frequency-specific peak levels, overall and / or frequency-specific root mean square level, overall and / or frequency-specific signal-to-noise ratio, moving average peak and / or root mean square level, and / or other metrics of audio input 136. In various embodiments, audio processing application 118 identifies fatigue events, brightness events, clarity events, impact events, etc. Each audio event 146 is mapped to one or more audio effects 148. For example, a high fatigue event may be mapped to one or more fatigue-reducing effects, while a low fatigue event may be mapped to an effect that increases impact (…). For example, To increase clarity, impact, brightness, etc.

[0036] If no audio event 146 is identified within the time window, the audio processing application 118 bypasses the audio effects adjustment 206 and generates and / or outputs audio output 138 based on the modified audio input 136, control data 140, and previous audio effects 148. However, if one or more audio events 146 are identified within the time window, the audio processing application 118 enables the audio effects adjustment 206 and generates and / or outputs audio output 138 based on the modified audio input 136, control data 140, and current or new audio effects 148.

[0037] Figure 3 This illustrates various implementation schemes. Figure 1 Another diagram illustrates the operation of the audio processing application 118. As shown, the audio processing application 118 and / or associated modules include or perform, but are not limited to, moving time window operation 302, audio event recognition 304, and audio effect adjustment 306. Input data for the audio processing application 118 includes, but is not limited to, audio input 136, audio effects 148, control data 140, and microphone audio 142. Output data for the audio processing application 118 includes, but is not limited to, audio output 138.

[0038] In the illustrated operational example, audio processing application 118 uses microphone audio 142 to identify audio event 146. In some embodiments, audio processing application 118 determines whether microphone audio 142 is available and utilizes microphone audio 142 if it is available. Audio processing application 118 performs one or more shift-time window operations 302 on microphone audio 142. Shift-time window operation 302 includes processing microphone audio 142 according to a time range or discrete time intervals to identify running averages or other shift or running time window parameters. In various embodiments, audio processing application 118 performs shift-time window operation 302 to identify the overall peak level of all frequencies within a shift-time window extending backward by a specified time window length from the current time, one or more frequency-specific peak levels of one or more frequency intervals or ranges, overall and / or frequency-specific root mean square levels, overall and / or frequency-specific signal-to-noise ratios, and other metrics of microphone audio 142.

[0039] Audio processing application 118 uses event module 120, for example, to perform audio event recognition 304. Audio processing application 118 identifies the presence of any audio event 146 within a moving time window or time block, for example, based on overall peak level, one or more frequency-specific peak levels, overall and / or frequency-specific root mean square level, overall and / or frequency-specific signal-to-noise ratio, moving average peak and / or root mean square level, and / or other metrics of audio input 136. In various embodiments, audio processing application 118 identifies fatigue events, brightness events, clarity events, impact events, etc. Each audio event 146 is mapped to one or more audio effects 148. For example, a high fatigue event may be mapped to one or more fatigue-reducing effects, while a low fatigue event may be mapped to an effect that increases impact (…). For example, To increase clarity, impact, brightness, etc.

[0040] If no audio event 146 is identified within the time window, the audio processing application 118 bypasses the audio effect adjustment 306 and generates and / or outputs audio output 138 based on control data 140, previous audio effects 148, and the modified audio input 136. For example, the audio processing application 118 multiplies, mixes, or otherwise modifies the audio input 136 based on control data 140 and previous audio effects 148 to generate a modified audio input 136. The audio processing application 118 generates and / or outputs audio output 138 based on the modified audio input 136, current control data 140, and previous audio effects 148.

[0041] However, if one or more audio events 146 are identified within the time window, the audio processing application 118 enables audio effects adjustment 306 and generates and / or outputs audio output 138 based on control data 140 and the current audio effect 148, which is based on the audio events 146 identified in the audio event recognition 304.

[0042] Figure 4 This illustrates various implementation schemes. Figure 1 A diagram illustrating the operation of the audio processing application 118. As shown, the audio processing application 118 and / or associated modules include or perform, but are not limited to, signal profiling 400, moving-time window operation 402, audio event recognition 404, and audio effects adjustment 406. The input and intermediate data of the audio processing application 118 include, but are not limited to, audio input 136, audio effects 148, control data 140, and audio profile 144. The output data of the audio processing application 118 includes, but is not limited to, audio output 138.

[0043] In the illustrated operational example, audio processing application 118 uses audio input 136, control data 140, previous audio effects 148, and audio profile 144 to identify audio event 146. Audio processing application 118 multiplies, mixes, or otherwise modifies audio input 136 based on control data 140 and previous audio effects 148 to generate modified audio input 136.

[0044] Audio processing application 118 performs signal profiling 400 based on the modified audio input 136. For example, audio processing application 118 uses profiling module 122 to identify one or more audio profiles 144 for the audio input 136. In the example shown, profiling module 122 uses the modified audio input 136 to identify one or more audio profiles 144. However, in other examples, profiling module 122 uses the unmodified audio input 136 to identify one or more audio profiles 144.

[0045] Audio processing application 118 performs one or more shift-time window operations 402 on modified audio input 136. Shift-time window operation 402 includes processing the modified audio input 136 according to a time range or discrete time intervals to identify running averages or other shift or running time window parameters. In various embodiments, audio processing application 118 performs shift-time window operation 402 to identify the overall peak level of all frequencies within a shift-time window extending backward by a specified time window length from the current time, one or more frequency-specific peak levels of one or more frequency intervals or ranges, overall and / or frequency-specific root-mean-square levels, overall and / or frequency-specific signal-to-noise ratios, and other metrics of the audio input 136.

[0046] Audio processing application 118 uses event module 120, for example, to perform audio event recognition 404. Audio processing application 118 identifies the presence of any audio event 146 within a moving time window, for example, based on overall peak level, one or more frequency-specific peak levels, overall and / or frequency-specific root mean square level, overall and / or frequency-specific signal-to-noise ratio, moving average peak and / or root mean square level, and / or other metrics of audio input 136. In various embodiments, audio processing application 118 identifies fatigue events, brightness events, clarity events, impact events, etc. In the illustrated embodiment, audio processing application 118 further identifies the presence of any audio event 146 based on audio profile 144 identified by signal profiling 400. Each audio event 146 is mapped to one or more audio effects 148. For example, a high fatigue event may be mapped to one or more fatigue-reducing effects, while a low fatigue event may be mapped to an effect that increases impact. For example, To increase clarity, impact, brightness, etc.

[0047] If no audio event 146 is identified within the time window, the audio processing application 118 bypasses the audio effects adjustment 406 and generates and / or outputs audio output 138 based on the modified audio input 136, control data 140, and previous audio effects 148. However, if one or more audio events 146 are identified within the time window, the audio processing application 118 enables the audio effects adjustment 406 and generates and / or outputs audio output 138 based on the modified audio input 136, control data 140, and current or new audio effects 148.

[0048] Figure 5 This illustrates various implementation schemes. Figure 1 Another diagram illustrates the operation of the audio processing application 118. As shown, the audio processing application 118 and / or associated modules include or perform, but are not limited to, signal profiling 500, moving-time window operation 502, audio event recognition 504, and audio effects adjustment 506. The input and intermediate data of the audio processing application 118 include, but are not limited to, audio input 136, audio effects 148, control data 140, microphone audio 142, and audio profile 144. The output data of the audio processing application 118 includes, but is not limited to, audio output 138.

[0049] In the illustrated operational example, the audio processing application 118 uses microphone audio 142 to identify audio event 146. In some implementations, the audio processing application 118 determines whether microphone audio 142 is available and utilizes microphone audio 142 if it is available.

[0050] Audio processing application 118 performs signal profiling 500 based on audio input 136. For example, audio processing application 118 uses profiling module 122 to identify one or more audio profiles 144 for audio input 136. In the example shown, profiling module 122 uses audio input 136 to identify one or more audio profiles 144. However, in other examples, profiling module 122 uses modified audio input 136 to identify one or more audio profiles 144.

[0051] Audio processing application 118 performs one or more shift-time window operations 502 on microphone audio 142. Shift-time window operations 502 include processing microphone audio 142 according to a time range or discrete time intervals to identify running averages or other shift or running time window parameters. In various embodiments, audio processing application 118 performs shift-time window operations 502 to identify the overall peak level of all frequencies within a shift-time window extending backward by a specified time window length from the current time, one or more frequency-specific peak levels of one or more frequency intervals or ranges, overall and / or frequency-specific root-mean-square levels, overall and / or frequency-specific signal-to-noise ratios, and other metrics of microphone audio 142.

[0052] Audio processing application 118 uses event module 120, for example, to perform audio event recognition 504. Audio processing application 118 identifies the presence of any audio event 146 within a moving time window or time block, for example, based on overall peak level, one or more frequency-specific peak levels, overall and / or frequency-specific root mean square level, overall and / or frequency-specific signal-to-noise ratio, moving average peak and / or root mean square level, and / or other metrics of audio input 136. In various embodiments, audio processing application 118 identifies fatigue events, brightness events, clarity events, impact events, etc. In the illustrated embodiment, audio processing application 118 further identifies the presence of any audio event 146 based on audio profile 144 identified by signal profiling 500. Each audio event 146 is mapped to one or more audio effects 148. For example, a high fatigue event may be mapped to one or more fatigue-reducing effects, while a low fatigue event may be mapped to an effect that increases impact. For example, To increase clarity, impact, brightness, etc.

[0053] If no audio event 146 is identified within the time window, the audio processing application 118 bypasses the audio effects adjustment 506 and generates and / or outputs audio output 138 based on control data 140, previous audio effects 148, and the modified audio input 136. For example, the audio processing application 118 multiplies, mixes, or otherwise modifies the audio input 136 based on control data 140 and previous audio effects 148 to generate a modified audio input 136. The audio processing application 118 generates and / or outputs audio output 138 based on the modified audio input 136, control data 140, and previous audio effects 148.

[0054] However, if one or more audio events 146 are identified within the time window, the audio processing application 118 enables audio effects adjustment 506 and generates and / or outputs audio output 138 based on the modified audio input 136, control data 140, and the current audio effects 148 identified in the audio event recognition 304.

[0055] Figure 6 This is a flowchart of method steps for generating audio output based on audio events such as fatigue events, according to various embodiments. Although the method steps are shown in sequence, those skilled in the art will understand that some method steps may be performed in a different order, repeated, omitted, and / or replaced by other methods. Figure 6 The description covers the execution of components other than those in the document. While the method steps are about... Figures 1 to 5 This is a system description, but those skilled in the art should understand that any system configured to perform method steps in any order falls within the scope of various implementations.

[0056] As shown, method 600 begins at step 602, where audio processing application 118 receives or identifies one or more audio signals. In various embodiments, the audio signals may include audio input 136, modified audio input 136, microphone audio 142, and / or combinations thereof. For example, audio processing application 118 retrieves audio input 136 from memory 114 and / or receives audio input 136 via a network (such as a local area network or a wide area network).

[0057] At step 604, the audio processing application 118 identifies audio profiles 144 based on audio signals such as audio input 136, modified audio input 136, microphone audio 142, and / or combinations thereof. The audio processing application 118 performs signal profiling based on the audio signals. For example, the audio processing application 118 uses a profiling module 122 to identify one or more audio profiles 144. The profiling module 122 includes any executable instructions for identifying one or more audio profiles 144 based on audio signals. In some embodiments, the profiling module 122 includes one or more machine learning models and / or one or more rule-based models that select one or more categories corresponding to at least a subset of the one or more audio profiles 144 based on inputs including audio input 136 and / or microphone audio 142, and other parameters such as control data and / or audio effects 148.

[0058] At step 606, the audio processing application 118 processes the audio signal based on a moving-time window to identify one or more audio events 146. The audio processing application 118 performs one or more moving-time window operations on the audio input 136 or microphone audio 142. Moving-time window operations include processing the audio signal according to a time range or discrete time intervals to identify running averages or other moving or running-time window parameters. In various embodiments, the audio processing application 118 performs moving-time window operations to identify the overall peak level of all frequencies within a moving-time window extending backward by a specified time window length from the current time, one or more frequency-specific peak levels of one or more frequency intervals or ranges, overall and / or frequency-specific root-mean-square levels, overall and / or frequency-specific signal-to-noise ratios, and other metrics of the audio signal.

[0059] At step 608, the audio processing application 118 applies or adjusts one or more audio effects 148 based on one or more audio events 146. The audio processing application 118 maps each of the one or more audio events 146 to a corresponding one or more audio effects 148. The audio processing application 118 applies one or more audio effects 148 in response to different types of audio events, including events that cause fatigue, and / or audio profiles 144, to automatically vary equalization, dynamics, and other effect parameters over time. The audio processing application 118 iteratively performs real-time adaptive processing of the audio signal by applying or adjusting one or more audio effects 148 over time. The audio processing application 118 generates an audio output 138 based on control data 140 and the current audio effect 148. In some embodiments, generating the audio output 138 involves progressively modifying one or more effect parameters of the previous one or more audio effects 148 to the new audio effect 148 over a specified amount of transition time.

[0060] At step 610, the audio processing application 118 provides audio output 138 to one or more speakers 160. The speakers 160 generate a sound field including audio effects 148. Therefore, the audio processing application 118 provides adaptive modification of the audio field to provide a safe and healthy listening experience. This adaptive modification of the audio field maintains short-term impact and other effects, thus providing greater accessibility, while maintaining long-term safety and / or adapting to optimized sound reproduction of the audio input 136. Therefore, the audio processing application 118 adaptively optimizes or otherwise modifies the listening experience based on the audio input 136 over time.

[0061] In summary, techniques for audio processing, and more specifically for equalizing audio to prevent or mitigate fatigue or adaptively optimize the listening experience based on audio input, are disclosed. The described techniques involve adaptive modification of the audio field to ensure that the audio system produces the desired effect while preventing fatigue and maximizing short-term effects such as impact, clarity, or brightness. The described techniques are also performed iteratively over time to achieve real-time adaptive processing. The described techniques include receiving an audio signal, analyzing the audio signal within a moving time window to determine audio events (such as fatigue events), applying audio effects based on the audio events, and outputting an audio output to generate a sound field including the audio effects using one or more speakers.

[0062] At least one technical advantage of the disclosed technology over the prior art is that it enables adaptive modification of the audio field to provide a safe and healthy listening experience while maintaining short-term impact, thereby providing greater accessibility. The disclosed technology allows audio systems to automatically vary equalization, dynamics, and other audio processing effects over time in response to different types of events, such as events that cause fatigue. These technical advantages represent one or more technical improvements over existing methods.

[0063] 1. In some embodiments, a computer-implemented method includes: processing the audio input based on a moving time window to identify an audio event; determining one or more audio effects based on the audio event; generating an audio output by applying the one or more audio effects to the audio input; and providing the audio output to one or more speakers.

[0064] 2. The computer-implemented method as described in Clause 1 further includes: retrieving the audio input from a memory; or receiving the audio input via a network.

[0065] 3. The computer-implemented method as described in Clause 1 or 2, further comprising: identifying an audio profile of the audio input.

[0066] 4. A computer-implemented method as described in any one of Clauses 1 to 3, wherein identifying the audio profile comprises: providing the audio input as input parameters to a machine learning model, the machine learning model outputting at least one value associated with the audio profile.

[0067] 5. A computer-implemented method as described in any one of Clauses 1 to 4, wherein identifying the audio profile comprises: providing the audio input as input parameters to a rule-based model, the rule-based model outputting at least one value associated with the audio profile.

[0068] 6. A computer-implemented method as described in any one of Clauses 1 to 5, wherein the audio event corresponds to a fatigue event identified based on the audio profile and the running average of the moving time window.

[0069] 7. A computer-implemented method as described in any one of Clauses 1 to 6, wherein the audio profile indicates at least one of the following: genre of the audio input, beats per minute, frequency, frequency range balance, clarity characteristics, detail characteristics, harshness characteristics, impact characteristics, warmth characteristics, muddiness characteristics, soundstage width, soundstage immersion, volume-based equalization, dynamic equalization, fatigue characteristics, dynamic range, or signal-to-noise ratio.

[0070] 8. A computer-implemented method as described in any one of Clauses 1 to 7, wherein the audio event is identified based on the moving time window and audio system control data.

[0071] 9. A computer-implemented method as described in any one of Clauses 1 to 8, wherein applying the one or more audio effects comprises at least one of: modifying at least one equalizer value, performing dynamic processing, or performing volume processing.

[0072] 10. A computer-implemented method as described in any one of clauses 1 to 9, wherein identifying the audio event comprises: comparing a running average of the moving time window with at least one loudness level threshold.

[0073] 11. In some embodiments, one or more non-transitory computer-readable media store instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: processing an audio signal based on a moving time window of an audio input to identify an audio event; determining one or more audio effects based on the audio event; applying the one or more audio effects to the audio input to generate an audio output; and generating a sound field based on the audio output.

[0074] 12. One or more non-transitory computer-readable media as described in Clause 11, wherein the audio signal includes the audio input or microphone audio detected using a microphone device.

[0075] 13. One or more non-transitory computer-readable media as described in Clause 11 or 12, wherein the one or more processors execute the instructions to perform the following further steps: retrieving the audio input from memory; or receiving the audio input via a network, the network including at least one of a local area network or a wide area network.

[0076] 14. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 13, wherein the one or more processors execute the instructions to perform the step of: identifying an audio profile of the audio input.

[0077] 15. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 14, wherein identifying the audio profile comprises: processing the audio input using a machine learning model that identifies the audio profile.

[0078] 16. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 15, wherein identifying the audio profile comprises: processing the audio input using a rule-based model that identifies the audio profile.

[0079] 17. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 16, wherein the audio event is a fatigue event.

[0080] 18. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 17, wherein the audio profile indicates at least one of the following: the genre of the audio input, beats per minute, frequency, dynamic range, or signal-to-noise ratio.

[0081] 19. One or more non-transitory computer-readable media as described in any one of Clauses 11 to 18, wherein the audio event is identified based on the moving time window and audio system control data.

[0082] 20. In some embodiments, a system includes: one or more speakers; a memory storing instructions; and one or more processors configured, when executing the instructions, to perform the following steps: processing a moving time window of an audio input to identify one or more audio events; determining one or more audio effects based on the one or more audio events; generating an audio output by applying the one or more audio effects to the audio input; and generating a sound field based on the audio output.

[0083] Various embodiments have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0084] Various aspects of this implementation can be embodied as a system, method, or computer program product. Therefore, various aspects of this disclosure can be implemented entirely in hardware or entirely in software (including firmware, resident software, and microcode). wait The embodiments may take the form of software and hardware aspects, which are generally referred to herein as “modules” or “systems”. Furthermore, aspects of this disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied thereon.

[0085] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination of the foregoing. More specific examples (not an exhaustive list) of computer-readable storage media may include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that may contain or store programs for use with or in connection with an instruction execution system, device, or apparatus.

[0086] The foregoing description of various aspects of this disclosure is based on flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implement the manner in which the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams are performed. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, application-specific processors, or field-programmable processors or gate arrays.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this respect, each box in a flowchart or block diagram may represent a module, segment, or portion of code comprising one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in some alternative implementations, the functions indicated in the boxes may occur in a different order than indicated in the drawings. For example, two boxes shown consecutively may actually be executed substantially simultaneously, or depending on the functionality involved, the boxes may sometimes be executed in reverse order. It should also be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware or a combination of dedicated hardware and computer instructions that performs the specified functions or actions.

[0088] Although the foregoing describes an embodiment of this disclosure, other and more embodiments of this disclosure may be devised without departing from the basic scope of this disclosure, the scope of which is defined by the appended claims.

Claims

1. A computer-implemented method, comprising: The audio input is processed based on a moving time window to identify audio events; One or more audio effects are determined based on the audio events; An audio output is generated by applying one or more of the audio effects to the audio input; as well as The audio output is provided to one or more speakers.

2. The computer-implemented method as described in claim 1, further comprising: Retrieve the audio input from memory; or The audio input is received via a network.

3. The computer-implemented method as described in claim 1, further comprising: Identify the audio profile of the audio input.

4. The computer-implemented method of claim 3, wherein identifying the audio profile includes: The audio input is provided as an input parameter to a machine learning model, which outputs at least one value associated with the audio profile.

5. The computer-implemented method of claim 3, wherein identifying the audio profile includes: The audio input is provided as an input parameter to a rule-based model, which outputs at least one value associated with the audio profile.

6. The computer-implemented method of claim 3, wherein the audio event corresponds to a fatigue event identified based on the audio profile and the running average of the moving time window.

7. The computer-implemented method of claim 3, wherein the audio profile indicates at least one of the following: genre of the audio input, beats per minute, frequency, frequency range balance, clarity characteristics, detail characteristics, harshness characteristics, impact characteristics, warmth characteristics, muddiness characteristics, soundstage width, soundstage immersion, volume-based equalization, dynamic equalization, fatigue characteristics, dynamic range, or signal-to-noise ratio.

8. The computer-implemented method of claim 1, wherein the audio event is identified based on the moving time window and audio system control data.

9. The computer-implemented method of claim 1, wherein applying the one or more audio effects includes at least one of: modifying at least one equalizer value, performing dynamic processing, or performing volume processing.

10. The computer-implemented method of claim 1, wherein identifying the audio event comprises: The running average of the moving time window is compared with at least one loudness level threshold.

11. One or more non-transitory computer-readable media, said non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause said one or more processors to perform the following steps: Audio signals are processed based on a moving time window of the audio input to identify audio events; One or more audio effects are determined based on the audio events; Apply one or more audio effects to the audio input to generate an audio output; as well as A sound field is generated based on the audio output.

12. One or more non-transitory computer-readable media as claimed in claim 11, wherein the audio signal includes the audio input or microphone audio detected using a microphone device.

13. The one or more non-transitory computer-readable media of claim 11, wherein the one or more processors execute the instructions to perform the following further steps: Retrieve the audio input from memory; or The audio input is received via a network, which includes at least one of a local area network (LAN) or a wide area network (WAN).

14. The one or more non-transitory computer-readable media of claim 11, wherein the one or more processors execute the instructions to perform the following steps: Identify the audio profile of the audio input.

15. One or more non-transitory computer-readable media as claimed in claim 14, wherein identifying the audio profile comprises: The audio input is processed using a machine learning model that identifies the audio profile.

16. One or more non-transitory computer-readable media as claimed in claim 14, wherein identifying the audio profile comprises: The audio input is processed using a rule-based model that identifies the audio profile.

17. One or more non-transitory computer-readable media as claimed in claim 14, wherein the audio event is a fatigue event.

18. One or more non-transitory computer-readable media as claimed in claim 14, wherein the audio profile indicates at least one of the following: genre of the audio input, beats per minute, frequency, dynamic range, or signal-to-noise ratio.

19. One or more non-transitory computer-readable media as claimed in claim 11, wherein the audio event is identified based on the moving time window and audio system control data.

20. A system comprising: One or more speakers; The memory stores instructions; and One or more processors, wherein the one or more processors are configured to perform the following steps when executing the instructions: Process the moving time window of audio input to identify one or more audio events; One or more audio effects are determined based on the one or more audio events; An audio output is generated by applying one or more of the audio effects to the audio input; as well as A sound field is generated based on the audio output.

Images