Intelligent sensing-based adaptive real-time reverberation processing method and system

Abstract

The application provides an adaptive real-time reverberation processing method and system based on intelligent sensing. The method comprises the following steps: dividing input audio into n frequency band regions according to sensing importance, and dynamically adjusting the frequency division point position; performing differential processing on the signals of each frequency band, and dynamically adjusting the operation parameters in the differential processing in combination with the real-time load of the system; performing frequency domain block convolution processing on the n frequency band signals based on a pre-constructed multi-level cache architecture; mixing the convolution results of the n frequency band signals according to the human ear hearing masking effect, and outputting the final reverberation audio through a dynamic range control mechanism while ensuring the phase continuity among the frequency bands. The application successfully realizes the dynamic balance between professional sound quality and high real-time performance in the limited computing resource scene in the field of real-time reverberation processing, significantly improves the real-time response speed of audio reverberation processing and the multi-device heterogeneous adaptation capability on the basis of ensuring the reverberation sound quality, and maximally reduces the invalid consumption of computing resources.

Description

Technical Field

[0001] This application relates to the field of audio reverberation, and more specifically, to an adaptive real-time reverberation processing method and system based on intelligent perception. Background Technology

[0002] Reverb processing is one of the core technologies in the field of audio signal processing, widely used in audio enhancement scenarios of consumer devices such as mobile terminals and embedded systems. Its core objective is to improve the spatial sense and naturalness of audio signals by simulating acoustic spatial reflection characteristics. However, existing reverb processing algorithms face several technical bottlenecks in practical applications, making it difficult to meet the professional-grade processing needs in scenarios with limited computing resources.

[0003] The computational complexity of traditional convolutional reverberation algorithms increases exponentially with the sampling rate. In high sampling rate scenarios of 48kHz and above, there is an irreconcilable contradiction between the real-time processing requirements for long impulse responses and the limited CPU computing power and memory bandwidth of consumer devices, making it difficult to popularize high-quality reverberation effects on mobile devices such as mobile phones and smart speakers. At the same time, existing algorithms generally adopt fixed processing strategies, which cannot perceive differences in device hardware characteristics and changes in real-time system state. They exhibit significant performance inconsistencies in heterogeneous computing environments, failing to fully utilize the hardware potential of high-performance devices and easily causing stuttering and latency issues on low-computing-power devices.

[0004] Therefore, a reverberation processing solution is needed to solve the above problems. Summary of the Invention

[0005] To address the problems existing in current technologies, this application provides an adaptive real-time reverberation processing method and system based on intelligent perception. The specific solution is as follows:

[0006] An adaptive real-time reverberation processing method based on intelligent perception includes:

[0007] Based on multiple preset critical frequency bands, the input audio is divided into n frequency band regions according to perceptual importance, and the position of the frequency division point between the frequency band regions is dynamically adjusted based on the real-time load status of the system.

[0008] Based on a preset signal processing scheme, signals in each frequency band are processed differently, and the operation parameters in the differentiating processing are dynamically adjusted according to the spectral characteristics of each frequency band region and the real-time load status of the system, so as to output n frequency band signals.

[0009] Frequency domain block convolution processing is performed on n frequency band signals based on a pre-built multi-level caching architecture;

[0010] Based on the human auditory masking effect, the convolution results of n frequency band signals are mixed, and the final reverberant audio is output through a dynamic range control mechanism while ensuring the phase continuity between frequency bands.

[0011] In some specific embodiments, 24 critical frequency bands are established based on the human hearing critical frequency band theory and the Bark psychoacoustic scale. The input audio signal is divided into three frequency band regions according to the perceptual importance: low frequency region, mid frequency region, and high frequency region. Each frequency band region corresponds to a signal processing scheme.

[0012] In some specific embodiments, the signal processing scheme for the low-frequency region includes employing high-precision temporal convolution processing to preserve spatial positioning and phase information;

[0013] The signal processing scheme for the mid-frequency band region includes a hybrid processing mode of convolution and synthesis.

[0014] The signal processing scheme for the high-frequency band includes intelligent synthesis based on a particle system model using acoustic physics principles.

[0015] In some specific embodiments, in a particle system based on acoustic physics principles, each acoustic particle represents an independent reflected sound path, and the particle state parameters include three-dimensional spatial position, velocity vector, frequency-related attenuation coefficient, propagation delay time, and reflection count; through the random distribution of particles, Brownian motion, and boundary reflection behavior, the spatial diffusion characteristics and temporal attenuation law of high-frequency reverberation are accurately simulated.

[0016] In some specific embodiments, the multi-level caching architecture includes a first cache level, a second cache level, and a third cache level;

[0017] Real-time data is processed through the first cache level, high-frequency multiplexed data is processed through the second cache level, and preloaded data for the next processing cycle is predicted through the third cache level, so as to perform frequency domain block convolution processing on n frequency band signals.

[0018] In some specific embodiments, the first cache level stores the FFT transformation result of the current processing block and uses the SIMD instruction set to optimize complex number operations to process real-time data; the second cache level caches impulse response frequency domain data in the mid-frequency band based on the LRU strategy to process high-frequency multiplexed data; and the third cache level preloads the computation data of the next processing cycle in the low-frequency band through an access pattern prediction algorithm.

[0019] In some specific embodiments, obtaining the reverberant audio specifically includes:

[0020] By utilizing the psychoacoustic principles of simultaneous masking and temporal masking, the mixing weights of the low-frequency convolution result and the high-frequency synthesized signal are dynamically adjusted; and a phase coherence preservation algorithm is used to ensure the phase continuity and spatial consistency of different frequency band components at the synthesis point.

[0021] Through an adaptive dynamic range control mechanism, the gain distribution is intelligently adjusted according to the audio characteristics of the input audio and the audio characteristics of the output device, effectively preventing clipping distortion and dynamic range compression.

[0022] In some specific embodiments, the computational parameters of the differentiated processing are adjusted by a machine learning-based adaptive computing power adjustment module. These computational parameters include hardware parameters, system status, and application scenarios.

[0023] In some specific embodiments, the hardware parameters include CPU architecture type, number of physical cores, cache hierarchy, memory bandwidth capacity, and floating-point operation capability;

[0024] The system status includes real-time CPU load, core temperature, battery level, network connection status, and background process contention.

[0025] The application scenarios include audio content type, user preference settings, ambient noise level, and latency sensitivity requirements.

[0026] An adaptive real-time reverberation processing system based on intelligent perception is provided for implementing the adaptive real-time reverberation processing method based on intelligent perception described in any of the above-mentioned embodiments. The adaptive real-time reverberation processing system includes:

[0027] The frequency band processing module is used to divide the input audio into n frequency band regions according to perceptual importance based on multiple preset critical frequency bands, and dynamically adjust the frequency division point position between frequency band regions based on the real-time load status of the system; and perform differentiated processing on the signals of each frequency band based on preset signal processing schemes.

[0028] The computing power adaptive adjustment module is used to dynamically adjust the computing parameters for differentiated processing and output n frequency band signals;

[0029] The frequency domain block convolution module is used to perform frequency domain block convolution processing on n frequency band signals based on a pre-built multi-level caching architecture.

[0030] The output synthesis module is used to mix the convolution results of n frequency band signals based on the human auditory masking effect, and output the final reverberant audio through a dynamic range control mechanism while ensuring phase continuity between frequency bands.

[0031] Beneficial Effects: This application proposes an adaptive real-time reverb processing method and system based on intelligent perception. Through a deep collaborative design of perception-driven optimization, hardware adaptation enhancement, and resource efficiency improvement, it precisely breaks through the technical bottleneck of the traditional reverb algorithm where computing power and sound quality cannot be simultaneously achieved. In the field of real-time reverb processing, it successfully achieves a dynamic balance between professional-grade sound quality and high real-time performance in scenarios with limited computing resources. While ensuring reverb sound quality, it significantly improves the real-time response speed and multi-device heterogeneous adaptation capability of audio reverb processing, while minimizing the ineffective consumption of computing resources. It not only meets users' core needs for immersive and high-fidelity reverb listening experience, but also effectively ensures the long-term stability of device operation and low-latency real-time performance of audio processing.

[0032] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of the adaptive real-time reverberation processing method of this application;

[0035] Figure 2 This is a schematic diagram illustrating the principle of the adaptive real-time reverberation processing method of this application;

[0036] Figure 3 This is a schematic diagram of the adaptive real-time reverberation processing system module of this application.

[0037] Figure labels: 1-Frequency band processing module; 2-Computing power adaptive adjustment module; 3-Frequency domain block convolution module; 4-Output synthesis module. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0039] This application proposes an adaptive real-time reverberation processing method based on intelligent perception, achieving a dynamic balance between professional-grade sound quality and high real-time performance in computationally limited scenarios. A flowchart of the method is attached. Figure 1 As shown, an example of the principle is attached. Figure 2 As shown, the specific solution is as follows:

[0040] An adaptive real-time reverberation processing method based on intelligent perception includes:

[0041] 101. Based on multiple preset critical frequency bands, the input audio is divided into n frequency band regions according to perceptual importance, and the position of the frequency division point between the frequency band regions is dynamically adjusted based on the real-time load status of the system.

[0042] 102. Based on the preset signal processing scheme, perform differentiated processing on signals of each frequency band, and dynamically adjust the operation parameters in the differentiated processing according to the spectral characteristics of each frequency band region and the real-time load status of the system, and output n frequency band signals;

[0043] 103. Perform frequency domain block convolution processing on n frequency band signals based on a pre-built multi-level caching architecture;

[0044] 104. Based on the human auditory masking effect, the convolution results of n frequency band signals are mixed, and the final reverberant audio is output through a dynamic range control mechanism while ensuring the phase continuity between frequency bands.

[0045] Step 101 forms the foundational partitioning logic of the entire method, involving frequency band division based on human auditory perception characteristics and dynamic adaptation based on system state. Dividing frequency bands according to perceptual importance avoids the computational waste of traditional algorithms that apply uniform high-precision processing to all frequency bands. Dynamically adjusting the crossover point achieves adaptation between system state and frequency band division. Under high load, adjusting the crossover point prioritizes real-time performance; under low load, it prioritizes sound quality, avoiding the problems of fixed crossover points causing stuttering under high load and idle computational power under low load.

[0046] The n frequency bands are not uniformly divided into n equal-width bands, but rather priority intervals obtained by merging or splitting based on the perceived contribution of critical frequency bands. For example, the 0-800Hz low-frequency band, which plays a key role in spatial positioning (the human ear is sensitive to phase changes and directly affects the judgment of the direction from which the sound comes), is divided into high-perception priority frequency bands, while the high-frequency bands above 4kHz, which are less sensitive to human ears, are divided into low-perception priority frequency bands. The specific value of n is dynamically determined by the algorithm according to the critical frequency band merging rules (rather than a fixed value). The core is to ensure that high-perception important frequency bands are divided independently and low-perception important frequency bands are merged.

[0047] The critical frequency band is not an arbitrarily defined frequency range, but rather a pre-defined range of human ear sensitivity to sound frequencies based on the psychoacoustic principles of human hearing. The human ear exhibits varying sensitivities to different frequencies; for example, it is most sensitive to the mid-frequency range of 2-4kHz, while sensitivity to high-frequency ranges above 16kHz decreases significantly. The critical frequency band simulates this difference in sensitivity, dividing the 20Hz-20kHz audio frequency range into multiple intervals with different perceptual priorities.

[0048] In some specific embodiments, 24 critical frequency bands are established based on the human auditory critical frequency band theory and the Bark psychoacoustic scale. The input audio signal is divided into three frequency bands according to perceptual importance: a low-frequency region, a mid-frequency region, and a high-frequency region. Each frequency band corresponds to a signal processing scheme. According to the Bark scale, the 20Hz~20kHz frequency range of the input audio is decomposed into 24 basic critical frequency bands that perfectly match the human ear's perceptual units. This provides a scientific minimum unit for subsequent merging according to perceptual importance, avoiding the problem of traditional algorithms dividing by fixed frequency intervals being out of touch with human auditory characteristics. The 24 critical frequency bands are merged into three macroscopic frequency band regions, based on the contribution of each critical frequency band to the perceived reverberation.

[0049] Step 102 involves targeted processing logic after frequency band division. The core is a frequency band adaptation scheme plus two-dimensional dynamic parameter tuning, which is a further implementation of the frequency band division of Feature 1. The preset signal processing scheme is a pre-designed processing strategy that matches complexity and sound quality for different perceptual priority frequency bands divided by Feature 1.

[0050] In this application, each frequency band employs a scheme adapted to its own priority, rather than sharing a single scheme across all bands. In some specific embodiments, the signal processing scheme for the low-frequency region includes high-precision temporal convolution processing to preserve spatial positioning and phase information; the signal processing scheme for the mid-frequency region includes a hybrid processing mode of convolution and synthesis; and the signal processing scheme for the high-frequency region includes intelligent synthesis based on a particle system model using acoustic physics principles. This differentiated processing scheme ensures an optimal balance between sound quality and computational power for each frequency band. High-priority frequency bands use high-precision schemes to guarantee listening quality, while low-priority frequency bands use low-complexity schemes to reduce computational power, avoiding the problem of high-priority frequency bands sacrificing sound quality due to simple schemes and low-priority frequency bands wasting computational power due to complex schemes.

[0051] The high-priority low-frequency band (0-800Hz) employs a pre-set high-precision temporal convolution scheme, which fully preserves phase information (phase directly affects spatial positioning), ensuring the realism of the low-frequency sound. The mid-priority mid-frequency band (800Hz-4kHz) uses a pre-set hybrid convolution and synthesis processing scheme, retaining some of the sound quality advantages of convolution while reducing computational complexity through synthesis. The low-priority high-frequency band (above 4kHz) uses a pre-set acoustic physical model synthesis scheme, generating high-frequency reverberation by simulating acoustic reflection patterns, significantly reducing computational load. It should be noted that the boundary points of the three frequency bands are not fixed but dynamically adjustable. The boundary between the mid-frequency and low-frequency bands is at 800Hz ± 25Hz, and the boundary between the mid-frequency and high-frequency bands is at 4kHz ± 100Hz.

[0052] Spectral characteristics refer to the frequency distribution of the input audio itself; computational parameters are adjustable technical parameters in differentiated processing schemes, such as the impulse response length of time-domain convolution, the particle density of the particle system, and the FFT block size of frequency-domain processing. For spectral characteristic adjustment: if the input audio has a high proportion of high-frequency signals, the particle density of high-frequency synthesis is increased to enhance high-frequency details; if the low-frequency signals have a high proportion, the impulse response length of low-frequency convolution is increased to enhance spatial sense. For system load adjustment: if the CPU load is high, the FFT block size is reduced and the particle density is lowered; if the load is low, the FFT block size is increased and the particle density is increased to improve sound quality. Through dual-dimensional dynamic parameter tuning of spectral characteristic adjustment and system load adjustment, a dual adaptation between signal characteristics and system state is achieved. This avoids both insufficient adaptation of fixed parameters to different types of audio and performance instability of fixed parameters under fluctuating system load, ensuring a balance between sound quality and real-time performance under different audio types and different device loads.

[0053] Step 103 involves efficiency optimization logic for frequency domain processing. Based on multi-level caching adapted to block convolution, it reduces computational load through block division and improves access efficiency through multi-level caching, thus solving the drawback of traditional frequency domain convolution on mobile devices, which cannot achieve real-time processing of long-impulse responses due to memory access issues. Multi-level caching is a hierarchical storage structure relative to traditional single-level caching. The function and data type of each level of caching are strictly divided and adapted to the processing needs of different frequency band signals.

[0054] Traditional frequency-domain convolution performs an FFT operation on the complete input signal and the complete impulse response. Traditional FFT-based convolution faces challenges when handling extremely long impulse responses, including discontinuous memory access patterns, low cache hit rates, and poor data locality. This is particularly problematic on mobile devices with limited memory bandwidth, where the theoretical advantages of frequency domain transformation are difficult to translate into practical performance improvements. When the impulse response is too long (e.g., >2 seconds), the computational cost of a single FFT is extremely high. Block convolution breaks down the long impulse response into multiple smaller impulse response blocks, and simultaneously breaks down the input audio into corresponding input signal blocks, performing frequency-domain convolution on each block (i.e., convolution between blocks), significantly reducing the computational cost per operation. Furthermore, a multi-level caching architecture optimizes data access for block convolution, storing data blocks of different frequency bands in appropriate cache levels, avoiding memory access latency caused by frequent data switching in block convolution.

[0055] In some specific embodiments, the multi-level caching architecture includes a first cache layer, a second cache layer, and a third cache layer. The first cache layer processes real-time data, the second cache layer processes high-frequency multiplexed data, and the third cache layer predicts pre-loaded data for the next processing cycle, enabling frequency-domain block convolution processing of n frequency band signals. The first cache layer is the fastest and has the smallest capacity. It stores data currently being processed or about to be processed. The second cache layer is the next fastest and has a medium capacity. It stores frequently used data to avoid frequent retrievals from slow main memory. The third cache layer is the slowest and has the largest capacity. It stores data that may be used in the near future and attempts to predict what the next processing stage will need. These three cache layers do not exist in isolation; they work together to serve the core operation of frequency-domain block convolution, translating the theoretical advantages of block convolution into actual performance improvements and reducing overall processing latency.

[0056] Furthermore, the first cache level stores the FFT transform result of the current processing block and uses the SIMD instruction set to optimize complex number operations for processing real-time data. The FFT result is a complex number, which is the core real-time computational data in frequency domain convolution. SIMD (Single Instruction Multiple Data) is a parallel computing technique that allows a single instruction to perform the same operation on multiple data simultaneously. Placing the FFT result in the fastest L1 cache and using the SIMD instruction set to perform complex number multiplication on it maximizes the core computational performance of frequency domain convolution. The second cache level caches the impulse response frequency domain data in the mid-frequency band based on the LRU strategy to handle high-frequency multiplexed data. The mid-frequency band (800Hz~4kHz) is the region most sensitive to the human ear, and its impulse response is usually long and complex, making it the most likely to be reused in block convolution. Placing it in the L2 cache maximizes reuse efficiency. The third cache level preloads the computational data for the next processing cycle in the low-frequency band using an access pattern prediction algorithm. Low-frequency band (0~800Hz) processing employs high-precision temporal convolution, which involves large datasets and lengthy processing times. Preloading low-frequency band data allows for thorough preparation for the next processing cycle, preventing overall latency caused by excessive low-frequency processing time. Deeply integrating underlying optimization techniques such as SIMD, LRU, and access pattern prediction with upper-layer perception-driven frequency band processing logic forms a complete system of hardware and software co-optimization. Each cache layer serves the frequency band data it is best suited for, maximizing efficiency.

[0057] Existing algorithms employ a uniform high-precision processing strategy across all frequency bands, failing to fully utilize the frequency masking effect and critical band characteristics of the human auditory system. This results in a waste of computational resources on frequency bands with low perceptual contribution, impacting overall processing efficiency. Step 104 is the integrated output logic after multi-band processing. Its core principles are perceptual optimization mixing and signal integrity assurance. This solves the problems of unnatural frequency mixing, phase discontinuity, and output distortion that can occur with segmented frequency processing, and is crucial for ensuring final sound quality. Masking effect mixing makes the audio more consistent with human perception, avoiding auditory conflicts between frequency bands. Phase continuity assurance ensures natural transitions between frequency bands without any discontinuity. Dynamic range control avoids output distortion and adapts to different devices.

[0058] In some specific embodiments, the acquisition of reverberant audio includes: dynamically adjusting the mixing weights of the low-frequency convolution result and the high-frequency synthesized signal using psychoacoustic principles of simultaneous masking and temporal masking; ensuring phase continuity and spatial consistency of different frequency band components at the synthesis point through a phase coherence preservation algorithm; and intelligently adjusting the gain distribution based on the audio characteristics of the input audio and the audio characteristics of the output device through an adaptive dynamic range control mechanism to effectively prevent clipping distortion and dynamic range compression. The system analyzes the energy distribution of the low-frequency convolution signal and the high-frequency synthesized signal in real time. When a strong signal is detected in the low-frequency band, the mixing weight of the high-frequency band is temporarily reduced because the human ear cannot hear the high-frequency details masked by the low frequencies at this time. Conversely, when the low-frequency energy is weak, it automatically increases the weight of the high-frequency band to highlight spatial details and a sense of airiness. The mixing weights are not fixed but are a function that changes continuously with time and frequency, perfectly simulating the human ear's perception process.

[0059] The masking effect of human hearing is a fundamental principle of psychoacoustics—the human ear's sensitivity to weak sounds near loud sounds is significantly reduced. Mixing involves superimposing the n frequency band signals optimized by feature 3 into a single audio stream. Mixing based on the masking effect means that the algorithm first analyzes the signal strength of each frequency band, appropriately reducing the weight of weak frequency bands masked by loud sounds, while maintaining the weight of unmasked frequency bands. For example, if the signal strength of the low-frequency band is much higher than that of the mid-frequency band, the weight of the mid-frequency band can be appropriately reduced to avoid the weak mid-frequency sounds still consuming computational power after being masked by strong low-frequency sounds, while simultaneously making the mixed sound more consistent with human auditory perception.

[0060] Because different processing schemes are used for different frequency bands, the phase information of each frequency band may differ after processing. For example, temporal convolution preserves the original phase, while the phase of the synthesized signal is generated analogically. Phase discontinuity can cause a sense of discontinuity in the audio. To ensure phase continuity, a phase coherence preservation algorithm is used to adjust the phase parameters of each frequency band, making the phases of adjacent frequency bands consistent at the crossover point, thus ensuring a natural sound transition.

[0061] Dynamic range is the difference between the maximum peak value and the minimum valley value of an audio signal. Dynamic range control is used to avoid problems caused by an excessively large dynamic range: if the peak value is too high, it will exceed the maximum tolerance of the output device, resulting in clipping distortion; if the valley value is too low, it will be masked by noise, resulting in loss of detail. The dynamic range control mechanism monitors the peak and valley values ​​of the input signal in real time, and intelligently adjusts the gain in combination with the characteristics of the output device to ensure that the dynamic range of the output audio is adapted to the device, without distortion and with clear details.

[0062] Time-domain and frequency-domain convolution requires storing and processing massive amounts of impulse response data, resulting in high computational complexity. Furthermore, the high-frequency impulse response data exhibits significant redundancy, leading to substantial waste of computational power and memory. This application employs a particle system based on acoustic physics principles. In some specific embodiments, within this acoustic physics-based particle system, each acoustic particle represents an independent reflected sound path. Particle state parameters include three-dimensional spatial position, velocity vector, frequency-dependent attenuation coefficient, propagation delay time, and reflection count. Through the random distribution, Brownian motion, and boundary reflection behavior of the particles, the spatial diffusion characteristics and temporal attenuation laws of high-frequency reverberation are accurately simulated. Compared to convolution, the computational cost of the particle system is proportional to the number of particles, rather than the impulse response length. Through physical behavior simulation, the generated high-frequency reverberation possesses a natural randomness and spatial quality.

[0063] The particle system transforms the complex high-frequency reverberation process from the traditional approach of recording the sum of all reflected sounds to simulating the physical processes that generate these reflected sounds. Each acoustic particle represents a beam of sound photons originating from the sound source, traveling through the room, reflecting, and finally reaching the ear. By simulating the behavior of each sound photon, the final result is calculated in real time. To allow the particles to move and attenuate like real sound, each particle must be assigned a series of precise state parameters. These parameters collectively determine the sonic characteristics of the reflected sound represented by the particle. Three-dimensional spatial position: the particle's real-time coordinates in the virtual room. Velocity vector: the particle's direction and velocity. Frequency-dependent attenuation coefficient: simulates the physical characteristic that high-frequency sound attenuates faster when propagating through the air. Different frequencies of sound have different attenuation coefficients, with higher frequencies having larger coefficients. Propagation delay time: the time it takes for the particle to travel from the sound source to its current position. This directly determines the timing of the reflected sound reaching the listener's ear and is the basis for the perception of reverberation and echo. Reflection count: reflects how many times the particle has reflected off boundaries such as the ground. This is related to the attenuation law; generally, the more reflections, the weaker the sound energy.

[0064] The particle system is not static; it is a dynamic, adaptive system. The system determines the number of particles generated and their initial states based on scene requirements. Scene requirements include: Room geometry parameters: the size and shape of the room. Larger rooms require more particles to simulate more complex reflection paths. Wall material properties: reflecting whether the walls are smooth or rough. Smooth materials reflect strongly, resulting in a lower particle attenuation coefficient; rough materials absorb more, resulting in a higher particle attenuation coefficient. Target reverberation characteristics: the reverberation quality desired by the user.

[0065] By simulating three core particle behaviors, this system recreates two key characteristics of high-frequency reverberation in the real world: Random distribution: Simulates the initial randomness of sound propagation in all directions after originating from the sound source. Brownian motion: Simulates the subtle influence of minute airflow disturbances on the propagation path of high-frequency sound, making reflected sound sound more natural. Boundary reflection: Simulates the reflection of sound on obstacles such as walls. When a particle collides with a boundary, the system calculates a new reflection angle based on the incident angle and the boundary material, and updates its attenuation coefficient according to the material properties. Through the combination of these three behaviors, the particle system can accurately simulate spatial diffusion characteristics and temporal attenuation patterns.

[0066] In some specific embodiments, a machine learning-based adaptive computing power adjustment module adjusts the computational parameters for differentiated processing. These parameters include hardware parameters, system status, and application scenarios. This module monitors the overall system's condition and external environment, making the most intelligent decisions and dynamically allocating computing resources. It uses a large number of different hardware devices, system loads, and application scenarios as input, recording the optimal balance between sound quality and real-time performance under these conditions. This data is then used to train a model, such as a decision tree or neural network. The module collects real-time information on the device's hardware parameters, system status, and application scenarios, inputting this information into the trained model. The model quickly considers and outputs a set of optimal computational parameters to guide subsequent differentiated processing. When the system load suddenly increases, the module reacts rapidly, reducing the computational complexity of non-critical parameters and prioritizing real-time performance; when the load decreases, it automatically reverts to high-quality mode, achieving a smooth experience under load fluctuations.

[0067] Preferably, the adaptive adjustment process of computing power can be modeled as a multi-objective optimization problem. Hardware features, system state, and application context (such as audio type, user preferences, noise, etc.) are abstracted into mathematical feature expressions (one-hot encoding, data normalization representation, etc. can be used). A backbone network with a fully connected + LSTM design is used, combined with an output layer designed with SoftMax as a machine learning network model to solve the multi-objective optimization problem.

[0068] The hardware parameters include CPU architecture type, number of physical cores, cache hierarchy, memory bandwidth capacity, and floating-point operation capability; system status includes real-time CPU load, core temperature, battery level, network connection status, and background process contention; and application scenarios include audio content type, user preference settings, environmental noise level, and latency sensitivity requirements.

[0069] CPU architecture types include ARM's Cortex-A series and x86's Intel Core series. Different architectures have different instruction sets and pipeline designs, and their optimization methods for specific algorithms also differ.

[0070] Physical core count: The more cores, the stronger the parallel processing capability.

[0071] Cache hierarchy: The size and speed of L1 / L2 / L3 caches directly affect memory access efficiency.

[0072] Memory bandwidth capacity: Memory read and write speed is the bottleneck of data throughput.

[0073] Real-time CPU load: The percentage of CPU currently being used, the most direct performance indicator.

[0074] Core temperature: Overheating of the CPU will trigger frequency throttling protection, resulting in performance degradation. Monitoring the temperature can help predict performance bottlenecks in advance.

[0075] Battery level: On mobile devices, battery life is a critical constraint. When the battery is low, some performance may need to be sacrificed to extend battery life.

[0076] Network connectivity status: network latency and stability.

[0077] Background process competition: The competition for CPU and memory resources among other running apps.

[0078] Floating-point performance (FLOPS): The speed at which the CPU processes decimals (the basic unit of audio signals) is a core indicator of audio processing performance.

[0079] Audio content type: Different types of audio have different reverb requirements.

[0080] User preference settings: The type of reverb the user might prefer.

[0081] Ambient noise level: On noisy streets, a stronger reverberation effect and dynamic range are needed to highlight the sound; in quiet indoor environments, a more delicate and natural effect can be achieved.

[0082] Delay sensitivity requirements: Requirements regarding latency.

[0083] This application also proposes an intelligent perception-based adaptive real-time reverberation processing system to implement any of the above-mentioned intelligent perception-based adaptive real-time reverberation processing methods, with modules as follows: Figure 2 As shown, the adaptive real-time reverberation processing system includes:

[0084] Frequency band processing module 1 is used to divide the input audio into n frequency band regions according to perceptual importance based on multiple preset critical frequency bands, and dynamically adjust the frequency division point position between frequency band regions based on the real-time load status of the system; and to perform differentiated processing on the signals of each frequency band based on a preset signal processing scheme.

[0085] The computing power adaptive adjustment module 2 is used to dynamically adjust the computing parameters of differentiated processing and output n frequency band signals;

[0086] Frequency domain block convolution module 3 is used to perform frequency domain block convolution processing on n frequency band signals based on a pre-built multi-level caching architecture.

[0087] Output synthesis module 4 is used to mix the convolution results of n frequency band signals based on the human auditory masking effect, and output the final reverberant audio through a dynamic range control mechanism while ensuring phase continuity between frequency bands.

[0088] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform an intelligent perception-based adaptive real-time reverberation processing method. Applying an intelligent perception-based adaptive real-time reverberation processing method to a computer program product facilitates execution.

[0089] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an adaptive real-time reverberation processing method based on intelligent perception as described above.

[0090] The computer storage medium of this application can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. This application applies an intelligent perception-based adaptive real-time reverberation processing method to a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the method steps provided in this application, which is simple, fast, easy to store, and not easily lost.

[0091] This application proposes an adaptive real-time reverberation processing method and system based on intelligent perception. It addresses the shortcomings of existing reverberation processing algorithms by dividing the input audio into n frequency bands based on perceptual importance using multiple preset critical frequency bands and dynamically adjusting the sub-band positions. This fully adapts to the characteristics of human hearing, avoiding the waste of computational resources caused by traditional uniform frequency band processing. Simultaneously, it combines real-time system load status to achieve flexible adaptation of frequency band processing strategies, ensuring processing quality for perceptually important frequency bands while reducing the computational load on low-perceptual-contribution frequency bands. By performing differentiated processing on signals of each frequency band based on a preset signal processing scheme and dynamically adjusting the computational parameters in the differentiated processing, it effectively adapts to the hardware characteristics of different devices and real-time system status, solving the problem of inconsistent performance of traditional fixed processing strategies in heterogeneous environments and achieving precise matching of computational resources and processing requirements. Furthermore, based on a pre-built multi-level caching architecture, the method further adapts to the n frequency bands. Frequency-domain block convolution processing of frequency band signals can significantly improve the memory access defects of traditional frequency-domain convolution, enhance cache hit rate and data locality. Especially in scenarios with limited memory bandwidth, such as mobile devices, it can fully unleash the performance advantages of frequency-domain processing and alleviate the contradiction between computational complexity and real-time performance. In addition, by mixing the frequency band signals after convolution based on the human auditory masking effect and combining it with a dynamic range control mechanism to ensure phase continuity between frequency bands, computational efficiency can be further optimized while ensuring the naturalness of the final output reverberant audio. This effectively prevents clipping distortion and dynamic range compression. Ultimately, in environments with limited computing resources, such as mobile devices and embedded systems, a dynamic balance between computational complexity and audio perception quality is achieved. This promotes the widespread deployment and industrial application of professional-grade reverberation algorithms on various consumer terminals, meeting users' needs for high-quality reverberation effects while ensuring the stability and real-time performance of device operation.

[0092] Those skilled in the art will understand that the modules described above can be implemented using general-purpose computing systems. They can be centralized on a single computing system or distributed across a network of multiple computing systems. Optionally, they can be implemented using computer-executable program code, allowing them to be stored in a storage system for execution by the computing system. Alternatively, they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0093] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

[0094] The above disclosures are only a few specific implementation scenarios of this application. However, this application is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for adaptive real-time reverberation processing based on intelligent sensing, characterized in that, The application comprises the following steps: Dividing the input audio into n frequency band regions according to the preset critical frequency bands based on perceptual importance, and dynamically adjusting the frequency division point position between the frequency band regions based on the system real-time load state; Processing each frequency band signal based on the preset signal processing scheme, and dynamically adjusting the operation parameters in the differential processing according to the spectral characteristics of each frequency band region and the system real-time load state, and outputting n frequency band signals; Performing frequency domain block convolution processing on the n frequency band signals based on the pre-constructed multi-level cache architecture; Mixing the convolution results of the n frequency band signals according to the human ear hearing masking effect, and outputting the final reverberation audio through a dynamic range control mechanism while ensuring the phase continuity between the frequency bands.

2. The adaptive reverberation processing method of claim 1, wherein, Based on the human ear hearing critical frequency band theory and Bark psychoacoustic scale, 24 critical frequency bands are established, and the input audio signal is divided into three frequency band regions, i.e. low frequency band region, middle frequency band region and high frequency band region, according to the perceptual importance, and each frequency band region corresponds to a signal processing scheme.

3. The adaptive reverberation processing method of claim 2, wherein, The signal processing scheme of the low frequency band region includes high-precision time domain convolution processing to retain spatial positioning and phase information. The signal processing scheme of the middle frequency band region includes a convolution and synthesis mixed processing mode. The signal processing scheme of the high frequency band region includes an intelligent synthesis based on the particle system model of acoustic physics principle.

4. The adaptive reverberation processing method of claim 3, wherein, In the particle system based on the acoustic physics principle, each acoustic particle represents an independent reflected sound path, and the particle state parameters include three-dimensional space position, velocity vector, frequency-dependent attenuation coefficient, propagation delay time and reflection count; through the random distribution, Brown motion and boundary reflection behavior of the particles, the spatial diffusion characteristics and time domain attenuation law of high frequency reverberation are accurately simulated.

5. The adaptive reverberation processing method of claim 1, wherein, The multi-level cache architecture includes a first cache level, a second cache level and a third cache level. The first cache level processes real-time data, the second cache level processes high frequency multiplexing data, and the third cache level predicts the preloaded data of the next processing period to perform frequency domain block convolution processing on the n frequency band signals.

6. The adaptive reverberation processing method of claim 5, wherein, The first cache level stores the FFT transformation results of the current processing block and optimizes complex operations using SIMD instruction set to process real-time data; the second cache level caches the impulse response frequency domain data of the middle frequency band based on LRU strategy to process high frequency multiplexing data; the third cache level preloads the operation data of the next processing period of the low frequency band through the access mode prediction algorithm.

7. The adaptive reverberation processing method of claim 1, wherein, The acquisition of the reverberation audio specifically comprises: Dynamically adjusting the mixing weight of the low frequency convolution result and the high frequency synthesis signal by using the psychoacoustic principle of simultaneous masking and temporal masking; ensuring the phase continuity and spatial consistency of different frequency band components at the synthesis point through a phase coherence preservation algorithm; Through an adaptive dynamic range control mechanism, intelligently adjusting the gain distribution according to the audio characteristics of the input audio and the audio characteristics of the output device to effectively prevent clipping distortion and dynamic range compression.

8. The adaptive reverberation processing method of claim 1, wherein, Adjusting the operation parameters of the differential processing through a machine learning-based computing power adaptive adjustment module, wherein the operation parameters include hardware parameters, system state and application scenario.

9. The adaptive reverberation processing method of claim 8, wherein, The hardware parameters include CPU architecture type, physical core number, cache hierarchy, memory bandwidth capacity, and floating-point operation capability. The system state includes real-time CPU load, core temperature, battery power level, network connection state, and background process competition. The application scenario includes audio content type, user preference setting, environment noise level, and delay sensitivity requirement.

10. An intelligent sensing based adaptive real-time reverberation processing system characterized in that, The adaptive real-time reverb processing system for implementing the method of any one of claims 1-9 comprises: A frequency band processing module, configured to divide input audio into n frequency band regions according to perceptual importance based on a plurality of preset critical frequency bands, and dynamically adjust the frequency division point position between the frequency band regions based on the real-time load state of the system; and perform differential processing on the frequency band signals based on a preset signal processing scheme. An algorithm power adaptive adjustment module, configured to dynamically adjust the operation parameters of the differential processing, and output n frequency band signals. A frequency domain block convolution module, configured to perform frequency domain block convolution processing on the n frequency band signals based on a pre-constructed multi-level cache architecture. An output synthesis module, configured to mix the convolution results of the n frequency band signals according to the human ear auditory masking effect, and output the final reverb audio through a dynamic range control mechanism while ensuring the phase continuity between the frequency bands.

Images