Self-adaptive real-time reverberation processing method and system based on intelligent perception

Abstract

The invention provides a self-adaptive real-time reverberation processing method and system based on intelligent perception, and the method comprises the steps: dividing an input audio into n frequency band regions according to the perception importance, and dynamically adjusting the position of a frequency division point; differential processing is carried out on the signals of all the frequency bands, and operation parameters in differential processing are dynamically adjusted in combination with the real-time load of the system; performing frequency domain block convolution processing on the n frequency band signals based on a pre-constructed multi-level cache architecture; the convolution results of the n frequency band signals are mixed according to the auditory masking effect of human ears, and the final reverberation audio is output through a dynamic range control mechanism while the phase continuity between the frequency bands is guaranteed. According to the invention, dynamic balance between professional tone quality and high real-time performance in a computing resource limited scene is successfully realized in the field of real-time reverberation processing, the real-time response speed and multi-device heterogeneous adaptation capability of audio reverberation processing are remarkably improved on the basis of guaranteeing the reverberation tone quality, and invalid consumption of computing resources is reduced to the greatest extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio reverberation, in particular, to a method and system for adaptive real-time reverberation processing based on intelligent perception. BACKGROUND

[0002] Reverberation processing technology is one of the core technologies in the field of audio signal processing, and is widely used in audio enhancement scenarios of consumer-level devices such as mobile terminals and embedded systems. The core goal is to simulate the reflection characteristics of acoustic space to improve the spatial sense and naturalness of the listening experience of audio signals. However, existing reverberation processing algorithms face many technical bottlenecks in actual application, making it difficult to meet professional-level processing needs in computing resource-limited scenarios.

[0003] The computational complexity of traditional convolution reverberation algorithms increases exponentially with the sampling rate. In high sampling rate scenarios of 48kHz and above, the real-time processing needs for long impulse responses are in conflict with the limited CPU computing power and memory bandwidth of consumer-level devices, making it difficult to popularize high-quality reverberation effects on mobile devices such as mobile phones and smart speakers. At the same time, existing algorithms generally use a fixed processing strategy and cannot perceive differences in device hardware characteristics and changes in real-time system state, resulting in significant performance inconsistency in heterogeneous computing environments. This not only fails to fully exploit the hardware potential of high-performance devices, but also easily leads to problems such as stuttering and delay on low-power devices.

[0004] Therefore, there is a need for a reverberation processing scheme to solve the above problems. SUMMARY

[0005] Based on the problems existing in the prior art, the present application provides a method and system for adaptive real-time reverberation processing based on intelligent perception. The specific scheme is as follows: A method for adaptive real-time reverberation processing based on intelligent perception, comprising: dividing the input audio into n frequency band regions according to the perceptual importance based on a plurality of preset critical frequency bands, and dynamically adjusting the frequency division point positions between the frequency band regions based on the real-time load state of the system; differentially processing each frequency band signal based on a preset signal processing scheme, and dynamically adjusting the operation parameters in the differential processing according to the spectral characteristics of each frequency band region and the real-time load state of the system, and outputting n frequency band signals; performing frequency domain block convolution processing on the n frequency band signals based on a pre-constructed multi-level cache architecture; mixing the convolution results of the n frequency band signals according to the human ear hearing masking effect, and outputting the final reverberation audio through a dynamic range control mechanism while ensuring the phase continuity between the frequency bands.

[0006] In some embodiments, 24 critical frequency bands are established based on the human ear auditory critical band theory and Bark psychoacoustic scale, and the input audio signal is divided into three frequency band regions, i.e., a low frequency band region, a middle frequency band region, and a high frequency band region, according to the perceptual importance, and each frequency band region corresponds to a signal processing scheme.

[0007] In some embodiments, the signal processing scheme of the low frequency band region includes high-precision time domain convolution processing to retain spatial positioning and phase information. The signal processing scheme of the middle frequency band region includes a convolution and synthesis hybrid processing mode. The signal processing scheme of the high frequency band region includes an intelligent synthesis based on a particle system model of acoustic physics.

[0008] In some embodiments, in the particle system based on acoustic physics, each acoustic particle represents an independent reflected sound path, and the particle state parameters include three-dimensional spatial position, velocity vector, frequency-dependent attenuation coefficient, propagation delay time, and reflection count; the spatial diffusion characteristics and time-domain attenuation law of high-frequency reverberation are accurately simulated through the random distribution, Brownian motion, and boundary reflection behavior of the particles.

[0009] In some embodiments, the multi-level cache architecture includes a first cache level, a second cache level, and a third cache level. The first cache level processes real-time data, the second cache level processes high-frequency multiplexing data, and the third cache level predicts preloaded data for the next processing period to perform frequency domain block convolution processing on n frequency band signals.

[0010] In some embodiments, the first cache level stores the FFT transformation results of the current processing block and uses the SIMD instruction set to optimize complex number operations to process real-time data; the second cache level caches the impulse response frequency domain data of the middle frequency band based on the LRU strategy to process high-frequency multiplexing data; and the third cache level preloads the operation data of the next processing period of the low frequency band through an access pattern prediction algorithm.

[0011] In some embodiments, the reverberation audio acquisition specifically includes: The mixing weights of the low frequency convolution result and the high frequency synthesis signal are dynamically adjusted using the psychoacoustic principles of simultaneous masking and temporal masking; and the phase continuity and spatial consistency of different frequency band components at the synthesis point are ensured through a phase coherence preservation algorithm. Through an adaptive dynamic range control mechanism, the gain distribution is intelligently adjusted according to the audio characteristics of the input audio and the audio characteristics of the output device, effectively preventing clipping distortion and dynamic range compression.

[0012] In some embodiments, the operation parameters of the differential processing are adjusted by a machine learning-based computing power adaptive adjustment module, and the operation parameters include hardware parameters, system states, and application scenarios.

[0013] In some embodiments, the hardware parameters include CPU architecture type, physical core number, cache level structure, memory bandwidth capacity, and floating point operation capability. The system states include real-time CPU load, core temperature, battery power level, network connection state, and background process competition. The application scenarios include audio content type, user preference setting, environmental noise level, and delay sensitivity requirement.

[0014] An adaptive real-time reverberation processing system based on intelligent sensing is used to implement the adaptive real-time reverberation processing method based on intelligent sensing described in any of the above, and the adaptive real-time reverberation processing system comprises: A frequency band processing module is used to divide the input audio into n frequency band regions according to the perceptual importance based on a plurality of preset critical frequency bands, and dynamically adjust the frequency division point positions between the frequency band regions based on the real-time load state of the system; and the signals of each frequency band are differentially processed based on a preset signal processing scheme. A computing power adaptive adjustment module is used to dynamically adjust the operation parameters of the differential processing, and output n frequency band signals. A frequency domain block convolution module is used to perform frequency domain block convolution processing on the n frequency band signals based on a pre-constructed multi-level cache architecture. An output synthesis module is used to mix the convolution results of the n frequency band signals according to the human ear hearing masking effect, and output the final reverberation audio through a dynamic range control mechanism while ensuring the phase continuity between the frequency bands.

[0015] Beneficial effects: The application proposes an adaptive real-time reverberation processing method and system based on intelligent sensing, which breaks through the technical bottleneck that the traditional reverberation algorithm cannot simultaneously achieve high computing power and high sound quality through the deep collaborative design of three parts of perception-driven optimization, hardware adaptation reinforcement, and resource efficiency improvement, successfully realizes the dynamic balance between professional sound quality and high real-time performance in the computing resource limited scene in the field of real-time reverberation processing, significantly improves the real-time response speed of audio reverberation processing and the multi-device heterogeneous adaptation capability on the basis of ensuring the reverberation sound quality, and maximally reduces the invalid consumption of computing resources, which not only meets the core needs of users for immersive and high-fidelity reverberation listening, but also effectively guarantees the long-term stability of device operation and the low-delay real-time performance of audio processing.

[0016] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Figure 1 is a flowchart of the adaptive real-time reverberation processing method of the present application; Figure 2 is a schematic diagram of the principle of the adaptive real-time reverberation processing method of the present application; Figure 3 is a schematic diagram of the adaptive real-time reverberation processing system module of the present application.

[0019] Reference signs: 1 - frequency band processing module; 2 - computing power adaptive adjustment module; 3 - frequency domain block convolution module; 4 - output synthesis module. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] The present application proposes an adaptive real-time reverberation processing method based on intelligent sensing, which realizes the dynamic balance between professional sound quality and high real-time performance in a computing resource limited scenario. The flowchart of the method is shown in FIG. 1, the principle is shown in FIG. 2, and the specific scheme is as follows: Figure 1 Figure 2 The present application proposes an adaptive real-time reverberation processing method based on intelligent sensing, which realizes the dynamic balance between professional sound quality and high real-time performance in a computing resource limited scenario. The flowchart of the method is shown in FIG. 1, the principle is shown in FIG. 2, and the specific scheme is as follows: An adaptive real-time reverberation processing method based on intelligent sensing, comprising: 101. Dividing the input audio into n frequency band regions according to the perceptual importance based on a plurality of preset critical frequency bands, and dynamically adjusting the frequency division point position between the frequency band regions based on the system real-time load state; 102. Differentiating the processing of each frequency band signal based on a preset signal processing scheme, and dynamically adjusting the operation parameters in the differentiated processing according to the spectral characteristics of each frequency band region and the system real-time load state, and outputting n frequency band signals; 103. Performing frequency domain block convolution processing on the n frequency band signals based on a pre-constructed multi-level cache architecture; ​104. The convolution results of the n frequency band signals according to the human ear hearing masking effect are mixed, and the final reverberation audio is output through a dynamic range control mechanism while ensuring the phase continuity between the frequency bands.

[0022] Step 101 is the basis of the entire method, which involves frequency band division according to human ear perception characteristics and dynamic adaptation according to system state. The frequency band is divided according to the perception importance, and the frequency band is divided according to the perception importance, which avoids the waste of algorithm power of traditional algorithms for unified high-precision processing of all frequency bands. Dynamic adjustment of frequency division point realizes the adaptation of system state and frequency band division, and through the adjustment of frequency division point, the real-time performance is preferentially guaranteed under high load, and the sound quality is preferentially guaranteed under low load, which avoids the problem of fixed frequency division under high load and idle algorithm under low load.

[0023] Among them, the n frequency bands are not n equal-width frequency bands divided uniformly, but priority intervals obtained by merging or splitting based on the perception contribution of critical bands, for example, the 0-800Hz low frequency band which plays a key role in spatial positioning (human ears are sensitive to phase changes, which directly affects the judgment of the direction from which the sound comes) is divided into a high-perception priority frequency band, and the high frequency band above 4kHz, which is less sensitive to human ears, is divided into a low-perception priority frequency band. The specific value of n is dynamically determined by the algorithm according to the merging rule of critical bands (not a fixed value), the core is to ensure that the frequency bands with high perception importance are independently divided, and the frequency bands with low perception importance are merged.

[0024] The critical band is not a frequency interval divided at will, but a preset sensitive interval of human ears to sound frequency based on the principle of human auditory psychoacoustics. Human ears have different sensitivities to different frequencies, such as being most sensitive to 2-4kHz mid-frequency band and significantly reducing sensitivity to high-frequency band above 16kHz. The critical band is to simulate this difference in sensitivity, dividing the 20Hz-20kHz audio frequency range into multiple intervals with different perception priorities.

[0025] In some specific embodiments, 24 critical bands are established based on the human auditory critical band theory and Bark psychoacoustic scale, and the input audio signal is divided into three frequency band regions, i.e. low frequency band region, mid frequency band region and high frequency band region, according to the perception importance, each frequency band region corresponds to a signal processing scheme. According to the Bark scale, the 20Hz-20kHz frequency range of the input audio is divided into 24 basic critical bands that completely match the human perception units, providing a scientific minimum unit for subsequent perception importance-based merging, avoiding the problem that traditional algorithms are out of touch with human auditory characteristics when dividing by fixed frequency intervals. The 24 critical bands are merged into 3 macro frequency band regions, and the merging is based on the contribution of each critical band to the reverberation perception.

[0026] Step 102 involves targeted processing logic after frequency band division. The core is a frequency band adaptation scheme plus two-dimensional dynamic parameter tuning, which is a further implementation of the frequency band division of Feature 1. The preset signal processing scheme is a pre-designed processing strategy that matches complexity and sound quality for different perceptual priority frequency bands divided by Feature 1.

[0027] In this application, each frequency band employs a scheme adapted to its own priority, rather than sharing a single scheme across all bands. In some specific embodiments, the signal processing scheme for the low-frequency region includes high-precision temporal convolution processing to preserve spatial positioning and phase information; the signal processing scheme for the mid-frequency region includes a hybrid processing mode of convolution and synthesis; and the signal processing scheme for the high-frequency region includes intelligent synthesis based on a particle system model using acoustic physics principles. This differentiated processing scheme ensures an optimal balance between sound quality and computational power for each frequency band. High-priority frequency bands use high-precision schemes to guarantee listening quality, while low-priority frequency bands use low-complexity schemes to reduce computational power, avoiding the problem of high-priority frequency bands sacrificing sound quality due to simple schemes and low-priority frequency bands wasting computational power due to complex schemes.

[0028] The high-priority low-frequency band (0-800Hz) employs a pre-set high-precision temporal convolution scheme, which fully preserves phase information (phase directly affects spatial positioning), ensuring the realism of the low-frequency sound. The mid-priority mid-frequency band (800Hz-4kHz) uses a pre-set hybrid convolution and synthesis processing scheme, retaining some of the sound quality advantages of convolution while reducing computational complexity through synthesis. The low-priority high-frequency band (above 4kHz) uses a pre-set acoustic physical model synthesis scheme, generating high-frequency reverberation by simulating acoustic reflection patterns, significantly reducing computational load. It should be noted that the boundary points of the three frequency bands are not fixed but dynamically adjustable. The boundary between the mid-frequency and low-frequency bands is at 800Hz ± 25Hz, and the boundary between the mid-frequency and high-frequency bands is at 4kHz ± 100Hz.

[0029] Spectral characteristics refer to the frequency distribution of the input audio itself; computational parameters are adjustable technical parameters in differentiated processing schemes, such as the impulse response length of time-domain convolution, the particle density of the particle system, and the FFT block size of frequency-domain processing. For spectral characteristic adjustment: if the input audio has a high proportion of high-frequency signals, the particle density of high-frequency synthesis is increased to enhance high-frequency details; if the low-frequency signals have a high proportion, the impulse response length of low-frequency convolution is increased to enhance spatial sense. For system load adjustment: if the CPU load is high, the FFT block size is reduced and the particle density is lowered; if the load is low, the FFT block size is increased and the particle density is increased to improve sound quality. Through dual-dimensional dynamic parameter tuning of spectral characteristic adjustment and system load adjustment, a dual adaptation between signal characteristics and system state is achieved. This avoids both insufficient adaptation of fixed parameters to different types of audio and performance instability of fixed parameters under fluctuating system load, ensuring a balance between sound quality and real-time performance under different audio types and different device loads.

[0030] Step 103 involves efficiency optimization logic for frequency domain processing. Based on multi-level caching adapted to block convolution, it reduces computational load through block division and improves access efficiency through multi-level caching, thus solving the drawback of traditional frequency domain convolution on mobile devices, which cannot achieve real-time processing of long-impulse responses due to memory access issues. Multi-level caching is a hierarchical storage structure relative to traditional single-level caching. The function and data type of each level of caching are strictly divided and adapted to the processing needs of different frequency band signals.

[0031] Traditional frequency-domain convolution performs an FFT operation on the complete input signal and the complete impulse response. Traditional FFT-based convolution faces challenges when handling extremely long impulse responses, including discontinuous memory access patterns, low cache hit rates, and poor data locality. This is particularly problematic on mobile devices with limited memory bandwidth, where the theoretical advantages of frequency domain transformation are difficult to translate into practical performance improvements. When the impulse response is too long (e.g., >2 seconds), the computational cost of a single FFT is extremely high. Block convolution breaks down the long impulse response into multiple smaller impulse response blocks, and simultaneously breaks down the input audio into corresponding input signal blocks, performing frequency-domain convolution on each block (i.e., convolution between blocks), significantly reducing the computational cost per operation. Furthermore, a multi-level caching architecture optimizes data access for block convolution, storing data blocks of different frequency bands in appropriate cache levels, avoiding memory access latency caused by frequent data switching in block convolution.

[0032] In some specific embodiments, the multi-level caching architecture includes a first cache layer, a second cache layer, and a third cache layer. The first cache layer processes real-time data, the second cache layer processes high-frequency multiplexed data, and the third cache layer predicts pre-loaded data for the next processing cycle, enabling frequency-domain block convolution processing of n frequency band signals. The first cache layer is the fastest and has the smallest capacity. It stores data currently being processed or about to be processed. The second cache layer is the next fastest and has a medium capacity. It stores frequently used data to avoid frequent retrievals from slow main memory. The third cache layer is the slowest and has the largest capacity. It stores data that may be used in the near future and attempts to predict what the next processing stage will need. These three cache layers do not exist in isolation; they work together to serve the core operation of frequency-domain block convolution, translating the theoretical advantages of block convolution into actual performance improvements and reducing overall processing latency.

[0033] Furthermore, the first cache level stores the FFT transform result of the current processing block and uses the SIMD instruction set to optimize complex number operations for processing real-time data. The FFT result is a complex number, which is the core real-time computational data in frequency domain convolution. SIMD (Single Instruction Multiple Data) is a parallel computing technique that allows a single instruction to perform the same operation on multiple data simultaneously. Placing the FFT result in the fastest L1 cache and using the SIMD instruction set to perform complex number multiplication on it maximizes the core computational performance of frequency domain convolution. The second cache level caches the impulse response frequency domain data in the mid-frequency band based on the LRU strategy to handle high-frequency multiplexed data. The mid-frequency band (800Hz~4kHz) is the region most sensitive to the human ear, and its impulse response is usually long and complex, making it the most likely to be reused in block convolution. Placing it in the L2 cache maximizes reuse efficiency. The third cache level preloads the computational data for the next processing cycle in the low-frequency band using an access pattern prediction algorithm. Low-frequency band (0~800Hz) processing employs high-precision temporal convolution, which involves large datasets and lengthy processing times. Preloading low-frequency band data allows for thorough preparation for the next processing cycle, preventing overall latency caused by excessive low-frequency processing time. Deeply integrating underlying optimization techniques such as SIMD, LRU, and access pattern prediction with upper-layer perception-driven frequency band processing logic forms a complete system of hardware and software co-optimization. Each cache layer serves the frequency band data it is best suited for, maximizing efficiency.

[0034] Existing algorithms employ a uniform high-precision processing strategy across all frequency bands, failing to fully utilize the frequency masking effect and critical band characteristics of the human auditory system. This results in a waste of computational resources on frequency bands with low perceptual contribution, impacting overall processing efficiency. Step 104 is the integrated output logic after multi-band processing. Its core principles are perceptual optimization mixing and signal integrity assurance. This solves the problems of unnatural frequency mixing, phase discontinuity, and output distortion that can occur with segmented frequency processing, and is crucial for ensuring final sound quality. Masking effect mixing makes the audio more consistent with human perception, avoiding auditory conflicts between frequency bands. Phase continuity assurance ensures natural transitions between frequency bands without any discontinuity. Dynamic range control avoids output distortion and adapts to different devices.

[0035] In some specific embodiments, the acquisition of reverberant audio includes: dynamically adjusting the mixing weights of the low-frequency convolution result and the high-frequency synthesized signal using psychoacoustic principles of simultaneous masking and temporal masking; ensuring phase continuity and spatial consistency of different frequency band components at the synthesis point through a phase coherence preservation algorithm; and intelligently adjusting the gain distribution based on the audio characteristics of the input audio and the audio characteristics of the output device through an adaptive dynamic range control mechanism to effectively prevent clipping distortion and dynamic range compression. The system analyzes the energy distribution of the low-frequency convolution signal and the high-frequency synthesized signal in real time. When a strong signal is detected in the low-frequency band, the mixing weight of the high-frequency band is temporarily reduced because the human ear cannot hear the high-frequency details masked by the low frequencies at this time. Conversely, when the low-frequency energy is weak, it automatically increases the weight of the high-frequency band to highlight spatial details and a sense of airiness. The mixing weights are not fixed but are a function that changes continuously with time and frequency, perfectly simulating the human ear's perception process.

[0036] The masking effect of human hearing is a fundamental principle of psychoacoustics—the human ear's sensitivity to weak sounds near loud sounds is significantly reduced. Mixing involves superimposing the n frequency band signals optimized by feature 3 into a single audio stream. Mixing based on the masking effect means that the algorithm first analyzes the signal strength of each frequency band, appropriately reducing the weight of weak frequency bands masked by loud sounds, while maintaining the weight of unmasked frequency bands. For example, if the signal strength of the low-frequency band is much higher than that of the mid-frequency band, the weight of the mid-frequency band can be appropriately reduced to avoid the weak mid-frequency sounds still consuming computational power after being masked by strong low-frequency sounds, while simultaneously making the mixed sound more consistent with human auditory perception.

[0037] Because different processing schemes are used for different frequency bands, the phase information of each frequency band may differ after processing. For example, temporal convolution preserves the original phase, while the phase of the synthesized signal is generated analogically. Phase discontinuity can cause a sense of discontinuity in the audio. To ensure phase continuity, a phase coherence preservation algorithm is used to adjust the phase parameters of each frequency band, making the phases of adjacent frequency bands consistent at the crossover point, thus ensuring a natural sound transition.

[0038] Dynamic range is the difference between the maximum peak value and the minimum valley value of an audio signal. Dynamic range control is used to avoid problems caused by an excessively large dynamic range: if the peak value is too high, it will exceed the maximum tolerance of the output device, resulting in clipping distortion; if the valley value is too low, it will be masked by noise, resulting in loss of detail. The dynamic range control mechanism monitors the peak and valley values ​​of the input signal in real time, and intelligently adjusts the gain in combination with the characteristics of the output device to ensure that the dynamic range of the output audio is adapted to the device, without distortion and with clear details.

[0039] Time-domain and frequency-domain convolution requires storing and processing massive amounts of impulse response data, resulting in high computational complexity. Furthermore, the high-frequency impulse response data exhibits significant redundancy, leading to substantial waste of computational power and memory. This application employs a particle system based on acoustic physics principles. In some specific embodiments, within this acoustic physics-based particle system, each acoustic particle represents an independent reflected sound path. Particle state parameters include three-dimensional spatial position, velocity vector, frequency-dependent attenuation coefficient, propagation delay time, and reflection count. Through the random distribution, Brownian motion, and boundary reflection behavior of the particles, the spatial diffusion characteristics and temporal attenuation laws of high-frequency reverberation are accurately simulated. Compared to convolution, the computational cost of the particle system is proportional to the number of particles, rather than the impulse response length. Through physical behavior simulation, the generated high-frequency reverberation possesses a natural randomness and spatial quality.

[0040] The particle system transforms the complex high-frequency reverberation process from the traditional approach of recording the sum of all reflected sounds to simulating the physical processes that generate these reflected sounds. Each acoustic particle represents a beam of sound photons originating from the sound source, traveling through the room, reflecting, and finally reaching the ear. By simulating the behavior of each sound photon, the final result is calculated in real time. To allow the particles to move and attenuate like real sound, each particle must be assigned a series of precise state parameters. These parameters collectively determine the sonic characteristics of the reflected sound represented by the particle. Three-dimensional spatial position: the particle's real-time coordinates in the virtual room. Velocity vector: the particle's direction and velocity. Frequency-dependent attenuation coefficient: simulates the physical characteristic that high-frequency sound attenuates faster when propagating through the air. Different frequencies of sound have different attenuation coefficients, with higher frequencies having larger coefficients. Propagation delay time: the time it takes for the particle to travel from the sound source to its current position. This directly determines the timing of the reflected sound reaching the listener's ear and is the basis for the perception of reverberation and echo. Reflection count: reflects how many times the particle has reflected off boundaries such as the ground. This is related to the attenuation law; generally, the more reflections, the weaker the sound energy.

[0041] The particle system is not static; it is a dynamic, adaptive system. The system determines the number of particles generated and their initial states based on scene requirements. Scene requirements include: Room geometry parameters: the size and shape of the room. Larger rooms require more particles to simulate more complex reflection paths. Wall material properties: reflecting whether the walls are smooth or rough. Smooth materials reflect strongly, resulting in a lower particle attenuation coefficient; rough materials absorb more, resulting in a higher particle attenuation coefficient. Target reverberation characteristics: the reverberation quality desired by the user.

[0042] By simulating three core particle behaviors, this system recreates two key characteristics of high-frequency reverberation in the real world: Random distribution: Simulates the initial randomness of sound propagation in all directions after originating from the sound source. Brownian motion: Simulates the subtle influence of minute airflow disturbances on the propagation path of high-frequency sound, making reflected sound sound more natural. Boundary reflection: Simulates the reflection of sound on obstacles such as walls. When a particle collides with a boundary, the system calculates a new reflection angle based on the incident angle and the boundary material, and updates its attenuation coefficient according to the material properties. Through the combination of these three behaviors, the particle system can accurately simulate spatial diffusion characteristics and temporal attenuation patterns.

[0043] In some specific embodiments, a machine learning-based adaptive computing power adjustment module adjusts the computational parameters for differentiated processing. These parameters include hardware parameters, system status, and application scenarios. This module monitors the overall system's condition and external environment, making the most intelligent decisions and dynamically allocating computing resources. It uses a large number of different hardware devices, system loads, and application scenarios as input, recording the optimal balance between sound quality and real-time performance under these conditions. This data is then used to train a model, such as a decision tree or neural network. The module collects real-time information on the device's hardware parameters, system status, and application scenarios, inputting this information into the trained model. The model quickly considers and outputs a set of optimal computational parameters to guide subsequent differentiated processing. When the system load suddenly increases, the module reacts rapidly, reducing the computational complexity of non-critical parameters and prioritizing real-time performance; when the load decreases, it automatically reverts to high-quality mode, achieving a smooth experience under load fluctuations.

[0044] Preferably, the adaptive adjustment process of computing power can be modeled as a multi-objective optimization problem. Hardware features, system state, and application context (such as audio type, user preferences, noise, etc.) are abstracted into mathematical feature expressions (one-hot encoding, data normalization representation, etc. can be used). A backbone network with a fully connected + LSTM design is used, combined with an output layer designed with SoftMax as a machine learning network model to solve the multi-objective optimization problem.

[0045] The hardware parameters include CPU architecture type, number of physical cores, cache hierarchy, memory bandwidth capacity, and floating-point operation capability; system status includes real-time CPU load, core temperature, battery level, network connection status, and background process contention; and application scenarios include audio content type, user preference settings, environmental noise level, and latency sensitivity requirements.

[0046] CPU architecture types include ARM's Cortex-A series and x86's Intel Core series. Different architectures have different instruction sets and pipeline designs, and their optimization methods for specific algorithms also differ.

[0047] Physical core count: The more cores, the stronger the parallel processing capability.

[0048] Cache hierarchy: The size and speed of L1 / L2 / L3 caches directly affect memory access efficiency.

[0049] Memory bandwidth capacity: Memory read and write speed is the bottleneck of data throughput.

[0050] Real-time CPU load: The percentage of CPU currently being used, the most direct performance indicator.

[0051] Core temperature: Overheating of the CPU will trigger frequency throttling protection, resulting in performance degradation. Monitoring the temperature can help predict performance bottlenecks in advance.

[0052] Battery level: On mobile devices, battery life is a critical constraint. When the battery is low, some performance may need to be sacrificed to extend battery life.

[0053] Network connectivity status: network latency and stability.

[0054] Background process competition: The competition for CPU and memory resources among other running apps.

[0055] Floating-point performance (FLOPS): The speed at which the CPU processes decimals (the basic unit of audio signals) is a core indicator of audio processing performance.

[0056] Audio content type: Different types of audio have different reverb requirements.

[0057] User preference settings: The type of reverb the user might prefer.

[0058] Ambient noise level: On noisy streets, a stronger reverberation effect and dynamic range are needed to highlight the sound; in quiet indoor environments, a more delicate and natural effect can be achieved.

[0059] Delay sensitivity requirement: Requirements regarding latency.

[0060] This application also proposes an intelligent perception-based adaptive real-time reverberation processing system to implement any of the above-mentioned intelligent perception-based adaptive real-time reverberation processing methods, with modules as follows: Figure 2 As shown, the adaptive real-time reverberation processing system includes: Frequency band processing module 1 is used to divide the input audio into n frequency band regions according to perceptual importance based on multiple preset critical frequency bands, and dynamically adjust the frequency division point position between frequency band regions based on the real-time load status of the system; and to perform differentiated processing on the signals of each frequency band based on a preset signal processing scheme. The computing power adaptive adjustment module 2 is used to dynamically adjust the computing parameters of differentiated processing and output n frequency band signals; Frequency domain block convolution module 3 is used to perform frequency domain block convolution processing on n frequency band signals based on a pre-built multi-level caching architecture. Output synthesis module 4 is used to mix the convolution results of n frequency band signals based on the human auditory masking effect, and output the final reverberant audio through a dynamic range control mechanism while ensuring phase continuity between frequency bands.

[0061] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform an intelligent perception-based adaptive real-time reverberation processing method. Applying an intelligent perception-based adaptive real-time reverberation processing method to a computer program product facilitates execution.

[0062] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an adaptive real-time reverberation processing method based on intelligent perception as described above.

[0063] The computer storage medium of this application can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. This application applies an intelligent perception-based adaptive real-time reverberation processing method to a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the method steps provided in this application, which is simple, fast, easy to store, and not easily lost.

[0064] This application proposes an adaptive real-time reverberation processing method and system based on intelligent perception. It addresses the shortcomings of existing reverberation processing algorithms by dividing the input audio into n frequency bands based on perceptual importance using multiple preset critical frequency bands and dynamically adjusting the sub-band positions. This fully adapts to the characteristics of human hearing, avoiding the waste of computational resources caused by traditional uniform frequency band processing. Simultaneously, it combines real-time system load status to achieve flexible adaptation of frequency band processing strategies, ensuring processing quality for perceptually important frequency bands while reducing the computational load on low-perceptual-contribution frequency bands. By performing differentiated processing on signals of each frequency band based on a preset signal processing scheme and dynamically adjusting the computational parameters in the differentiated processing, it effectively adapts to the hardware characteristics of different devices and real-time system status, solving the problem of inconsistent performance of traditional fixed processing strategies in heterogeneous environments and achieving precise matching of computational resources and processing requirements. Furthermore, based on a pre-built multi-level caching architecture, the method further adapts to the n frequency bands. Frequency-domain block convolution processing of frequency band signals can significantly improve the memory access defects of traditional frequency-domain convolution, enhance cache hit rate and data locality. Especially in scenarios with limited memory bandwidth, such as mobile devices, it can fully unleash the performance advantages of frequency-domain processing and alleviate the contradiction between computational complexity and real-time performance. In addition, by mixing the frequency band signals after convolution based on the human auditory masking effect and combining it with a dynamic range control mechanism to ensure phase continuity between frequency bands, computational efficiency can be further optimized while ensuring the naturalness of the final output reverberant audio. This effectively prevents clipping distortion and dynamic range compression. Ultimately, in environments with limited computing resources, such as mobile devices and embedded systems, a dynamic balance between computational complexity and audio perception quality is achieved. This promotes the widespread deployment and industrial application of professional-grade reverberation algorithms on various consumer terminals, meeting users' needs for high-quality reverberation effects while ensuring the stability and real-time performance of device operation.

[0065] Those skilled in the art will understand that the modules described above can be implemented using general-purpose computing systems. They can be centralized on a single computing system or distributed across a network of multiple computing systems. Optionally, they can be implemented using computer-executable program code, allowing them to be stored in a storage system for execution by the computing system. Alternatively, they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0066] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

[0067] The above disclosures are only a few specific implementation scenarios of this application. However, this application is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. An adaptive real-time reverberation processing method based on intelligent perception, characterized in that, include: Based on multiple preset critical frequency bands, the input audio is divided into n frequency band regions according to perceptual importance, and the position of the frequency division point between the frequency band regions is dynamically adjusted based on the real-time load status of the system. Based on a preset signal processing scheme, signals in each frequency band are processed differently, and the operation parameters in the differentiating processing are dynamically adjusted according to the spectral characteristics of each frequency band region and the real-time load status of the system, so as to output n frequency band signals. Frequency domain block convolution processing is performed on n frequency band signals based on a pre-built multi-level caching architecture; Based on the human auditory masking effect, the convolution results of n frequency band signals are mixed, and the final reverberant audio is output through a dynamic range control mechanism while ensuring the phase continuity between frequency bands.

2. The adaptive real-time reverberation processing method according to claim 1, characterized in that, Based on the human hearing critical frequency band theory and the Bark psychoacoustic scale, 24 critical frequency bands were established. The input audio signal was divided into three frequency band regions according to the perceptual importance: low frequency region, mid frequency region, and high frequency region. Each frequency band region corresponds to a signal processing scheme.

3. The adaptive real-time reverberation processing method according to claim 2, characterized in that, The signal processing scheme for the low-frequency region includes high-precision temporal convolution processing to preserve spatial positioning and phase information; The signal processing scheme for the mid-frequency band region includes a hybrid processing mode of convolution and synthesis. The signal processing scheme for the high-frequency band includes intelligent synthesis based on a particle system model using acoustic physics principles.

4. The adaptive real-time reverberation processing method according to claim 3, characterized in that, In a particle system based on acoustic physics principles, each acoustic particle represents an independent reflected sound path. Particle state parameters include three-dimensional spatial position, velocity vector, frequency-dependent attenuation coefficient, propagation delay time, and reflection count. Through the random distribution of particles, Brownian motion, and boundary reflection behavior, the spatial diffusion characteristics and temporal attenuation laws of high-frequency reverberation are accurately simulated.

5. The adaptive real-time reverberation processing method according to claim 1, characterized in that, The multi-level caching architecture includes a first cache level, a second cache level, and a third cache level; Real-time data is processed through the first cache level, high-frequency multiplexed data is processed through the second cache level, and preloaded data for the next processing cycle is predicted through the third cache level, so as to perform frequency domain block convolution processing on n frequency band signals.

6. The adaptive real-time reverberation processing method according to claim 5, characterized in that, The first cache level stores the FFT transformation result of the current processing block and uses the SIMD instruction set to optimize complex number operations to process real-time data; the second cache level caches impulse response frequency domain data in the mid-frequency band based on the LRU strategy to process high-frequency multiplexed data; the third cache level preloads the operation data of the next processing cycle in the low-frequency band through an access pattern prediction algorithm.

7. The adaptive real-time reverberation processing method according to claim 1, characterized in that, The acquisition of the reverberant audio specifically includes: By utilizing the psychoacoustic principles of simultaneous masking and temporal masking, the mixing weights of the low-frequency convolution result and the high-frequency synthesized signal are dynamically adjusted; and a phase coherence preservation algorithm is used to ensure the phase continuity and spatial consistency of different frequency band components at the synthesis point. Through an adaptive dynamic range control mechanism, the gain distribution is intelligently adjusted according to the audio characteristics of the input audio and the audio characteristics of the output device, effectively preventing clipping distortion and dynamic range compression.

8. The adaptive real-time reverberation processing method according to claim 1, characterized in that, The computational parameters of the differentiated processing are adjusted by a machine learning-based adaptive computing power adjustment module. These computational parameters include hardware parameters, system status, and application scenarios.

9. The adaptive real-time reverberation processing method according to claim 8, characterized in that, The hardware parameters include CPU architecture type, number of physical cores, cache hierarchy, memory bandwidth capacity, and floating-point operation capability. The system status includes real-time CPU load, core temperature, battery level, network connection status, and background process contention. The application scenarios include audio content type, user preference settings, ambient noise level, and latency sensitivity requirements.

10. An adaptive real-time reverberation processing system based on intelligent perception, characterized in that, For implementing the intelligent perception-based adaptive real-time reverberation processing method according to any one of claims 1-9, the adaptive real-time reverberation processing system comprises: The frequency band processing module is used to divide the input audio into n frequency band regions according to perceptual importance based on multiple preset critical frequency bands, and dynamically adjust the frequency division point position between frequency band regions based on the real-time load status of the system; and perform differentiated processing on the signals of each frequency band based on preset signal processing schemes. The computing power adaptive adjustment module is used to dynamically adjust the computing parameters for differentiated processing and output n frequency band signals; The frequency domain block convolution module is used to perform frequency domain block convolution processing on n frequency band signals based on a pre-built multi-level caching architecture. The output synthesis module is used to mix the convolution results of n frequency band signals based on the human auditory masking effect, and output the final reverberant audio through a dynamic range control mechanism while ensuring phase continuity between frequency bands.

Images