Estimation method and device for audio reverberation space clue parameters and storage medium

Abstract

The invention discloses an audio reverberation space clue parameter estimation method and device and a storage medium, and relates to the technical field of audio processing. The method comprises the following steps: acquiring a target wet sound signal, and carrying out dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal; obtaining parameter estimation rules corresponding to a plurality of target audio space cue parameters used for representing reverberation space characteristics; the parameter estimation rule is pre-constructed based on the correlation between the signal statistical characteristic of the audio signal and the audio space clue parameter; and according to the parameter estimation rule, analyzing the target wet sound signal, the dry sound signal and the pure reverberation signal to obtain a target audio space clue parameter of the target wet sound signal. Automatic audio reverberation space clue parameter estimation can be realized, the audio space clue parameter estimation efficiency and accuracy are improved, and the method is suitable for linear and nonlinear reverberation effects.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a method, apparatus, and storage medium for estimating audio reverberation spatial cue parameters. Background Technology

[0002] In music production and vocal conversion, it is often necessary to apply the reverberation style of a reference audio to a piece of audio to be processed. Existing technologies employ impulse response (IR) extraction methods based on deconvolution, such as Wiener deconvolution. These methods assume that the reverberation process is a linear time-invariant (LTI) system, meaning that the wet sound is obtained by convolving the dry sound with the reverberation impulse response. However, this approach requires both dry and wet audio samples of a perfectly synchronized and clean reference audio source, making blind estimation (estimating audio spatial cue parameters with only wet audio signals and no prior information from the original dry audio or reverberator parameters) impossible. Furthermore, it is only applicable to LTI reverberation; the deconvolution method is not suitable for nonlinear, time-varying reverberation effects (such as dynamic reverberation, modulation reverberation, etc.). In addition, the "reverberation impulse response" obtained by deconvolution is only an impulse response waveform, which cannot provide intuitive audio spatial cue parameters. Secondary analysis of the reverberation impulse response is required to obtain some parameters, making it difficult to map to the control parameters of different reverberators.

[0003] In existing technologies, reverberation effects that are nonlinear and time-varying are achieved through manual auditory matching and adjustment. This relies on the professional auditory experience of mixing engineers to replicate a reverberation effect by repeatedly listening to reference audio and continuously adjusting the parameters on the reverberator. However, this method is entirely manual, costly, inefficient, and its results are limited by the engineer's experience and auditory state, resulting in strong subjectivity and a lack of objectivity and consistency. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, device, and storage medium for estimating audio reverberation spatial cue parameters, which can achieve automated estimation of audio reverberation spatial cue parameters, improve the efficiency and accuracy of audio spatial cue parameter estimation, and is applicable to both linear and nonlinear reverberation effects. The specific solution is as follows: Firstly, this application discloses a method for estimating audio reverberation spatial cue parameters, including: Acquire the target wet sound signal, and perform dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal; Obtain parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; Based on the parameter estimation rules, the target wet sound signal, the dry sound signal, and the pure reverberation signal are analyzed to obtain the target audio spatial cue parameters of the target wet sound signal.

[0005] Optionally, the target audio spatial cue parameters include reverberation time, damping, wet sound level, and stereo width; The step of analyzing the target wet sound signal, the dry sound signal, and the pure reverberation signal according to the parameter estimation rule to obtain the target audio spatial cue parameters of the target wet sound signal includes: The reverberation time of the target wet sound signal is calculated based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal. The damping of the target wet acoustic signal is determined based on the difference in the spectral slope of the power spectral density corresponding to the dry acoustic signal and the target wet acoustic signal, respectively. The wet sound level of the target wet sound signal is obtained based on the energy ratio of the pure reverberation signal to the dry sound signal. The stereo width of the target wet sound signal is determined based on the left and right channels of the pure reverberation signal.

[0006] Optionally, calculating the reverberation time of the target wet sound signal based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal includes: Based on the pure reverberation signal and the corresponding energy envelope, the target reverberation attenuation segment is selected according to the validity judgment rule; The reverberation time of the target reverberation attenuation section is calculated based on the energy attenuation slope of the target reverberation attenuation section. Based on the reverberation time of all the target reverberation attenuation segments in the pure reverberation signal and the natural attenuation reference value, the final reverberation time corresponding to the target wet sound signal is calculated.

[0007] Optionally, determining the damping of the target wet acoustic signal based on the difference in the spectral slope of the power spectral density corresponding to the dry acoustic signal and the target wet acoustic signal respectively includes: The first spectral slope is calculated based on the power spectral density of the dry sound signal; The second spectral slope is calculated based on the power spectral density of the target wet acoustic signal; The damping of the target wet acoustic signal is obtained based on the difference between the first frequency slope and the second spectral slope.

[0008] Optionally, determining the stereo width of the target wet sound signal based on the left and right channels of the pure reverberation signal includes: The stereo width of the target wet sound signal is calculated based on the cross-correlation coefficient between the left and right channels of the pure reverberation signal. Alternatively, the center signal and side signal can be determined based on the left and right channels of the pure reverberation signal, and the stereo width of the target wet sound signal can be obtained based on the energy ratio of the side signal to the center signal.

[0009] Optionally, after obtaining the target audio spatial cue parameters of the target wet sound signal, the method further includes: Determine the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm; The target dry audio is obtained, and reverberation is added to the target dry audio using the target reverberator according to the reverberation control parameters to obtain the processed wet audio; the target reverberator uses the target reverberation algorithm.

[0010] Optionally, determining the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm includes: Query the mapping relationship between the spatial cue parameters and reverberation parameters corresponding to the target reverberation algorithm, and determine the reverberation control parameters corresponding to the target audio spatial cue parameters; The process of constructing the mapping relationship between the spatial cue parameters and the reverberation parameters includes: The target reverberator is used to add reverberation to the test signal according to each combination of reverberation control parameters in sequence, so as to obtain the reverberated signal corresponding to each combination of reverberation control parameters. The target audio spatial cue parameters of the reverberated signal are determined, and based on the correspondence between the reverberation control parameter combination and the target audio spatial cue parameters of the reverberated signal, a mapping relationship between the spatial cue parameters and the reverberation parameters of the target reverberation algorithm is constructed.

[0011] Secondly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the aforementioned method for estimating audio reverberation spatial cue parameters.

[0012] Thirdly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned method for estimating audio reverberation spatial cue parameters.

[0013] Fourthly, this application discloses a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for estimating audio reverberation spatial cue parameters.

[0014] In this application, a target wet sound signal is acquired, and the target wet sound signal is separated into dry and wet sound signals to obtain a dry sound signal and a pure reverberation signal; parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize the reverberation spatial characteristics are acquired; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; according to the parameter estimation rules, the target wet sound signal, the dry sound signal, and the pure reverberation signal are analyzed to obtain the target audio spatial cue parameters of the target wet sound signal.

[0015] As can be seen, by pre-constructing corresponding parameter estimation rules for different types of target audio spatial cue parameters based on the correlation between signal statistical characteristics and audio spatial cue parameters, estimation using these rules only requires the wet sound signal, the dry sound signal separated from the wet sound signal, and the pure reverberation signal, without the need for the original dry sound signal, thus achieving blind estimation of audio spatial cue parameters. The parameter estimation rules enable automated and unified analysis, avoiding reliance on manual intervention. They are applicable to the estimation of audio spatial cue parameters for both linear and nonlinear reverberation effects. Furthermore, specific target audio spatial cue parameters can be directly obtained, facilitating subsequent mapping to control parameters for different reverberators. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for estimating spatial cue parameters of audio reverberation provided in this application; Figure 2 A flowchart illustrating a specific method for estimating audio reverberation spatial cue parameters provided in this application; Figure 3 A flowchart of a specific reverberation method provided in this application; Figure 4 This application provides a structural diagram of an electronic device. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In existing technologies, impulse response extraction methods based on deconvolution treat wet sound as equal to dry sound. Reverberation impulse response (IR) (“ "" indicates convolution); however, this method has the following drawbacks: 1. It has stringent requirements for the input, requiring the original dry sound and the corresponding wet sound pair. If the dry sound is separated from the wet sound rather than the original dry sound, the slight distortion and residue in the separated dry sound will cause the result to deviate significantly from reality, or even fail completely, thus making blind estimation impossible; 2. It is only applicable to LTI reverberation. However, in modern music production, for artistic effect, mixing engineers often use non-linear, time-varying reverberation effects, making the deconvolution method unsuitable; 3. The reverberation impulse response obtained by deconvolution is only an impulse response waveform, which cannot provide intuitive audio spatial cue parameters. Secondary analysis of the reverberation impulse response is required to obtain the desired results. Some parameters are difficult to map onto the control parameters of different reverberators. For nonlinear, time-varying reverberation, existing technologies still employ manual auditory matching and adjustment, relying on the mixing engineer's professional auditory experience to replicate a reverberation effect by repeatedly listening to reference audio and continuously adjusting the parameters on the reverberator. However, this method is entirely manual, costly, inefficient, and its results are limited by the engineer's experience and auditory state, exhibiting strong subjectivity and lacking objectivity and consistency. To overcome these technical problems, this application proposes a method for estimating audio reverberation spatial cue parameters, enabling automated blind estimation of audio spatial cue parameters, improving the efficiency and accuracy of audio spatial cue parameter estimation, and applicable to both linear and nonlinear reverberation effects.

[0020] This application discloses a method for estimating audio reverberation spatial cue parameters, see [link to relevant documentation]. Figure 1 As shown, the method may include the following steps: Step S11: Obtain the target wet sound signal, and perform dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal.

[0021] Wet audio refers to audio with added reverb or other effects, while dry audio refers to audio without any artificially added effects (such as reverb). Pure reverb audio refers to a reverb signal without dry audio. Blind estimation refers to estimating reverb parameters using only the wet audio signal, without any prior information such as the original dry audio or reverb unit parameters. The wet audio signal is obtained by adding reverb to the original dry audio; the dry audio signal obtained by separating the wet audio signal from the original dry audio signal is not part of the original dry audio.

[0022] In this embodiment, the target wet sound signal, i.e., a wet sound audio segment used as a reference, is first acquired. The audio spatial cue parameters of the wet sound audio are determined through blind estimation. The audio spatial cue parameters can characterize the reverberation characteristics of the target wet sound signal, so that subsequent singing conversion or reverberation style transfer can be performed based on the estimation of the obtained audio reverberation spatial cue parameters. Specifically, a trained artificial intelligence reverberation removal model (such as Mel-band Roformer) can be used to process the wet sound signal to obtain an approximate dry sound signal and a pure reverberation signal.

[0023] Step S12: Obtain parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize the reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters.

[0024] This application does not rely on deconvolution processing. Instead, it utilizes the correlation between the statistical characteristics of the audio signal in the time and frequency domains (such as energy attenuation and spectral slope) and the audio spatial cue parameters to construct parameter estimation rules. In actual estimation, the actual signal statistical characteristics are first obtained by analyzing the target wet sound signal, dry sound signal, or pure reverberation signal. Then, the target audio spatial cue parameters are determined by combining the parameter estimation rules. Therefore, it also has good robustness to non-LTI reverberation. The parameter estimation rules constructed based on signal statistical characteristics quantify the audio spatial cue parameters by analyzing the overall attenuation and spectral characteristics of the target wet sound signal in the time and frequency domains. It does not rely on impulse response reconstruction and is suitable for non-ideal (such as nonlinear and time-varying) reverberation scenarios.

[0025] Among them, the target audio spatial cue parameters characterizing the reverberation space characteristics can be general parameters describing the reverberation listening experience. These target audio spatial cue parameters include, but are not limited to, reverberation time, damping, wet level, stereo width, and the ratio of early reflection time to reverberation time. Reverberation time (RT60) refers to the time required for the sound pressure level to decay by 60 dB after the sound source stops emitting sound; damping (also called attenuation) is a parameter used to control the phenomenon that high-frequency components decay faster than low-frequency components in reverberation; wet level refers to the energy level of the pure reverberation signal added to the dry sound signal; stereo width refers to the distribution width of the reverberation effect in the stereo sound field; early reflections refer to the sound that reaches the human ear after the sound source emits direct sound and is reflected once or a few times by interfaces such as walls and ceilings.

[0026] Step S13: Analyze the target wet sound signal, the dry sound signal and the pure reverberation signal according to the parameter estimation rules to obtain the target audio spatial cue parameters of the target wet sound signal.

[0027] For each target audio spatial cue parameter, a digital signal processing estimation workflow is designed for parameter estimation. This includes calculating reverberation time by analyzing the slope of the reverberation tail energy decay curve; quantifying damping by comparing the slope differences of dry and wet sound power spectral densities in specific frequency bands; determining the wet sound level by calculating the root-mean-square energy ratio of the pure reverberation signal and the dry sound signal; and evaluating stereo width by analyzing the cross-correlation between the left and right channels of the reverberation signal. This enables blind estimation of spatial cue parameters from a single wet sound audio file, requiring only one wet sound audio file as input. Estimation is performed using the separated pseudo-dry sound, achieving true blind estimation and eliminating dependence on the original dry sound. The entire estimation process is fully automated and can be completed within seconds, greatly improving the efficiency of reverberation effect replication. Finally, standardized target audio spatial cue parameters such as RT60 and damping are directly output.

[0028] This reduces the manual adjustment process, which previously required professional mixing engineers tens of minutes or even hours, to a few seconds of automated calculation, significantly improving the efficiency of music production and audio processing, and enabling efficient and automated reverb replication. It successfully solves the problems of traditional deconvolution methods being unable to handle non-LTI reverb and unable to perform blind estimation, broadening the applicability of automated reverb analysis technology. The target audio spatial cue parameters use a universal auditory description, making it possible to compare and transfer effects between different reverberators. Even users without in-depth acoustic knowledge and listening experience can easily achieve professional-grade reverb effect imitation through this solution, lowering the technical threshold for professional audio processing and demonstrating promising application prospects. This solution can be deployed on cloud servers or local computing devices.

[0029] In a preferred embodiment, the above-mentioned analysis of the target wet sound signal, the dry sound signal, and the pure reverberation signal according to the parameter estimation rule to obtain the target audio spatial cue parameters of the target wet sound signal includes the following steps: S21: Calculate the reverberation time of the target wet sound signal based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal. That is, calculate the reverberation time by analyzing the slope of the energy attenuation curve at the tail of the reverberation.

[0030] In some embodiments, calculating the reverberation time of the target wet sound signal based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal includes: S311: Based on the pure reverberation signal and the corresponding energy envelope, the target reverberation attenuation segment is selected according to the validity judgment rule.

[0031] The aforementioned validity criteria include duration, initial energy, and energy decay conditions. In other words, only reverberation decay segments that simultaneously meet these three conditions are considered target reverberation decay segments. It can be understood that a reverberation decay segment is the reverberation tail segment, and a target reverberation decay segment refers to an effective reverberation tail segment. For example, if the last note of a lyric meets the above conditions, it is a valid reverberation tail segment. However, if a lyric's middle word is immediately followed by the next word, meaning the energy decays only briefly and then immediately increases again, it does not meet the energy decay condition and is not considered a valid tail segment.

[0032] For example, a reverberant human voice is acquired as the target wet signal x_wet(t). From this, an approximate dry signal x_dry(t) and a pure reverberation signal x_reverb(t) are separated. x_reverb(t) can be specifically calculated as x_wet(t) - x_dry(t). A scan is performed on the energy envelope of the pure reverberation signal x_reverb(t) to identify all valid reverberation tails. The validity criteria include: the duration must exceed a preset threshold T_min; the initial energy must be higher than a preset threshold E_start (dB); and the energy decay from start to finish must exceed a preset threshold E_drop (dB).

[0033] S312: Calculate the reverberation time of the target reverberation attenuation section based on the energy attenuation slope of the target reverberation attenuation section.

[0034] First, the energy decay curve of the target reverberation decay segment is determined. Then, the energy decay slope of the target reverberation decay segment is obtained by linearly fitting the stable segment of the energy decay curve. Specifically, for each detected target reverberation decay segment, its logarithmic curve of energy decay over time is calculated. Since the energy decay rate is proportional to the current energy, the energy decay is close to a straight line on logarithmic coordinates, making it easy to accurately quantify the decay rate by measuring the slope of the straight line. Therefore, a logarithmic curve (i.e., an exponential decay model) is used to describe the decay. Then, in the stable segment of energy decay (e.g., from -5dB to -25dB), a least squares method is used for linear fitting to obtain a straight line with a slope k, which is taken as the energy decay slope. The RT60 value of the target reverberation decay segment is calculated based on the energy decay slope k: RT60_i = -60 / k; thus, the reverberation time of each target reverberation decay segment is obtained.

[0035] S313: Based on the reverberation time of all the target reverberation attenuation segments in the pure reverberation signal and the natural attenuation reference value, calculate the final reverberation time corresponding to the target wet sound signal.

[0036] Specifically, the calculation of the final reverberation time of the target wet sound signal based on the reverberation times of all target reverberation decay segments in the pure reverberation signal and the natural decay reference value includes: determining the initial reverberation time of the target wet sound signal statistically based on the reverberation times of all target reverberation decay segments in the pure reverberation signal; calculating the difference between the initial reverberation time and the natural decay reference value to obtain the final reverberation time of the target wet sound signal. For example, the RT60_i values ​​calculated from all target reverberation decay segments are collected, and the median is taken as the initial reverberation time RT60_median of the target wet sound signal, or the truncated mean (the mean of the remaining data after removing a specific proportion of extreme values ​​at both ends of the data) is taken as the initial reverberation time of the target wet sound signal. Subtracting a pre-determined natural decay reference value T_base, representing the natural decay time without reverberation, yields the final reverberation time of the target wet sound signal: RT60_final = RT60_median - T_base. Of course, you can also subtract the natural decay reference value from the reverberation time of the target reverberation decay section, and then take the median or the average value to get the final reverberation time.

[0037] In addition to the above-mentioned method of predicting reverberation time by fitting the decay curve using the least squares method, machine learning models can also be used to directly predict the reverberation time from the features of the reverberation signal (such as Mel frequency cepstral coefficients, spectral centroid, etc.). By pre-training a model for predicting reverberation time, the pure reverberation signal is input into the model to obtain the reverberation time of the target wet sound signal output by the model.

[0038] S22: Determine the damping of the target wet acoustic signal based on the difference in the spectral slope of the power spectral density corresponding to the dry acoustic signal and the target wet acoustic signal, respectively. That is, quantify the damping by comparing the slope difference of the power spectral density (PSD) of the dry and wet acoustic signals in a specific frequency band.

[0039] In some embodiments, determining the damping of the target wet acoustic signal based on the difference in the spectral slope of the power spectral density corresponding to the dry acoustic signal and the target wet acoustic signal respectively includes: S321: Calculate the first spectral slope based on the power spectral density of the dry sound signal, and calculate the second spectral slope based on the power spectral density of the target wet sound signal.

[0040] The spectral slope is obtained by linearly fitting the logarithmic curve of the power spectral density within the target frequency band; that is, the first spectral slope is obtained by linearly fitting the logarithmic curve of the power spectral density of the dry sound signal within the target frequency band, and the second spectral slope is obtained by linearly fitting the logarithmic curve of the power spectral density of the target wet sound signal within the target frequency band.

[0041] The target frequency band mentioned above is determined based on the target reverb algorithm. In subsequent scenarios such as vocal transitions, the reverb control parameters will be determined based on the audio spatial cue parameters, and reverb will be added to the target dry audio using the target reverb unit based on the reverb control parameters. Here, the target reverb algorithm refers to the reverb algorithm used by the target reverb unit.

[0042] S322: The damping of the target wet sound signal is obtained based on the difference between the first frequency slope and the second spectral slope.

[0043] For example, Welch's method is applied to calculate the power spectral density of the wet signal x_wet(t) and the dry signal x_dry(t), respectively, yielding PSD_wet(f) and PSD_dry(f). Within a preset target frequency band (e.g., [1000, 6000] Hz for the Freeverb reverberation algorithm), linear regression is performed on the logarithmic coordinate curves of PSD_wet(f) and PSD_dry(f), yielding the second spectral slope slope_wet and the first spectral slope slope_dry, respectively. The difference between the two slopes is used as the quantized representation of the damping parameter: Damping = slope_dry - slope_wet; a larger difference indicates greater damping, i.e., faster attenuation at high frequencies.

[0044] Besides quantifying damping based on spectral slope, it can also be achieved by calculating the energy ratio of different frequency bands. For example, calculating the ratio of high-frequency energy (e.g., 4-8kHz) to mid-low-frequency energy (e.g., 200-1kHz) and comparing the changes in this ratio between dry and wet sounds quantifies damping; that is, the degree of ratio change (e.g., ratio or dB difference) is used as a quantitative indicator of damping. The ratio of high-frequency energy to mid-low-frequency energy can be called spectral brightness, and the selected high-frequency and low-frequency bands are also related to the characteristics of the final reverberator used. Comparing the high-frequency (4–8kHz) to mid-low-frequency (200Hz–1kHz) energy ratios of dry and wet sounds... S23: Obtain the wet sound level of the target wet sound signal based on the energy ratio of the pure reverberation signal to the dry sound signal. For example, the wet sound level can be determined by calculating the root mean square (RMS) energy ratio of the pure reverberation signal to the dry sound signal. Specifically, the wet sound level of the target wet sound signal is obtained based on the ratio of the root mean square (RMS) value of the pure reverberation signal to the RMS value of the dry sound signal. For example, the RMS value of the pure reverberation signal x_reverb(t) and the RMS value of the dry sound signal x_dry(t) are calculated; the wet sound level Wet_Level = RMS_reverb / RMS_dry, and this ratio directly reflects the intensity of the reverberation effect.

[0045] Understandably, the root mean square (RMS) value of a signal can represent its average power level. Although it is not energy itself, it is a physically meaningful intensity indicator directly related to the signal energy. In addition, indicators related to signal energy, such as mean absolute error or peak value, can also be used to calculate the wet sound level.

[0046] S24: Determine the stereo width of the target wet sound signal based on the left and right channels of the pure reverberation signal. For example, the stereo width can be evaluated by analyzing the cross-correlation between the left and right channels of the reverberation signal.

[0047] The step of determining the stereo width of the target wet sound signal based on the left and right channels of the pure reverberation signal includes: calculating the stereo width of the target wet sound signal based on the cross-correlation coefficient between the left and right channels of the pure reverberation signal.

[0048] That is, based on ensuring that the pure reverberation signal x_reverb(t) is a stereo signal, we obtain the left channel x_reverb_L(t) and the right channel x_reverb_R(t) of the pure reverberation signal; calculate the cross-correlation coefficient cor(x_reverb_L, x_reverb_R) of the left and right channels of the pure reverberation signal; the stereo width Width = max{1 - cor(x_reverb_L, x_reverb_R), 1}, that is, the width value takes the range of 0 to 1. The closer it is to 0, the higher the correlation between the left and right channels, and the sound image tends to be mono; the closer it is to 1, the lower the correlation between the left and right channels and the wider the sound image.

[0049] Alternatively, the stereo width of the target wet sound signal can be determined based on the left and right channels of the pure reverberation signal, including: determining the center signal and side signal based on the left and right channels of the pure reverberation signal, and obtaining the stereo width of the target wet sound signal based on the energy ratio of the side signal to the center signal. That is, in addition to calculating the cross-correlation between the left and right channels, mid / side (M / S) decomposition can be used to decompose the stereo reverberation signal into a center signal (x_reverb_L(t) + x_reverb_R(t)) and a side signal (x_reverb_L(t) - x_reverb_R(t)), and calculating the ratio of the side signal energy to the center signal energy (Energy(S) / Energy(M)), which is directly related to the stereo width.

[0050] As can be seen from the above, this embodiment acquires the target wet sound signal, performs dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal; acquires parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize the reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; according to the parameter estimation rules, the target wet sound signal, the dry sound signal, and the pure reverberation signal are analyzed to obtain the target audio spatial cue parameters of the target wet sound signal. It is evident that by pre-constructing corresponding parameter estimation rules for different types of target audio spatial cue parameters based on the correlation between signal statistical characteristics and audio spatial cue parameters, estimation using these rules only requires the wet sound signal, and the dry sound signal and pure reverberation signal after separation from the wet sound signal, without needing the original dry sound signal, thus achieving blind estimation of audio spatial cue parameters; the parameter estimation rules enable automated and unified analysis, avoiding reliance on manual intervention; it is applicable to the estimation of audio spatial cue parameters for both linear and nonlinear reverberation effects; simultaneously, specific target audio spatial cue parameters can be directly obtained, facilitating subsequent mapping to control parameters of different reverberators.

[0051] Based on the above embodiments, this application also discloses a reverberation method, for example... Figure 3 As shown, the method may include the following steps: Step S41: Obtain the target wet sound signal, and perform dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal; Step S42: Obtain parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; Step S43: Analyze the target wet sound signal, the dry sound signal and the pure reverberation signal according to the parameter estimation rules to obtain the target audio spatial cue parameters of the target wet sound signal; Step S44: Determine the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm.

[0052] For example Figure 2 As shown, the spatial cue parameter estimation DSP (digital signal processing) module is used to analyze the target wet sound signal, the dry sound signal, and the pure reverberation signal according to the parameter estimation rules to obtain the target audio spatial cue parameters of the target wet sound signal. After calculating the target audio spatial cue parameters of the target wet sound signal according to the parameter estimation rules, the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm are further determined, and the reverberation control parameters are the configuration parameters of the reverberator.

[0053] In some embodiments, determining the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm includes: querying the mapping relationship between spatial cue parameters and reverberation parameters corresponding to the target reverberation algorithm, and determining the reverberation control parameters corresponding to the target audio spatial cue parameters. The process of constructing the mapping relationship between spatial cue parameters and reverberation parameters includes: using the target reverberator to sequentially add reverberation to the test signal according to each combination of reverberation control parameters, obtaining the reverberated signal corresponding to each combination of reverberation control parameters; determining the target audio spatial cue parameters of the reverberated signal; and constructing the mapping relationship between the spatial cue parameters and reverberation parameters of the target reverberation algorithm based on the correspondence between the reverberation control parameter combinations and the target audio spatial cue parameters of the reverberated signal.

[0054] During the offline calibration phase, for a target reverb (such as Freeverb or a specified VST reverb plugin), the reverb control parameter combinations of that reverb are iterated; for example, room_size ranges from 0 to 1, damping from 0 to 1, and width from 0 to 1, with a step size of 0.05. The wet level (wet_level) usually does not need to be iterated, saving resources consumed during table lookups, because the reverb signal output by both linear and nonlinear reverbs is usually linearly related to this parameter. For each parameter combination, a standard test signal (such as a sinusoidal sweep signal) is passed through the reverb to generate the output audio. For this output audio, the parameter estimation rules in step S13 are used to back-analyze and derive the corresponding target spatial cue parameters (such as RT60, Damping, Wet Level, Width). Based on the correspondence between the reverberator control parameter combination (Reverberator_Param_Combination) and the analyzed target spatial cue parameters (Estimated_Cue_Parameters): {Reverberator_Param_Combination}-{Estimated_Cue_Parameters}, a multidimensional lookup table (LUT) is constructed.

[0055] In the application phase, based on the target audio spatial cue parameters of the target wet sound signal, a reverse lookup table is performed to find the item closest to the target parameters. After finding the best match, its corresponding reverberator parameter combination is extracted. By setting this set of reverberator parameters onto the target reverberator, it can be used to process new dry audio, thereby reproducing the reverberation effect of the reference audio.

[0056] Besides determining reverberation control parameters by constructing a mapping relationship, this can also be achieved using neural networks. Specifically, a small neural network (such as a multilayer perceptron, MLP) is pre-trained to learn a continuous mapping function from spatial cue parameters to reverberation control parameters. By optimizing the model online, after estimating the target audio spatial cue parameters, the reverberator is treated as a black-box function. The error between the estimated target audio spatial cue parameters and the actual effect parameters produced by the current reverberator parameters is used as the loss function. Gradient descent or other optimization algorithms are used to adjust the reverberator parameters in real time until the error is minimized. The model allows for smoother parameter interpolation, potentially achieving a more accurate match than a lookup table.

[0057] Step S45: Obtain the target dry audio, and add reverberation to the target dry audio using the target reverberator according to the reverberation control parameters to obtain the processed wet audio; the target reverberator uses the target reverberation algorithm.

[0058] Based on the target audio spatial cue parameters of the target wet audio, the corresponding reverberation control parameters are determined, and then the dry audio is reverberated using these reverberation control parameters. The resulting reverberated wet audio has a similar reverberation style to the target wet audio.

[0059] As can be seen from the above, in this embodiment, after obtaining the target audio spatial cue parameters of the target wet sound signal, the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm are determined; the target dry sound audio is obtained, and reverberation is added to the target dry sound audio using the target reverberator according to the reverberation control parameters to obtain the processed wet sound audio; the target reverberator uses the target reverberation algorithm.

[0060] As can be seen, by constructing a parameter mapping table for the target reverb, the estimated general spatial cue parameters are mapped to the control parameters of a specific reverb, thereby achieving accurate replication of the reverb effect in any reference audio. By pre-constructing mapping tables for different reverb algorithms, the estimated standardized parameters can be seamlessly converted into the control parameters of any target reverb plugin, demonstrating strong versatility and practicality.

[0061] This application solution can be applied to music production, audio post-production, and vocal conversion (SVC) fields. Taking reverb style transfer as an example, if someone really likes the reverb effect of the vocals in a song and wants to process their own recorded dry vocals to achieve a similar effect, the specific process is as follows: 1. Obtain the vocal portion (wet vocals) of a user-uploaded reference song as the reference audio, and obtain the target wet vocal signal based on this reference audio; 2. Obtain the user-uploaded recorded dry vocals (or other dry vocal audio) as the target dry vocal audio, which is the audio to be processed; 3. Analyze the reference audio by calling the method of this application, and automatically estimate its four parameters: RT60, damping, wet vocal level, and width; 4. Use these four parameters to drive a preset reverb engine to process the audio to be processed; 5. Output a new audio file with a reverb effect similar to the reference audio.

[0062] Taking SVC post-processing as an example, the typical process in an SVC task is to separate the vocals from the song (obtaining the dry vocals), perform timbre conversion, and then remix the converted vocals into the original accompaniment. To ensure a natural blending effect, the reverb of the converted vocals needs to be as similar as possible to the reverb of other instruments or ambient sounds in the original song. The specific reverb process is as follows: 1. Separate the vocals and accompaniment from the original song; 2. Using the original vocals (wet vocals) as reference audio, estimate the target audio spatial cue parameters, i.e., the original reverb parameters, using the estimation method of the audio reverb spatial cue parameters proposed in this application; 3. Perform SVC processing on the separated dry vocals to obtain the converted dry vocals; 4. Using the reverb method proposed in this application, add reverb to the converted dry vocals using the estimated target audio spatial cue parameters; 5. Mix the converted vocals with the added matching reverb with the original accompaniment to obtain the final product. By precisely matching the reverb environment of the original song, the converted vocals can be naturally integrated into the accompaniment, avoiding a disconnect or hollow feeling in the listening experience and significantly improving the quality of the final synthesized music.

[0063] Accordingly, embodiments of this application also disclose an estimation device for audio reverberation spatial cue parameters, the device comprising: The signal acquisition module is used to acquire the target wet sound signal, and to perform dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal. The rule acquisition module is used to acquire parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize the reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; The audio spatial cue parameter determination module is used to analyze the target wet sound signal, the dry sound signal and the pure reverberation signal according to the parameter estimation rules to obtain the target audio spatial cue parameters of the target wet sound signal.

[0064] As can be seen from the above, this embodiment acquires the target wet sound signal, performs dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal; acquires parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize the reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; according to the parameter estimation rules, the target wet sound signal, the dry sound signal, and the pure reverberation signal are analyzed to obtain the target audio spatial cue parameters of the target wet sound signal. It is evident that by pre-constructing corresponding parameter estimation rules for different types of target audio spatial cue parameters based on the correlation between signal statistical characteristics and audio spatial cue parameters, estimation using these rules only requires the wet sound signal, and the dry sound signal and pure reverberation signal after separation from the wet sound signal, without needing the original dry sound signal, thus achieving blind estimation of audio spatial cue parameters; the parameter estimation rules enable automated and unified analysis, avoiding reliance on manual intervention; it is applicable to the estimation of audio spatial cue parameters for both linear and nonlinear reverberation effects; simultaneously, specific target audio spatial cue parameters can be directly obtained, facilitating subsequent mapping to control parameters of different reverberators.

[0065] In some specific embodiments, the target audio spatial cue parameters include reverberation time, damping, wet sound level, and stereo width; the audio spatial cue parameter determination module may specifically include: The reverberation time determination unit is used to calculate the reverberation time of the target wet sound signal based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal. The damping determination unit is used to determine the damping of the target wet sound signal based on the difference in the spectral slope of the power spectral density corresponding to the dry sound signal and the target wet sound signal, respectively. A wet sound level determination unit is used to obtain the wet sound level of the target wet sound signal based on the energy ratio of the pure reverberation signal to the dry sound signal; A stereo width determination unit is used to determine the stereo width of the target wet sound signal based on the left and right channels of the pure reverberation signal.

[0066] In some specific embodiments, the reverberation time determination unit may specifically include: The target reverberation attenuation segment screening unit is used to screen out the target reverberation attenuation segment based on the pure reverberation signal and the corresponding energy envelope, according to the validity judgment rule. The first reverberation time determination unit is used to calculate the reverberation time of the target reverberation attenuation segment based on the energy attenuation slope of the target reverberation attenuation segment. The second reverberation time determination unit is used to calculate the final reverberation time corresponding to the target wet sound signal based on the reverberation time of all the target reverberation attenuation segments in the pure reverberation signal and the natural attenuation reference value.

[0067] In some specific embodiments, the damping determination unit may specifically include: The spectral slope calculation unit is used to calculate a first spectral slope based on the power spectral density of the dry sound signal and a second spectral slope based on the power spectral density of the target wet sound signal. The damping determination unit is used to obtain the damping of the target wet sound signal based on the difference between the first frequency slope and the second spectral slope.

[0068] In some specific embodiments, the stereo width determination unit may specifically include: The first stereo width calculation unit is used to calculate the stereo width of the target wet sound signal based on the cross-correlation coefficient between the left and right channels of the pure reverberation signal. Alternatively, a second stereo width calculation unit is used to determine the center signal and side signal based on the left and right channels of the pure reverberation signal, and to obtain the stereo width of the target wet sound signal based on the energy ratio of the side signal to the center signal.

[0069] In some specific embodiments, the device for estimating the audio reverberation spatial cue parameters may specifically include: The reverberation control parameter determination unit is used to determine the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm after obtaining the target audio spatial cue parameters of the target wet sound signal. A reverberation unit is used to acquire the target dry audio, and add reverberation to the target dry audio using a target reverberator according to the reverberation control parameters to obtain the processed wet audio; the target reverberator uses the target reverberation algorithm.

[0070] In some specific embodiments, the reverberation control parameter determination unit is used to query the mapping relationship between the spatial cue parameters and reverberation parameters corresponding to the target reverberation algorithm, and determine the reverberation control parameters corresponding to the target audio spatial cue parameters; The process of constructing the mapping relationship between spatial cue parameters and reverberation parameters includes: using the target reverberator to add reverberation to the test signal sequentially according to each combination of reverberation control parameters to obtain the reverberated signal corresponding to each combination of reverberation control parameters; determining the target audio spatial cue parameters of the reverberated signal; and constructing the mapping relationship between the spatial cue parameters and reverberation parameters of the target reverberation algorithm based on the correspondence between the combination of reverberation control parameters and the target audio spatial cue parameters of the reverberated signal.

[0071] Furthermore, this application also discloses an electronic device, see [link to relevant documentation]. Figure 4 As shown, the content in the figure should not be considered as any limitation on the scope of use of this application.

[0072] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the audio reverberation spatial cue parameter estimation method disclosed in any of the foregoing embodiments.

[0073] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0074] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include operating system 221, computer program 222 and data 223 including target wet sound signal, etc. The storage method can be temporary storage or permanent storage.

[0075] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the audio reverberation spatial cue parameter estimation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0076] Furthermore, this application also discloses a computer storage medium storing computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, they implement the steps of the method for estimating audio reverberation spatial cue parameters disclosed in any of the foregoing embodiments.

[0077] Furthermore, embodiments of this application also disclose a computer program product, including a computer program that, when executed by a processor, implements the method steps for estimating audio reverberation spatial cue parameters disclosed in any of the foregoing embodiments.

[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0079] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0080] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0081] The above provides a detailed description of the method, device, and storage medium for estimating audio reverberation spatial cue parameters provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of estimating an audio reverberation spatial cue parameter, characterized by, include: Acquire the target wet sound signal, and perform dry and wet sound separation on the target wet sound signal to obtain a dry sound signal and a pure reverberation signal; Obtain parameter estimation rules corresponding to multiple target audio spatial cue parameters used to characterize reverberation space characteristics; the parameter estimation rules are pre-constructed based on the correlation between the signal statistical characteristics of the audio signal and the audio spatial cue parameters; Based on the parameter estimation rules, the target wet sound signal, the dry sound signal, and the pure reverberation signal are analyzed to obtain the target audio spatial cue parameters of the target wet sound signal.

2. The method of estimating an audio reverberation spatial cue parameter according to claim 1, characterized in that, The target audio spatial cue parameters include reverberation time, damping, wet sound level, and stereo width. The step of analyzing the target wet sound signal, the dry sound signal, and the pure reverberation signal according to the parameter estimation rule to obtain the target audio spatial cue parameters of the target wet sound signal includes: The reverberation time of the target wet sound signal is calculated based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal. The damping of the target wet acoustic signal is determined based on the difference in the spectral slope of the power spectral density corresponding to the dry acoustic signal and the target wet acoustic signal, respectively. The wet sound level of the target wet sound signal is obtained based on the energy ratio of the pure reverberation signal to the dry sound signal. The stereo width of the target wet sound signal is determined based on the left and right channels of the pure reverberation signal.

3. The method of estimating an audio reverberation spatial cue parameter according to claim 2, characterized in that, The calculation of the reverberation time of the target wet sound signal based on the energy attenuation slope corresponding to the reverberation attenuation segment in the pure reverberation signal includes: Based on the pure reverberation signal and the corresponding energy envelope, the target reverberation attenuation segment is selected according to the validity judgment rule; The reverberation time of the target reverberation attenuation section is calculated based on the energy attenuation slope of the target reverberation attenuation section. Based on the reverberation time of all the target reverberation attenuation segments in the pure reverberation signal and the natural attenuation reference value, the final reverberation time corresponding to the target wet sound signal is calculated.

4. The method for estimating audio reverberation spatial cue parameters according to claim 2, characterized in that, The step of determining the damping of the target wet acoustic signal based on the difference in the spectral slope of the power spectral density corresponding to the dry acoustic signal and the target wet acoustic signal respectively includes: The first spectral slope is calculated based on the power spectral density of the dry sound signal; The second spectral slope is calculated based on the power spectral density of the target wet acoustic signal; The damping of the target wet acoustic signal is obtained based on the difference between the first frequency slope and the second spectral slope.

5. The method for estimating audio reverberation spatial cue parameters according to claim 2, characterized in that, Determining the stereo width of the target wet sound signal based on the left and right channels of the pure reverberation signal includes: The stereo width of the target wet sound signal is calculated based on the cross-correlation coefficient between the left and right channels of the pure reverberation signal. Alternatively, the center signal and side signal can be determined based on the left and right channels of the pure reverberation signal, and the stereo width of the target wet sound signal can be obtained based on the energy ratio of the side signal to the center signal.

6. The method for estimating audio reverberation spatial cue parameters according to any one of claims 1 to 5, characterized in that, After obtaining the target audio spatial cue parameters of the target wet acoustic signal, the method further includes: Determine the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm; The target dry audio is obtained, and reverberation is added to the target dry audio using the target reverberator according to the reverberation control parameters to obtain the processed wet audio; the target reverberator uses the target reverberation algorithm.

7. The method for estimating audio reverberation spatial cue parameters according to claim 6, characterized in that, The determination of the reverberation control parameters corresponding to the target audio spatial cue parameters under the target reverberation algorithm includes: Query the mapping relationship between the spatial cue parameters and reverberation parameters corresponding to the target reverberation algorithm, and determine the reverberation control parameters corresponding to the target audio spatial cue parameters; The process of constructing the mapping relationship between the spatial cue parameters and the reverberation parameters includes: The target reverberator is used to add reverberation to the test signal according to each combination of reverberation control parameters in sequence, so as to obtain the reverberated signal corresponding to each combination of reverberation control parameters. The target audio spatial cue parameters of the reverberated signal are determined, and based on the correspondence between the reverberation control parameter combination and the target audio spatial cue parameters of the reverberated signal, a mapping relationship between the spatial cue parameters and the reverberation parameters of the target reverberation algorithm is constructed.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method for estimating audio reverberation spatial cue parameters as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein the computer program, when executed by a processor, implements the method for estimating the audio reverberation spatial cue parameters as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method for estimating the audio reverberation spatial cue parameters as described in any one of claims 1 to 7.

Images