Audio processing method, apparatus, device, and storage medium

Abstract

The application belongs to the technical field of audio, and discloses an audio processing method, device and equipment and a storage medium. The application determines a transfer function matrix based on a preset window function, a short-time Fourier transform mode and room impulse response; updates an initial audio matrix corresponding to an audio signal according to a plurality of audio frame data corresponding to the audio signal, to obtain an updated audio matrix; performs convolution processing on the transfer function matrix and the updated audio matrix based on a preset convolution mode, to obtain a convolution time domain vector; and obtains a target output vector according to an initial audio vector corresponding to the audio signal and the convolution time domain vector. The transfer function matrix is obtained by processing the room impulse response through the preset window function and the short-time Fourier transform, the convolution time domain vector is obtained by processing the transfer function matrix and the updated audio matrix based on the preset convolution mode, and the target output vector is obtained according to the initial audio vector and the convolution time domain vector, so that the calculation amount is reduced while the reverberation effect is ensured.

Description

Technical Field

[0001] This invention relates to the field of audio technology, and in particular to an audio processing method, apparatus, and storage medium. Background Technology

[0002] Adding various reverb effects to pure speech signals to create a layered and aesthetically pleasing sound is a perennial topic in music production and speaker algorithms. Commonly used software like Audition and Pro Tools integrate a large number of reverb effects, providing users with a wide range of choices. However, these effects are offline algorithms and cannot be run in real-time on a DSP. Currently, mainstream real-time algorithms include delay lines, comb filters, all-pass filters, and Feedback Delay Networks (FDNs). Each has its advantages in terms of reverb effect and computational complexity, but none are as effective as temporal convolutional reverb. Temporal convolutional reverb first obtains the room impulse response of the reverb through acoustic recording or mirroring. The method involves convolving the RIR signal with the music signal to add a reverb effect. This method produces a very realistic reverb effect, but its main drawback is the high computational cost. Furthermore, for DSP systems that process frames, truncation is required after convolution, and the RIR length is generally much longer than the length of a single frame. To avoid significant errors, the convolution operation needs to be converted into an FIR filter operation, which further increases the computational cost of convolutional reverb. For example, if the RIR has 1000 points, the output of one sample point of convolutional reverb requires 1000 multiplications and 999 additions, or 1999 FLOPs (in computers, one complex multiplication or addition is often considered as one FLOP to analyze the computational cost of the algorithm). Therefore, reducing the computational cost while maintaining the reverb effect has become an urgent problem to be solved.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide an audio processing method, apparatus, and storage medium, aiming to solve the technical problem of reducing computational load while ensuring reverberation effect.

[0005] To achieve the above objectives, the present invention provides an audio processing method, the method comprising the following steps: The transfer function matrix is ​​determined based on the preset window function, short-time Fourier transform method, and room impact reverberation. The initial audio matrix corresponding to the audio signal is updated based on multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix; The transfer function matrix and the updated audio matrix are convolved using a preset convolution method to obtain a convolutional temporal vector. The target output vector is obtained based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector.

[0006] Optionally, the determination of the transfer function matrix based on a preset window function, a short-time Fourier transform, and room impact reverberation includes: The room's impact reverberation is overlaid according to a preset data overlay method to obtain multiple overlay vectors; Multiple overlapping vectors are windowed according to a preset window function to obtain multiple windowed vectors; Multiple windowed vectors are converted into multiple frequency domain vectors using the short-time Fourier transform method. Multiple frequency domain vectors are arranged according to a preset arrangement to obtain the transfer function matrix.

[0007] Optionally, updating the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix includes: Windowing is applied to multiple audio frame data according to a preset window function to obtain multiple windowed frame data; Multiple windowed frame data are converted into multiple audio frame signals using the short-time Fourier transform method. The initial audio matrix corresponding to the audio signal is updated according to the preset update method and multiple audio frame signals to obtain the updated audio matrix.

[0008] Optionally, the step of convolving the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector includes: The transfer function matrix and the updated audio matrix are calculated according to a preset calculation method to obtain a hybrid calculation matrix; The hybrid calculation vector is obtained based on the hybrid calculation matrix; The convolution time-domain vector is determined based on the preset window function, the inverse short-time Fourier transform method, and the hybrid computation vector.

[0009] Optionally, obtaining the hybrid computation vector based on the hybrid computation matrix includes: The data corresponding to each column in the hybrid calculation matrix is ​​obtained based on the hybrid calculation matrix; Calculate the data corresponding to each column in the hybrid calculation matrix to obtain multiple calculated data; Multiple computational data are combined to obtain a hybrid computational vector.

[0010] Optionally, obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector includes: The first truncation vector is obtained by truncating the data of the convolutional temporal vector according to the first truncation method. The initial audio vector and the first truncated vector corresponding to the audio signal are calculated according to the preset calculation method; The target output vector is obtained based on the calculation results.

[0011] Optionally, after obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector, the method further includes: The convolutional temporal vector is truncated according to the second truncation method to obtain the second truncated vector; The initial audio vector corresponding to the audio signal is assigned a value based on the second intercepted vector; The assigned audio vector corresponding to the audio signal is obtained based on the assignment result.

[0012] Furthermore, to achieve the above objectives, the present invention also proposes an audio processing apparatus, the audio processing apparatus comprising: The processing module is used to determine the transfer function matrix based on the preset window function, short-time Fourier transform method, and room impact reverberation. The update module is used to update the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal, so as to obtain the updated audio matrix. The processing module is also used to perform convolution processing on the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector; The output module is used to obtain the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector.

[0013] Furthermore, to achieve the above objectives, the present invention also proposes an audio processing device, the audio processing device comprising: a memory, a processor, and an audio processing program stored in the memory and executable on the processor, the audio processing program being configured to implement the steps of the audio processing method described above.

[0014] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing an audio processing program, which, when executed by a processor, implements the steps of the audio processing method described above.

[0015] This invention determines the transfer function matrix based on a preset window function, a short-time Fourier transform, and room impact reverberation. It then updates the initial audio matrix corresponding to the audio signal using multiple audio frame data, obtaining an updated audio matrix. Finally, it performs convolution processing on the transfer function matrix and the updated audio matrix using a preset convolution method, yielding a convolutional time-domain vector. The target output vector is then obtained from the initial audio vector and the convolutional time-domain vector. By using a preset window function and short-time Fourier transform to process room impact reverberation to obtain the transfer function matrix, processing the transfer function matrix and the updated audio matrix using a preset convolution method to obtain the convolutional time-domain vector, and then obtaining the target output vector from the initial audio vector and the convolutional time-domain vector, this invention achieves reduced computational load while maintaining reverberation effects. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of the audio processing device in the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the audio processing method of the present invention; Figure 3 This is a flowchart illustrating the second embodiment of the audio processing method of the present invention; Figure 4 This is a structural block diagram of the first embodiment of the audio processing device of the present invention.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] Reference Figure 1 , Figure 1 This is a schematic diagram of the audio processing device structure of the hardware operating environment involved in the embodiments of the present invention.

[0020] like Figure 1As shown, the audio processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0021] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the audio processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0022] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an audio processing program.

[0023] exist Figure 1 In the audio processing device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the audio processing device of the present invention can be set in the audio processing device, and the audio processing device calls the audio processing program stored in the memory 1005 through the processor 1001 and executes the audio processing method provided in the embodiment of the present invention.

[0024] This invention provides an audio processing method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of an audio processing method according to the present invention.

[0025] In this embodiment, the audio processing method includes the following steps: Step S10: Determine the transfer function matrix based on the preset window function, short-time Fourier transform method, and room impact reverberation.

[0026] It should be noted that the execution subject in this embodiment is an audio processing device, which has functions such as data processing, data communication and program execution. The audio processing device can be an integrated controller, a control computer or other devices, or other devices with similar functions. This embodiment does not limit the scope of the embodiments.

[0027] It is understandable that the preset window function refers to the pre-set Hanning window function, the room impact reverberation is the reverberation signal obtained through acoustic recording or mirroring, and the transfer function matrix refers to the matrix obtained by performing a short-time Fourier transform and windowing on the room impact reverberation.

[0028] In practice, the room impact reverberation is processed according to the pre-set Hanning window function and short-time Fourier transform method to obtain the transfer function matrix.

[0029] It should be noted that, in order to accurately obtain the transfer function matrix, the determination of the transfer function matrix based on the preset window function, the short-time Fourier transform method, and the room impact reverberation further includes: performing overlapping processing on the room impact reverberation according to the preset data overlapping method to obtain multiple overlapping vectors; performing windowing processing on the multiple overlapping vectors according to the preset window function to obtain multiple windowed vectors; performing format conversion on the multiple windowed vectors according to the short-time Fourier transform method to obtain multiple frequency domain vectors; and arranging the multiple frequency domain vectors according to the preset arrangement method to obtain the transfer function matrix.

[0030] It is understandable that the preset data overlap method refers to the way the room impact reverberation (RIR) data is processed in advance, multiple overlap vectors refer to multiple vectors with data overlap, multiple windowed vectors refer to multiple windowed overlap vectors, multiple frequency domain vectors refer to multiple windowed vectors converted from the time domain to the frequency domain, and the preset arrangement method refers to the way multiple frequency domain vectors are arranged in advance.

[0031] In practice, the room impact reverberation data is processed according to a pre-defined method for processing room impact reverberation data, resulting in multiple vectors with data overlap. Then, the multiple overlapping vectors are windowed according to a pre-defined Hanning window function, resulting in multiple windowed overlapping vectors. The multiple windowed vectors are then format-converted according to a short-time Fourier transform method, resulting in multiple windowed vectors converted from the time domain to the frequency domain. Finally, the multiple frequency domain vectors are arranged according to a pre-defined method for arranging multiple frequency domain vectors, resulting in the transfer function matrix.

[0032] It should be noted that STFT is used to process RIR, generating an RIR frequency domain matrix. The RIR data is then overlapped with the music data, r(1) = [r(1), r(2)...r(2048)], r(2) = [r(1025), r(1026)...r(3072)], and so on until all RIR data is collected. This yields three overlapping vectors r(1), r(2), and r(3), each with 1024 data points. A short-time Fourier transform is then performed, first multiplying each frame's RIR vector by the corresponding window function: r w (1)=r(1)⊙w hanning Where ⊙ represents the Hadamard product, which is the product of corresponding data points, followed by an FFT transform to obtain the frequency domain signal r. FFT (1) This step can be performed using the DSP's built-in FFT function. Using this method, r can be obtained sequentially. FFT (2) and r FFT (3); Arrange the windowed vectors into a transfer function matrix. .

[0033] Step S20: Update the initial audio matrix corresponding to the audio signal based on the multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix.

[0034] It is understandable that multiple audio frame data refers to the data after the audio signal has been discretized, the initial audio matrix refers to the audio matrix with all elements being 0, and the updated audio matrix refers to the initial audio matrix updated by multiple audio frame data.

[0035] In practice, the audio matrix with all elements set to 0 is updated based on multiple audio frame data corresponding to the audio signal, resulting in the updated audio matrix.

[0036] It should be noted that, in order to accurately obtain the updated audio matrix, the step of updating the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix further includes: windowing multiple audio frame data according to a preset window function to obtain multiple windowed frame data; converting the format of multiple windowed frame data according to a short-time Fourier transform to obtain multiple audio frame signals; and updating the initial audio matrix corresponding to the audio signal according to a preset update method and multiple audio frame signals to obtain the updated audio matrix.

[0037] It is understandable that multiple windowed frame data refers to multiple windowed audio frame data, multiple audio frame signals refer to multiple windowed frame data converted from the time domain to the frequency domain, and the preset update method refers to the pre-set update matrix data method.

[0038] In practice, multiple audio frame data are processed according to a pre-set Hanning window function to obtain multiple windowed audio frame data. Then, multiple windowed frame data are processed by short-time Fourier transform to obtain multiple windowed frame data converted from the time domain to the frequency domain. Finally, the initial audio matrix is ​​updated according to the multiple audio frame signals using a pre-set update matrix data method to obtain the updated audio matrix.

[0039] It should be noted that a single frame of data is input, and windowing and short-time Fourier transform are performed. For example, if the first frame of data is input, m is obtained. w (1)=m(1)⊙w hanning Then, the frequency domain signal m is obtained by FFT transformation. FFT (1) The same logic applies to other frames, updating the music data buffer matrix M by continuously discarding the last row of data and putting the new data into the first row. For the first frame signal... For the second frame signal, For the i-th frame signal, .

[0040] Step S30: Perform convolution processing on the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector.

[0041] It is understandable that the preset convolution method refers to the pre-defined frequency domain convolution method, and the convolution time domain vector refers to the vector obtained by processing the transfer function matrix and the updated audio matrix through convolution. The convolution time domain vector is obtained by processing the transfer function matrix and the updated audio matrix through the pre-defined frequency domain convolution method.

[0042] Step S40: Obtain the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector.

[0043] It is understandable that the initial audio vector refers to an initial vector in which all elements are 0, and the target output vector refers to the vector obtained by reverberation processing of the audio signal. The target output vector is calculated based on the initial audio vector and the convolutional time-domain vector.

[0044] It should be noted that the music data buffer matrix is ​​initialized. Its column count is 2048, which is the frame length of the STFT. The row count is related to the RIR length and frame shift. It is easy to see that the parameter value in this embodiment is 3, that is, the initial value of M is a 3*2048 matrix with all elements being 0. It can be seen that M is a 3*2048 matrix, which is exactly the same size as matrix R. The output buffer vector is initialized, y buf=[0,0...0], its length is equal to the frame shift, i.e., 1024.

[0045] It should be noted that, assuming the music signal is m(t), where m represents music and t represents time, digital signals cannot be represented by a continuous signal m(t) in the time domain. Therefore, a discrete domain signal m(n) is used, where n=1,2,3L represents the sampling points. In this embodiment, the frame length is 2048, and the frame shift is 1024 points, meaning that each frame of data is 2048. After each operation, the input is updated by 1024 points, and the output is also 1024 points (corresponding to these updated 1024 points). For example, the first frame of data m(1)=[m(1),m(2)...m(2048)], the second frame of data m(1)=[m(1),m(2)...m(2048)], the third frame of data m(1)=[m(1),m(2),...m(2048)], the fourth frame of data m(1)=[m(1),m(2),...m(2048)], the fifth frame of data m(1),m(2),...m(2048)], the sixth frame of data m(1),m(2),...m(2048) ... Frame data m(2) = [m(1025), m(1026)...m(3072)], and so on, can be summarized as: m(j) = [m(i), m(i+1)...m(i+2047)], where i = 1, 1025, 3073..., j = 1, 2, 3... represent the frame index. Assuming the RIR signal vector used is r = [r(1), r(2), r(3)...r(L)], where L represents the length of the RIR, for ease of explanation, this embodiment takes L = 4096. In the STFT operation process, the signal needs to be windowed. This patent uses a Hanning window, the value of which is w. hanning =[w(1),w(2),w(3)...w(K)], where K represents the length of the window function, which needs to be consistent with the frame length of the STFT, i.e., K = 2048.

[0046] This embodiment determines the transfer function matrix based on a preset window function, a short-time Fourier transform, and room impact reverberation. It updates the initial audio matrix corresponding to the audio signal using multiple audio frame data, obtaining an updated audio matrix. A preset convolution method is then applied to the transfer function matrix and the updated audio matrix to obtain a convolutional time-domain vector. Finally, the target output vector is obtained from the initial audio vector and the convolutional time-domain vector. By using a preset window function and short-time Fourier transform to process room impact reverberation and obtain the transfer function matrix, processing the transfer function matrix and the updated audio matrix using a preset convolution method to obtain the convolutional time-domain vector, and then obtaining the target output vector from the initial audio vector and the convolutional time-domain vector, this approach reduces computational load while maintaining reverberation quality.

[0047] refer to Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of an audio processing method according to the present invention.

[0048] Based on the first embodiment described above, the audio processing method of this embodiment includes the following in step S30: Step S31: Calculate the transfer function matrix and the updated audio matrix according to the preset calculation method to obtain the hybrid calculation matrix.

[0049] It is understandable that the preset calculation method refers to the pre-defined Hadamard product calculation method, and the hybrid calculation matrix refers to the matrix obtained by calculating the transfer function matrix and the updated audio matrix through the Hadamard product. The Hadamard product is calculated on the transfer function matrix and the updated audio matrix through the preset Hadamard product calculation method, and the hybrid calculation matrix is ​​obtained based on the calculation result.

[0050] Step S32: Obtain the hybrid calculation vector based on the hybrid calculation matrix.

[0051] It is understandable that a hybrid computation vector refers to the vector corresponding to a hybrid computation matrix.

[0052] It should be noted that, in order to accurately obtain the hybrid calculation vector, the step of obtaining the hybrid calculation vector based on the hybrid calculation matrix further includes: obtaining the data corresponding to each column in the hybrid calculation matrix based on the hybrid calculation matrix; performing calculations on the data corresponding to each column in the hybrid calculation matrix to obtain multiple calculation data; and combining the multiple calculation data to obtain the hybrid calculation vector.

[0053] It is understandable that the data corresponding to each column refers to the data in each column of the hybrid calculation matrix, and multiple calculated data refers to multiple data obtained from the data in each column of the hybrid calculation matrix.

[0054] In practice, the data corresponding to each column in the hybrid calculation matrix is ​​determined based on the obtained hybrid calculation matrix. Then, the data corresponding to each column are added together to obtain multiple calculation data. Finally, the multiple calculation data are combined to obtain the hybrid calculation vector.

[0055] Step S33: Determine the convolution time-domain vector based on the preset window function, the inverse short-time Fourier transform method, and the hybrid calculation vector.

[0056] It is understandable that a convolutional time-domain vector refers to a time-domain vector obtained by processing a hybrid computation vector. The hybrid computation vector is processed by a pre-defined Hanning window function and an inverse short-time Fourier transform to obtain a vector that is converted from the frequency domain to the time domain.

[0057] It should be noted that, in order to accurately obtain the target output vector, the step of obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector further includes: truncating the convolutional time-domain vector according to a first truncation method to obtain a first truncation vector; calculating the initial audio vector corresponding to the audio signal and the first truncation vector according to a preset operation method; and obtaining the target output vector based on the calculation result.

[0058] It is understandable that the first truncation method refers to the pre-defined method of truncating the first 1024 data points of the convolutional temporal vector, and the first truncation vector refers to the vector with the first 1024 data points of the convolutional temporal vector.

[0059] In practice, the convolutional time-domain vector is truncated by pre-setting the method of extracting the first 1024 data points of the convolutional time-domain vector, resulting in a vector with the first 1024 data points of the convolutional time-domain vector. Finally, the target output vector is obtained by adding the initial audio vector corresponding to the audio signal and the first truncated vector.

[0060] It should be noted that, in order to update the initial audio vector, after obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector, the method further includes: truncating the convolutional time-domain vector according to the second truncation method to obtain a second truncation vector; assigning a value to the initial audio vector corresponding to the audio signal based on the second truncation vector; and obtaining the assigned audio vector corresponding to the audio signal based on the assignment result.

[0061] It is understandable that the second truncation method refers to the pre-defined method of truncating the last 1024 data points of the convolutional temporal vector, the second truncation vector refers to the vector with the last 1024 data points of the convolutional temporal vector, and the assigned audio vector refers to the updated initial audio vector.

[0062] In practice, the convolutional time-domain vector is truncated by pre-setting a method of extracting the last 1024 data points of the convolutional time-domain vector, resulting in a vector with the last 1024 data points of the convolutional time-domain vector. Finally, the second truncated vector is assigned to the initial audio vector corresponding to the audio signal to obtain the updated initial audio vector.

[0063] It should be noted that the music data buffer matrix M and the transfer function matrix R are convolved in the frequency domain and subjected to inverse short-time Fourier transform. Their Hadamard product, Y = M⊙R, is then calculated. The vector y is obtained by adding the matrix Y row by row. sum =sum(Y,1) sums the three elements of each column of matrix Y, resulting in a vector y of length 2048. sum ; for the obtained vector y sumWindowing and iFFT operations are performed to obtain the time-domain signal y. iFFT Alternatively, the DSP's built-in iFFT function can be used. Although the frame length is 2048, it can be seen that each update length is a frame shift of 1024 points, so the output of each frame is also 1024 points of data. The final output vector is y. out =y buf +y iFFT (1, 2...1024), i.e., y buf Add y iFFT The first 1024 data points are used to output a buffer vector, y. buf =y iFFT (1025, 1026...2048), y is about to be... iFFT The last 1024 points are assigned to y buf Thus, the entire DSP system has completed one input and output cycle.

[0064] It should be noted that, assuming each frame of the data stream is 1024, the signal sampling rate is 48 kHz, and the RIR length is 49152 (reverberation time is approximately 1.03 s), if an FIR filter is used for convolutional reverberation, then each frame of the signal output will require (49152 + 49152 - 1). 1024 is approximately 100,000,000 FLOPs. Using the method described in this patent, each frame is 2048 pixels long with a frame shift of 1024, requiring a buffer of 47 frames. Without considering the computational complexity of FFT, iFFT, and buffer updates, the final output of 1024 points per frame requires 2048 frames. 47 + 2048 46 is approximately 190,000 FLOPs. It can be seen that, in terms of FLOPs, the computational cost of temporal convolutional reverberation is about 500 times greater than that of the fast reverberation method based on STFT. However, in practice, FFT, iFFT and buffer updates also require a certain amount of computing power. After testing on ADSP-SC594, the final computational cost was reduced by about 10 times, while the reverberation effect of the two was very close.

[0065] This embodiment calculates the transfer function matrix and the updated audio matrix according to a preset calculation method to obtain a hybrid calculation matrix; obtains a hybrid calculation vector based on the hybrid calculation matrix; and determines the convolution time-domain vector based on a preset window function, an inverse short-time Fourier transform method, and the hybrid calculation vector. By calculating the transfer function matrix and the updated audio matrix using a preset calculation method to obtain the hybrid calculation matrix, then obtaining the hybrid calculation vector based on the hybrid calculation matrix, and finally obtaining the convolution time-domain vector based on the preset window function, the inverse short-time Fourier transform method, and the hybrid calculation vector, the calculation efficiency is improved and the reverberation effect is guaranteed.

[0066] Furthermore, embodiments of the present invention also propose a storage medium storing an audio processing program, which, when executed by a processor, implements the steps of the audio processing method described above.

[0067] Reference Figure 4 , Figure 4 This is a structural block diagram of the first embodiment of the audio processing device of the present invention.

[0068] like Figure 4 As shown, the audio processing apparatus proposed in this embodiment of the invention includes: Processing module 10 is used to determine the transfer function matrix based on a preset window function, a short-time Fourier transform method, and room impact reverberation.

[0069] The update module 20 is used to update the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal, so as to obtain the updated audio matrix.

[0070] The processing module 10 is further configured to perform convolution processing on the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector.

[0071] The output module 30 is used to obtain the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector.

[0072] This embodiment determines the transfer function matrix based on a preset window function, a short-time Fourier transform, and room impact reverberation. It updates the initial audio matrix corresponding to the audio signal using multiple audio frame data, obtaining an updated audio matrix. A preset convolution method is then applied to the transfer function matrix and the updated audio matrix to obtain a convolutional time-domain vector. Finally, the target output vector is obtained from the initial audio vector and the convolutional time-domain vector. By using a preset window function and short-time Fourier transform to process room impact reverberation and obtain the transfer function matrix, processing the transfer function matrix and the updated audio matrix using a preset convolution method to obtain the convolutional time-domain vector, and then obtaining the target output vector from the initial audio vector and the convolutional time-domain vector, this approach reduces computational load while maintaining reverberation quality.

[0073] In one embodiment, the processing module 10 is further configured to perform overlapping processing on the room impact reverberation according to a preset data overlapping method to obtain multiple overlapping vectors; Multiple overlapping vectors are windowed according to a preset window function to obtain multiple windowed vectors; Multiple windowed vectors are converted into multiple frequency domain vectors using the short-time Fourier transform method. Multiple frequency domain vectors are arranged according to a preset arrangement to obtain the transfer function matrix.

[0074] In one embodiment, the update module 20 is further configured to perform windowing processing on multiple audio frame data according to a preset window function to obtain multiple windowed frame data; Multiple windowed frame data are converted into multiple audio frame signals using the short-time Fourier transform method. The initial audio matrix corresponding to the audio signal is updated according to the preset update method and multiple audio frame signals to obtain the updated audio matrix.

[0075] In one embodiment, the processing module 10 is further configured to calculate the transfer function matrix and the updated audio matrix according to a preset calculation method to obtain a hybrid calculation matrix; The hybrid calculation vector is obtained based on the hybrid calculation matrix; The convolution time-domain vector is determined based on the preset window function, the inverse short-time Fourier transform method, and the hybrid computation vector.

[0076] In one embodiment, the processing module 10 is further configured to obtain data corresponding to each column in the hybrid calculation matrix based on the hybrid calculation matrix; Calculate the data corresponding to each column in the hybrid calculation matrix to obtain multiple calculated data; Multiple computational data are combined to obtain a hybrid computational vector.

[0077] In one embodiment, the output module 30 is further configured to truncate the convolutional temporal vector according to the first truncation method to obtain a first truncation vector; The initial audio vector and the first truncated vector corresponding to the audio signal are calculated according to the preset calculation method; The target output vector is obtained based on the calculation results.

[0078] In one embodiment, the output module 30 is further configured to truncate the convolutional temporal vector according to the second truncation method to obtain a second truncation vector; The initial audio vector corresponding to the audio signal is assigned a value based on the second intercepted vector; The assigned audio vector corresponding to the audio signal is obtained based on the assignment result.

[0079] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0080] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0081] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0082] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0083] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0085] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An audio processing method, characterized in that, The audio processing method includes: The transfer function matrix is ​​determined based on the preset window function, short-time Fourier transform method, and room impact reverberation. The initial audio matrix corresponding to the audio signal is updated based on multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix; The transfer function matrix and the updated audio matrix are convolved using a preset convolution method to obtain a convolutional temporal vector. The target output vector is obtained based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector. The determination of the transfer function matrix based on a preset window function, short-time Fourier transform, and room impact reverberation includes: The room's impact reverberation is overlaid according to a preset data overlay method to obtain multiple overlay vectors; Multiple overlapping vectors are windowed according to a preset window function to obtain multiple windowed vectors; Multiple windowed vectors are converted into multiple frequency domain vectors using the short-time Fourier transform method. Multiple frequency domain vectors are arranged according to a preset arrangement to obtain a transfer function matrix; The step of updating the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix includes: Windowing is applied to multiple audio frame data according to a preset window function to obtain multiple windowed frame data; Multiple windowed frame data are converted into multiple audio frame signals using the short-time Fourier transform method. The initial audio matrix corresponding to the audio signal is updated according to the preset update method and multiple audio frame signals to obtain the updated audio matrix; The step of convolving the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector includes: The transfer function matrix and the updated audio matrix are calculated according to a preset calculation method to obtain a hybrid calculation matrix; The hybrid calculation vector is obtained based on the hybrid calculation matrix; The convolution time-domain vector is determined based on the preset window function, the inverse short-time Fourier transform method, and the hybrid calculation vector. The step of obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional temporal vector includes: The first truncation vector is obtained by truncating the data of the convolutional temporal vector according to the first truncation method. The initial audio vector and the first truncated vector corresponding to the audio signal are calculated according to the preset calculation method; The target output vector is obtained based on the calculation results.

2. The method as described in claim 1, characterized in that, The step of obtaining the hybrid calculation vector based on the hybrid calculation matrix includes: The data corresponding to each column in the hybrid calculation matrix is ​​obtained based on the hybrid calculation matrix; Calculate the data corresponding to each column in the hybrid calculation matrix to obtain multiple calculated data; Multiple computational data are combined to obtain a hybrid computational vector.

3. The method as described in claim 1, characterized in that, After obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector, the method further includes: The convolutional temporal vector is truncated according to the second truncation method to obtain the second truncated vector; The initial audio vector corresponding to the audio signal is assigned a value based on the second intercepted vector; The assigned audio vector corresponding to the audio signal is obtained based on the assignment result.

4. An audio processing device, characterized in that, The audio processing device includes: The processing module is used to determine the transfer function matrix based on the preset window function, short-time Fourier transform method, and room impact reverberation. The update module is used to update the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal, so as to obtain the updated audio matrix. The processing module is also used to perform convolution processing on the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector; The output module is used to obtain the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional time-domain vector; The determination of the transfer function matrix based on a preset window function, short-time Fourier transform, and room impact reverberation includes: The room's impact reverberation is overlaid according to a preset data overlay method to obtain multiple overlay vectors; Multiple overlapping vectors are windowed according to a preset window function to obtain multiple windowed vectors; Multiple windowed vectors are converted into multiple frequency domain vectors using the short-time Fourier transform method. Multiple frequency domain vectors are arranged according to a preset arrangement to obtain a transfer function matrix; The step of updating the initial audio matrix corresponding to the audio signal based on multiple audio frame data corresponding to the audio signal to obtain the updated audio matrix includes: Windowing is applied to multiple audio frame data according to a preset window function to obtain multiple windowed frame data; Multiple windowed frame data are converted into multiple audio frame signals using the short-time Fourier transform method. The initial audio matrix corresponding to the audio signal is updated according to the preset update method and multiple audio frame signals to obtain the updated audio matrix; The step of convolving the transfer function matrix and the updated audio matrix based on a preset convolution method to obtain a convolutional temporal vector includes: The transfer function matrix and the updated audio matrix are calculated according to a preset calculation method to obtain a hybrid calculation matrix; The hybrid calculation vector is obtained based on the hybrid calculation matrix; The convolution time-domain vector is determined based on the preset window function, the inverse short-time Fourier transform method, and the hybrid calculation vector. The step of obtaining the target output vector based on the initial audio vector corresponding to the audio signal and the convolutional temporal vector includes: The first truncation vector is obtained by truncating the data of the convolutional temporal vector according to the first truncation method. The initial audio vector and the first truncated vector corresponding to the audio signal are calculated according to the preset calculation method; The target output vector is obtained based on the calculation results.

5. An audio processing device, characterized in that, The device includes: a memory, a processor, and an audio processing program stored in the memory and executable on the processor, the audio processing program being configured to implement the steps of the audio processing method as described in any one of claims 1 to 3.

6. A storage medium, characterized in that, The storage medium stores an audio processing program, which, when executed by a processor, implements the steps of the audio processing method as described in any one of claims 1 to 3.

Images