Reverberation processing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of audio processing, and relates to a reverberation processing method and device, computer equipment and a storage medium, and the method comprises the steps: receiving a to-be-processed audio signal sent by a user terminal; calling an adaptive reverberation filter, and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal; and performing conversion processing on the convolution frequency domain signal to obtain a target audio signal. According to the method, convolution operation is converted into multiplication operation in a frequency domain, fast Fourier transform is carried out by using an FFT algorithm, the calculation complexity is reduced to O (NlogN), and the calculation amount is effectively reduced, so that the processing speed is remarkably improved, and the processing delay is reduced.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a reverberation processing method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of digital audio processing technology, reverberation processing has become an indispensable part of audio effect processing. Reverberation refers to the process of sound gradually attenuating after being reflected on different surfaces, which can simulate different spatial senses and sound depths. Reverberation processing technology is widely used in music production, movie sound effects, virtual reality, game sound effects and communication systems, etc., which enhances the audio listening experience.

[0003] Traditional reverberation processing methods rely on convolution operations, which require a lot of computing resources. Especially in real-time processing scenarios, they are prone to high latency and are difficult to meet real-time requirements. Therefore, traditional reverberation processing methods have the problems of low processing efficiency and high processing latency. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a reverberation processing method, apparatus, computer equipment and storage medium to solve the problems of low processing efficiency and high processing delay in traditional reverberation processing methods.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a reverberation processing method, which adopts the following technical solution: Receiving an audio signal to be processed sent by a user terminal; An adaptive reverberation filter is called, and convolution processing is performed on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as:

[0006] Wherein, FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; Performing conversion processing on the convolution frequency domain signal to obtain a target audio signal; The target audio signal is output to the user terminal.

[0007] Further, after the step of receiving the audio signal to be processed sent by the user terminal and before the step of calling the adaptive reverberation filter and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and the fast convolution algorithm to obtain the convolution frequency domain signal, the following steps are also included: Get the trained deep neural network; Inputting the audio signal to be processed into the trained deep neural network for reverberation parameter adjustment training processing to obtain filter parameter adjustment values; An initial reverberation filter is called, and parameter adjustment processing is performed on the initial reverberation filter according to the filter parameter adjustment value to obtain the adaptive reverberation filter.

[0008] Furthermore, the step of obtaining the trained deep neural network specifically includes the following steps: Acquire training audio data and standard parameter adjustment values ​​corresponding to the training audio data; Calling an initial deep neural network, wherein the initial deep neural network includes a convolutional layer, a residual network module, an attention mechanism module, a pooling layer, a fully connected layer, and an output layer; Performing feature extraction processing on the training audio data according to the convolution layer to obtain training audio features; Performing feature enhancement processing on the training audio features according to the residual network module to obtain training audio enhanced features; Performing feature screening processing on the training audio enhancement feature according to the attention mechanism module to obtain a training audio screening feature; Performing feature dimensionality reduction processing on the training audio screening features according to the pooling layer to obtain training audio dimensionality reduction features; Performing feature fusion processing on the training audio features and the training audio dimensionality reduction features according to the fully connected layer to obtain training audio fusion features; Perform parameter adjustment value prediction processing on the training audio fusion feature according to the output layer to obtain a training parameter adjustment value; The initial deep neural network is subjected to parameter adjustment processing according to the training parameter adjustment value, the standard parameter adjustment value and a preset loss function until the parameters of the initial deep neural network meet convergence conditions, thereby obtaining the trained deep neural network.

[0009] Furthermore, the step of calling the adaptive reverberation filter and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and the fast convolution algorithm to obtain the convolution frequency domain signal specifically includes the following steps: If the audio signal to be processed is a multi-channel input audio signal, performing fast Fourier transform on the audio time domain representation of the audio signal to be processed of each channel and the filter time domain representation of the adaptive filter respectively; Performing parallel convolution operation on the audio time domain representation after fast Fourier transformation and the filter time domain representation to obtain a parallel convolution operation result; Performing inverse fast Fourier transform on the parallel convolution operation results according to the original channels respectively to obtain inverse fast Fourier transform results of each channel; The inverse fast Fourier transform results of each channel are synthesized to obtain the convolution frequency domain signal.

[0010] In order to solve the above technical problems, the embodiment of the present application further provides a reverberation processing device, which adopts the following technical solution: The to-be-processed audio acquisition module is used to receive the to-be-processed audio signal sent by the user terminal; A convolution module is used to call an adaptive reverberation filter and perform convolution processing on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as:

[0011] Wherein, FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; A conversion module, used for converting the convolution frequency domain signal to obtain a target audio signal; An output module is used to output the target audio signal to the user terminal.

[0012] Furthermore, the device also includes: Neural network acquisition module, used to obtain the trained deep neural network; A parameter adjustment value acquisition module, used for inputting the audio signal to be processed into the trained deep neural network for reverberation parameter adjustment training processing to obtain a filter parameter adjustment value; The parameter adjustment module is used to call the initial reverberation filter and perform parameter adjustment processing on the initial reverberation filter according to the filter parameter adjustment value to obtain the adaptive reverberation filter.

[0013] Furthermore, the neural network acquisition module includes: A training data acquisition submodule, used to acquire training audio data and standard parameter adjustment values ​​corresponding to the training audio data; A neural network calling submodule, used to call an initial deep neural network, wherein the initial deep neural network includes a convolutional layer, a residual network module, an attention mechanism module, a pooling layer, a fully connected layer, and an output layer; A feature extraction submodule, used for performing feature extraction processing on the training audio data according to the convolution layer to obtain training audio features; A feature enhancement submodule, used for performing feature enhancement processing on the training audio features according to the residual network module to obtain training audio enhanced features; A feature screening submodule, used for performing feature screening processing on the training audio enhancement feature according to the attention mechanism module to obtain a training audio screening feature; A feature dimension reduction submodule, used for performing feature dimension reduction processing on the training audio screening features according to the pooling layer to obtain training audio dimension reduction features; A feature fusion submodule, used for performing feature fusion processing on the training audio features and the training audio dimensionality reduction features according to the fully connected layer to obtain training audio fusion features; An adjustment value prediction submodule, used for performing parameter adjustment value prediction processing on the training audio fusion feature according to the output layer to obtain a training parameter adjustment value; The convergence confirmation submodule is used to perform parameter adjustment processing on the initial deep neural network according to the training parameter adjustment value, the standard parameter adjustment value and the preset loss function until the parameters of the initial deep neural network meet the convergence conditions to obtain the trained deep neural network.

[0014] Furthermore, the convolution module includes: A fast Fourier transform submodule, configured to perform fast Fourier transform on the audio time domain representation of the audio signal to be processed of each channel and the filter time domain representation of the adaptive filter respectively if the audio signal to be processed is a multi-channel input audio signal; A parallel convolution submodule is used to perform parallel convolution operations on the audio time domain representation after fast Fourier transformation and the filter time domain representation to obtain parallel convolution operation results; An inverse fast Fourier transform result submodule is used to perform inverse fast Fourier transform on the parallel convolution operation results according to the original channels respectively to obtain inverse fast Fourier transform results of each channel; The synthesis submodule is used to perform synthesis processing on the inverse fast Fourier transform results of each channel to obtain the convolution frequency domain signal.

[0015] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the reverberation processing method as described above when executing the computer-readable instructions.

[0016] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the reverberation processing method described above are implemented.

[0017] The present application provides a reverberation processing method, comprising: receiving an audio signal to be processed sent by a user terminal; calling an adaptive reverberation filter, and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as: , wherein FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; the convolved frequency domain signal is converted and processed to obtain a target audio signal; and the target audio signal is output to the user terminal. Compared with the prior art, the present application reduces the computational complexity to O(NlogN) by converting the convolution operation into a multiplication operation in the frequency domain and using the FFT algorithm for fast Fourier transform, thereby effectively reducing the amount of calculation, thereby significantly improving the processing speed and reducing the processing delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 is an exemplary system architecture diagram to which the present application may be applied; Figure 2 is a flowchart of the implementation of the reverberation processing method provided in Example 1 of the present application; Figure 3 It is a structural diagram of a specific implementation method of the convolution operation provided in Example 1 of the present application; Figure 4 is a structural schematic diagram of a reverberation processing device provided in Embodiment 2 of the present application; Figure 5 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0021] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0023] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0024] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0025] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0026] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0027] It should be noted that the reverberation processing method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the reverberation processing device is generally arranged in the server / terminal device.

[0028] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0029] Embodiment 1 Continue to refer Figure 2 , shows a flow chart of an embodiment of a reverberation processing method according to the present application. The reverberation processing method comprises: step S201, step S202, step S203 and step S204.

[0030] In step S201, an audio signal to be processed sent by a user terminal is received.

[0031] In the embodiments of the present application, the user terminal refers to a terminal device used to execute the image processing method for preventing document abuse provided by the present application. The user terminal can be a mobile terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, etc., as well as a fixed terminal such as a digital TV, a desktop computer, etc. It should be understood that the examples of user terminals here are only for the convenience of understanding and are not used to limit the present application.

[0032] In the embodiment of the present application, after obtaining the input audio signal to be processed, it is necessary to convert the input audio signal to be processed into a frequency domain representation.

[0033] In step S202, the adaptive reverberation filter is called, and the audio signal to be processed is convolved according to the adaptive reverberation filter and the fast convolution algorithm to obtain a convolved frequency domain signal, wherein the fast convolution algorithm is expressed as:

[0034] Among them, FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal.

[0035] In some optional implementations of the embodiments of the present application, FFT and IFFT are essentially approximate calculations, which are prone to introduce signal loss, especially in low-precision or low sampling rate conditions, thereby affecting the accuracy of the reverberation effect.

[0036] In the embodiment of the present application, a weighted compensation term of high-precision convolution calculation can be introduced to reduce the error in FFT / IFFT conversion. Specifically, after FFT calculation, an error compensation term is added. To improve the accuracy of frequency domain convolution calculation, as follows: In the convolution formula:

[0037] On the basis of , modified to:

[0038] in, It can be obtained by the following formula:

[0039] in, Indicates the compensation coefficient, which is used to control the correction strength; Represents the approximation of the frequency domain signal after convolution.

[0040] In the embodiment of the present application, the above compensation item can learn a suitable correction coefficient by training a neural network.

[0041] In the embodiment of the present application, the present application innovatively adopts an improved fast convolution algorithm to perform convolution processing on the input audio signal, wherein the fast convolution algorithm includes the following formula optimization:

[0042] in, is the frequency domain representation of the output audio signal, is the frequency domain representation of the input audio signal, is the frequency domain representation of the reverberation filter.

[0043] Specifically, the improved fast convolution algorithm is implemented by the following formula:

[0044] Wherein, FFT stands for Fast Fourier Transform, IFFT stands for Inverse Fast Fourier Transform, and x and y are the time domain representations of the input signal and the adaptive filter, respectively.

[0045] In the embodiment of the present application, through FFT optimization, the computational complexity is reduced from O(N2) of traditional convolution to O(NlogN) In step S203, the convolution frequency domain signal is converted to obtain a target audio signal.

[0046] In an embodiment of the present application, the frequency domain signal after convolution processing is converted back into a time domain signal to generate a final output audio signal.

[0047] In step S204, the target audio signal is output to the user terminal.

[0048] In an embodiment of the present application, a reverberation processing method is provided, comprising: receiving an audio signal to be processed sent by a user terminal; calling an adaptive reverberation filter, and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as: , where FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; the convolved frequency domain signal is converted and processed to obtain a target audio signal; and the target audio signal is output to the user terminal. Compared with the prior art, the present application reduces the computational complexity to O(NlogN) by converting the convolution operation into a multiplication operation in the frequency domain and using the FFT algorithm for fast Fourier transform, thereby effectively reducing the amount of calculation, thereby significantly improving the processing speed and reducing the processing delay.

[0049] In some optional implementations of the embodiments of the present application, after step S201 and before step S202, the following steps are further included: Get the trained deep neural network; Inputting the audio signal to be processed into the trained deep neural network for reverberation parameter adjustment training processing to obtain the filter parameter adjustment value; The initial reverberation filter is called, and the parameters of the initial reverberation filter are adjusted according to the filter parameter adjustment value to obtain an adaptive reverberation filter.

[0050] In an embodiment of the present application, a machine learning algorithm is used to automatically adjust reverberation filter parameters according to the characteristics of the input audio signal to optimize the reverberation effect.

[0051] In some optional implementations of the embodiments of the present application, the above step of obtaining a trained deep neural network specifically includes the following steps: Acquire training audio data and standard parameter adjustment values ​​corresponding to the training audio data; Calling the initial deep neural network, where the initial deep neural network includes a convolutional layer, a residual network module, an attention mechanism module, a pooling layer, a fully connected layer, and an output layer; Perform feature extraction processing on the training audio data according to the convolution layer to obtain training audio features; Perform feature enhancement processing on the training audio features according to the residual network module to obtain the training audio enhanced features; Perform feature screening processing on the training audio enhancement features according to the attention mechanism module to obtain the training audio screening features; Perform feature dimensionality reduction processing on the training audio screening features according to the pooling layer to obtain the training audio dimensionality reduction features; Perform feature fusion processing on the training audio features and the training audio dimensionality reduction features according to the fully connected layer to obtain the training audio fusion features; Perform parameter adjustment value prediction processing on the training audio fusion features according to the output layer to obtain the training parameter adjustment value; The parameters of the initial deep neural network are adjusted according to the training parameter adjustment values, the standard parameter adjustment values ​​and the preset loss function until the parameters of the initial deep neural network meet the convergence conditions, thereby obtaining a trained deep neural network.

[0052] In the embodiment of the present application, in order to realize adaptive reverberation parameter adjustment, the present application innovatively proposes an innovative deep neural network architecture, including the following main parts: ① Input layer: MFCC features of the input audio signal, with a dimension of D.

[0053] ② Convolutional layer: Use multiple convolutional layers to extract high-level representations of audio features. Assume that the size of the convolution kernel in the i-th layer is , the output of the convolutional layer is:

[0054] in, is the convolution kernel weight of the i-th layer, F is the input feature, is the bias, ∗ represents the convolution operation, and ReLU is the activation function.

[0055] ③Residual network (ResNet) module: Introduces the residual network structure to solve the degradation problem in deep network training and enhance feature extraction capabilities.

[0056]

[0057] In the embodiment of the present application, the residual network (ResNet) is introduced to solve the gradient vanishing and degradation problems in deep network training. When the number of layers of traditional deep neural networks is increased, the model training may become difficult or the performance may degrade due to the gradient vanishing or gradient exploding problems. The residual network introduces short-circuit connections to allow the network layers to directly transmit information, thereby alleviating these problems and allowing deeper networks to be effectively trained, thereby improving the network's feature extraction capabilities.

[0058] ④Attention mechanism module: Use the self-attention mechanism to enhance the network's selective attention to different features and improve the accuracy of feature expression.

[0059]

[0060] where, denote the query, key, and value vectors, respectively, and denotes the dimension of the key vector.

[0061] In the embodiment of the present application, the self-attention mechanism is used to enhance the network's selective attention to different features, thereby improving the accuracy and robustness of feature expression. The attention mechanism allows the network to adaptively adjust the degree of attention to different features according to the context when processing audio features, thereby improving the quality of feature representation and the overall performance of the model.

[0062] ⑤Pooling layer: Use the maximum pooling layer to reduce the feature dimension and obtain a more compact feature representation.

[0063]

[0064] In the embodiment of the present application, multi-layer convolution and pooling layers are used to extract multi-level representations of audio features, thereby improving the network's ability to capture audio features. Multi-layer convolution can capture richer local features, while pooling layers can reduce feature dimensions while retaining important information, thereby improving the generalization ability and processing efficiency of the model.

[0065] ⑥Fully connected layer: Expand the output of the convolutional layer and the pooling layer and process it through the fully connected layer. Assuming the weight of the fully connected layer is, the output is:

[0066] ⑦Output layer: outputs the adjustment values ​​of the reverberation filter parameters.

[0067]

[0068] Among them, and are the weight and bias of the output layer respectively.

[0069] ⑧Loss function: Mean square error (MSE) is used as the loss function to optimize the parameters of the neural network.

[0070]

[0071] In the embodiment of the present application, the reverberation parameters are automatically adjusted according to the characteristics of the input audio signal through the deep neural network architecture to achieve dynamic optimization and improve the intelligence and effect of audio processing. By learning the complex characteristics of the input audio signal, the neural network can adaptively adjust the parameters of the reverberation filter to ensure that the best reverberation effect can be obtained in different audio environments.

[0072] In some optional implementations of the embodiments of the present application, an online learning mechanism can be introduced to enable the deep neural network to be dynamically adjusted according to the real-time input audio data. At the same time, it can be considered to introduce an adaptive filter into the traditional LMS (Least Mean Squares) algorithm to further enhance the filter's adaptability to rapidly changing audio environments.

[0073] Implementation steps: 1. Dynamically update DNN weights: Through the online learning algorithm, after each audio signal is processed, the DNN weights are slightly adjusted based on real-time feedback. Assume that the loss function is , then the weight update formula for each round is:

[0074] in, is the learning rate, is the gradient calculated based on the current input audio.

[0075] 2. Combine the LMS algorithm to adjust the filter: The filter parameter adjustment can be achieved by the following formula:

[0076] in, is the error signal, is the step size factor, is the current audio input.

[0077] In some optional implementations of the embodiments of the present application, the above step S202 includes the following steps: If the audio signal to be processed is a multi-channel input audio signal, performing fast Fourier transform on the audio time domain representation of the audio signal to be processed of each channel and the filter time domain representation of the adaptive filter respectively; Performing parallel convolution operation on the audio time domain representation after fast Fourier transformation and the filter time domain representation to obtain a parallel convolution operation result; According to the original channels, the parallel convolution operation results are respectively subjected to inverse fast Fourier transform to obtain the inverse fast Fourier transform results of each channel; The inverse fast Fourier transform results of each channel are synthesized to obtain the convolution frequency domain signal.

[0078] In the embodiment of the present application, fast convolution processing is performed on multi-channel input audio signals respectively and synthesized into a final output signal. Specifically, the following steps are included: (1) Multi-channel FFT: Perform FFT on the audio signal of each channel separately.

[0079] (2) Parallel convolution processing: Use multi-core processors or GPUs to perform parallel convolution operations.

[0080] (3) Multi-channel IFFT: Perform IFFT on the convolution result of each channel separately.

[0081] In the embodiment of the present application, the parallel computing capability of the GPU is utilized to perform FFT and IFFT operations in parallel, which can greatly improve the computing speed. The calculation of FFT is essentially parallelizable because the butterfly structure of FFT allows multiple independent operations to be performed simultaneously. Distributing the calculation tasks of FFT / IFFT to multiple GPU cores can significantly reduce the computing time when processing large-scale audio data. Specifically: 1. Basic theory of parallel FFT: FFT is essentially a recursive algorithm based on the divide-and-conquer method. In each step, the input is divided into subsequences with even and odd indexes, and then the FFT of each subsequence is recursively calculated, and the results are finally merged. This process can be parallelized by GPU. For an input signal of length N , the calculation formula of FFT is:

[0082] This formula can be parallelized by following the steps below: The input signal is divided into two parts: even-numbered and odd-numbered, and their FFTs are calculated separately.

[0083] Each GPU core is assigned a portion of data to perform multiplication and addition operations in the butterfly structure.

[0084] By taking advantage of the divide-and-conquer property of the Cooley-Tukey FFT algorithm, the GPU can process these subsequences simultaneously on multiple cores.

[0085] 2. Application of CUDA parallel programming model: CUDA is a tool provided by NVIDIA for GPU programming, which supports parallel operations on large-scale data. Through CUDA, the calculation process of FFT and IFFT is decomposed into multiple thread blocks, and each thread block performs a part of the butterfly operation.

[0086] Steps of linear FFT: 1. Transfer the input data to the GPU's global memory.

[0087] 2. Split the input data into multiple small blocks, and assign each block to a thread block.

[0088] 3. Multiple threads in each thread block execute butterfly operations in parallel to complete part of the FFT calculation.

[0089] 4. Merge the calculation results of all thread blocks to complete the FFT operation.

[0090] CUDA kernel implementation: In CUDA, write an FFT kernel, where each thread performs the following calculations:

[0091] in, is the twiddle factor, which can be calculated in parallel in each thread.

[0092] 3. Standardization of Inverse Fast Fourier Transform (IFFT): The formula of IFFT is:

[0093] The calculation of IFFT is similar to that of FFT. It only needs to take the complex conjugate of the rotation factor and finally normalize the result. Therefore, the GPU can directly reuse the calculation process of FFT and only needs to divide all the results by N at the end of the calculation.

[0094] 4. Parallel convolution processing of multi-channel audio: In multi-channel audio processing, the parallel capability of GPU is more significant. The convolution calculation of each channel can be performed independently on the GPU FFT and IFFT, and the data of all channels can be processed in parallel.

[0095] For each audio channel and filter , the convolution calculation can be expressed as:

[0096] The FFT and IFFT calculations of each channel c can be assigned to independent GPU thread blocks to achieve parallel acceleration.

[0097] In the embodiment of the present application, the above convolution refers to regional multi-channel fast convolution, and the specific implementation process is summarized as follows: The general process of regional multi-channel fast convolution includes input transformation, weight transformation, matrix multiplication (GEMM), and output transformation. The weight transformation is only performed once and can be reused on different inputs. Generally, considering the runtime efficiency, the weight transformation can be performed before the convolution operation is actually performed, so the actual execution process of regional multi-channel fast convolution can be summarized as the following steps: 1. Convert the input audio block to the regional multi-channel fast convolution domain; 2. Perform matrix multiplication of transformed input and weights in the regional multi-channel fast convolution domain; 3. Convert the result of the matrix multiplication back to the spatial domain; If an r-tap FIR filter with m outputs is used, the size of the input received by F(m,r) is (m+r-1).

[0098] The input audio features are first partitioned in the time dimension. The audio is divided into small blocks of (m+r-1) scale. Assuming the number of input channels is C, the scale of each small block is (m+r-1)×(m+r-1)×C. The number of small blocks divided here is R, and these R small blocks can implement the input transformation of the fast convolution domain in parallel. In this transformation process, the transformation of the channel dimension is independent of each other, so the input transformation is implemented in parallel in the channel dimension through SIMD (single instruction multiple data).

[0099] After the transformation, each small block of input is converted to a 4×4×C matrix in the fast convolution domain, and there are a total of R such matrices. Corresponding to the subsequent steps in the convolution, the 4×4 values ​​in each channel need to be multiplied with the 4×4 weight values ​​in one channel from the weight transformation. Instruction parallelization during the transformation process is implemented in the channel dimension. That is, after performing a transformation, there are 4×4 vector registers, each of which contains the values ​​in the corresponding position of the continuous channel. These 4×4 values ​​need to be distributed to 16 GEMM (general matrix multiplication) operations, and the output values ​​after the transformation are also stored in a continuous manner in the channel dimension.

[0100] In terms of weight transformation, it is necessary to ensure that the output channel dimension is in the innermost layer of the data layout, that is, the stride corresponding to the output channel dimension is 1. This is quite inconsistent with the data representation in general convolutional networks. Fortunately, the value of the weight is fixed during the inference process, so the weight value can be transposed in advance to ensure that the output channel is in the innermost layer of the data layout (in fact, the entire process of weight transformation can be completed before the network runs the workload, and only the transformation value needs to be reused during model operation). Assume that the number of input channels of a 3×3 convolution is C, the number of output channels is M, and each 3×3 weight is transformed into a 4×4 matrix. Use vector calculation instructions in the channel dimension to synchronously calculate the result of the weight transformation and write it to storage according to the 16×C×M layout.

[0101] In both transformations, the limited number of registers prevents us from processing all channels or feature blocks at once. The concept of matrix stride makes it very intuitive to process the transformation results of each sub-block and write them out according to the above layout. As an example, the 4×4×C values ​​of the rth sub-block processed in the input transformation can be written out as follows:

[0102] The fast convolution transformation of the above input and weights is the input of the matrix multiplication in the fast convolution calculation. The plane scale after the transformation is 4x4, so 16 GEMM (general matrix multiplication) calculations are required here to implement the multiplication operation in the fast convolution domain. The corresponding matrix multiplication in each GEMM implementation is the product of an R×C matrix and a C×M matrix. The previous restrictions on the output layout during the transformation process are to make it easier to implement these GEMM calculations here.

[0103] In actual convolution calculations, R, C, and M may all be relatively large values, so it is necessary to carefully design matrix multiplication and make full use of cache and vector registers to achieve efficient calculations. After all, reducing multiplication operations is the biggest advantage of fast convolution, and efficient GEMM implementation can truly reflect the acceleration effect of fast convolution. On the other hand, if the time share of each stage (input and output transformation, GEMM) in fast convolution is split, under the premise of ensuring efficient implementation of GEMM, the greater the proportion of GEMM, the more multiplication operations are saved by fast convolution, and the more obvious the acceleration effect is.

[0104] After GEMM is executed, the calculation results are written out in a 16×R×M data arrangement. Similar to the input transformation, the output transformation process can be regarded as the inverse process of the input transformation. Here, the multi-channel intermediate results involved in the output transformation are read in from the 16 positions in the GEMM calculation results, and then the results of multiple channels are processed in parallel. At the same time, according to the index of the corresponding sub-block and the scale information of the convolution output, the results of the output transformation are filled into the corresponding positions of the convolution output, thereby obtaining the HxWxM convolution output and completing the entire convolution operation. This complete process is as follows: Figure 3 shown.

[0105] In some optional implementations of the embodiments of the present application, parallel convolution is used for processing multi-channel audio, and a load balancing scheduling algorithm is introduced. In combination with the characteristics of the audio signal, adaptive load distribution is performed on multiple cores or GPUs to ensure that the workload of each core or stream processor is balanced. The implementation steps are as follows: 1. Load balancing algorithm: based on the audio block length of each channel and the convolution kernel size , calculate the load of each core:

[0106] Then the tasks are assigned to computing resources through a greedy algorithm:

[0107] 2. Adaptive allocation: In each round of task allocation, the workload of the core is dynamically adjusted according to the load of each channel to ensure that all resources can be processed evenly.

[0108] In some optional implementations of the embodiments of the present application, the above-mentioned deep neural network can use a lightweight convolutional neural network (CNN) architecture, such as MobileNet or SqueezeNet, which improves the reasoning speed by reducing the number of parameters and computational complexity while maintaining good prediction accuracy. To this end, the present invention proposes a separable deep convolutional network that can effectively reduce the amount of calculation.

[0109] The implementation steps are as follows: 1. Use depth-wise separable convolution instead of standard convolution: The computational complexity of standard convolution is , where N is the input size, k is the convolution kernel size, and ℃ is the number of input channels. The computational complexity can be reduced by decomposing the standard convolution into depthwise convolution and pointwise convolution:

[0110] in is a separate convolution kernel for each input channel, It is a 1x1 point-by-point convolution kernel. This can reduce the number of model parameters and the amount of calculation, making the speed of each filter parameter adjustment significantly improved.

[0111] 2. Model pruning and quantization: Model pruning: Prune the model according to the actual application scenario, remove unimportant neuron connections, reduce the network size and reduce the amount of forward propagation calculations. Model quantization: Further accelerates reasoning by converting floating-point operations to low-precision integer operations (such as 8-bit quantization). Using a quantized neural network can effectively reduce the time for adjusting filter parameters.

[0112] 3. Lightweight network training: Use lightweight network architecture to train deep neural networks and adjust filter parameters. Use the following loss function Conduct training: · : True filter parameters.

[0113] · : Filter parameters for neural network prediction.

[0114] · : A collection of model parameters.

[0115] By continuously optimizing the loss function, we ensure that the model can accurately predict the filter parameters.

[0116] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0117] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0119] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0120] Embodiment 2 Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of a reverberation processing device, and the embodiment of the device is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0121] like Figure 4 As shown, the reverberation processing device 200 of the embodiment of the present application includes: The to-be-processed audio acquisition module 210 is used to receive the to-be-processed audio signal sent by the user terminal; The convolution module 220 is used to call the adaptive reverberation filter and perform convolution processing on the audio signal to be processed according to the adaptive reverberation filter and the fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as:

[0122] Wherein, FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; The conversion module 230 is used to convert the convolution frequency domain signal to obtain a target audio signal; The output module 240 is used to output the target audio signal to the user terminal.

[0123] In an embodiment of the present application, a reverberation processing device 200 is provided, including: a to-be-processed audio acquisition module 210, for receiving an to-be-processed audio signal sent by a user terminal; a convolution module 220, for calling an adaptive reverberation filter, and performing convolution processing on the to-be-processed audio signal according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal; a conversion module 230, for performing conversion processing on the convolution frequency domain signal to obtain a target audio signal; and an output module 240, for outputting the target audio signal to the user terminal. Compared with the prior art, the present application reduces the computational complexity to O(NlogN) by converting the convolution operation into a multiplication operation in the frequency domain and performing a fast Fourier transform using an FFT algorithm, thereby effectively reducing the amount of computation, thereby significantly improving the processing speed and reducing the processing delay.

[0124] In some optional implementations of the embodiments of the present application, the reverberation processing device 200 further includes: Neural network acquisition module, used to obtain the trained deep neural network; A parameter adjustment value acquisition module is used to input the audio signal to be processed into a trained deep neural network for reverberation parameter adjustment training processing to obtain a filter parameter adjustment value; The parameter adjustment module is used to call the initial reverberation filter and perform parameter adjustment processing on the initial reverberation filter according to the filter parameter adjustment value to obtain an adaptive reverberation filter.

[0125] In some optional implementations of the embodiments of the present application, the neural network acquisition module includes: A training data acquisition submodule, used to acquire training audio data and standard parameter adjustment values ​​corresponding to the training audio data; The neural network calling submodule is used to call the initial deep neural network, where the initial deep neural network includes a convolutional layer, a residual network module, an attention mechanism module, a pooling layer, a fully connected layer, and an output layer; A feature extraction submodule is used to perform feature extraction processing on the training audio data according to the convolution layer to obtain training audio features; A feature enhancement submodule, used to perform feature enhancement processing on the training audio features according to the residual network module to obtain training audio enhanced features; A feature screening submodule is used to perform feature screening processing on the training audio enhancement features according to the attention mechanism module to obtain training audio screening features; The feature dimension reduction submodule is used to perform feature dimension reduction processing on the training audio screening features according to the pooling layer to obtain the training audio dimension reduction features; A feature fusion submodule is used to perform feature fusion processing on the training audio features and the training audio dimensionality reduction features according to the fully connected layer to obtain the training audio fusion features; An adjustment value prediction submodule is used to perform parameter adjustment value prediction processing on the training audio fusion features according to the output layer to obtain the training parameter adjustment value; The convergence confirmation submodule is used to adjust the parameters of the initial deep neural network according to the training parameter adjustment value, the standard parameter adjustment value and the preset loss function until the parameters of the initial deep neural network meet the convergence conditions to obtain a trained deep neural network.

[0126] In some optional implementations of the embodiments of the present application, the convolution module includes: A fast Fourier transform submodule, for performing fast Fourier transform on the audio time domain representation of the audio signal to be processed of each channel and the filter time domain representation of the adaptive filter respectively if the audio signal to be processed is a multi-channel input audio signal; A parallel convolution submodule is used to perform parallel convolution operations on the audio time domain representation after fast Fourier transformation and the filter time domain representation to obtain parallel convolution operation results; The inverse fast Fourier transform result submodule is used to perform inverse fast Fourier transform on the parallel convolution operation results according to the original channels to obtain the inverse fast Fourier transform results of each channel; The synthesis submodule is used to synthesize the inverse fast Fourier transform results of each channel to obtain the convolution frequency domain signal.

[0127] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0128] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0129] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0130] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions of the reverberation processing method, etc. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0131] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as computer-readable instructions for running the reverberation processing method.

[0132] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0133] The computer device provided in the present application reduces the computational complexity to O(NlogN) by converting convolution operations into multiplication operations in the frequency domain and performing fast Fourier transform using the FFT algorithm, thereby effectively reducing the amount of calculation, thereby significantly improving the processing speed and reducing processing delays.

[0134] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the reverberation processing method as described above.

[0135] The computer-readable storage medium provided in the present application reduces the computational complexity to O(NlogN) by converting the convolution operation into a multiplication operation in the frequency domain and performing a fast Fourier transform using the FFT algorithm, thereby effectively reducing the amount of calculation, thereby significantly improving the processing speed and reducing the processing delay.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0137] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A reverberation processing method, characterized in that: The steps include: Receiving an audio signal to be processed sent by a user terminal; An adaptive reverberation filter is called, and convolution processing is performed on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as: Wherein, FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; Performing conversion processing on the convolution frequency domain signal to obtain a target audio signal; The target audio signal is output to the user terminal.

2. The reverberation processing method according to claim 1, characterized in that: After the step of receiving the audio signal to be processed sent by the user terminal and before the step of calling the adaptive reverberation filter and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and the fast convolution algorithm to obtain the convolution frequency domain signal, the following steps are also included: Get the trained deep neural network; Inputting the audio signal to be processed into the trained deep neural network for reverberation parameter adjustment training processing to obtain filter parameter adjustment values; An initial reverberation filter is called, and parameter adjustment processing is performed on the initial reverberation filter according to the filter parameter adjustment value to obtain the adaptive reverberation filter.

3. The reverberation processing method according to claim 2, characterized in that: The step of obtaining a trained deep neural network specifically includes the following steps: Acquire training audio data and standard parameter adjustment values ​​corresponding to the training audio data; Calling an initial deep neural network, wherein the initial deep neural network includes a convolutional layer, a residual network module, an attention mechanism module, a pooling layer, a fully connected layer, and an output layer; Performing feature extraction processing on the training audio data according to the convolution layer to obtain training audio features; Performing feature enhancement processing on the training audio features according to the residual network module to obtain training audio enhanced features; Performing feature screening processing on the training audio enhancement feature according to the attention mechanism module to obtain a training audio screening feature; Performing feature dimensionality reduction processing on the training audio screening features according to the pooling layer to obtain training audio dimensionality reduction features; Performing feature fusion processing on the training audio features and the training audio dimensionality reduction features according to the fully connected layer to obtain training audio fusion features; Perform parameter adjustment value prediction processing on the training audio fusion feature according to the output layer to obtain a training parameter adjustment value; The initial deep neural network is subjected to parameter adjustment processing according to the training parameter adjustment value, the standard parameter adjustment value and a preset loss function until the parameters of the initial deep neural network meet convergence conditions, thereby obtaining the trained deep neural network.

4. The reverberation processing method according to claim 1, characterized in that: The step of calling the adaptive reverberation filter and performing convolution processing on the audio signal to be processed according to the adaptive reverberation filter and the fast convolution algorithm to obtain the convolution frequency domain signal specifically includes the following steps: If the audio signal to be processed is a multi-channel input audio signal, performing fast Fourier transform on the audio time domain representation of the audio signal to be processed of each channel and the filter time domain representation of the adaptive filter respectively; Performing parallel convolution operation on the audio time domain representation after fast Fourier transformation and the filter time domain representation to obtain a parallel convolution operation result; Performing inverse fast Fourier transform on the parallel convolution operation results according to the original channels respectively to obtain inverse fast Fourier transform results of each channel; The inverse fast Fourier transform results of each channel are synthesized to obtain the convolution frequency domain signal.

5. A reverberation processing device, characterized in that: include: The to-be-processed audio acquisition module is used to receive the to-be-processed audio signal sent by the user terminal; A convolution module is used to call an adaptive reverberation filter and perform convolution processing on the audio signal to be processed according to the adaptive reverberation filter and a fast convolution algorithm to obtain a convolution frequency domain signal, wherein the fast convolution algorithm is expressed as: Wherein, FFT represents fast Fourier transform, IFFT represents inverse fast Fourier transform, x represents the time domain representation of the audio signal to be processed, y represents the time domain representation of the adaptive filter, and H(f) represents the frequency domain representation of the convolved audio signal; A conversion module, used for converting the convolution frequency domain signal to obtain a target audio signal; An output module is used to output the target audio signal to the user terminal.

6. The reverberation processing device according to claim 5, characterized in that: The device also includes: Neural network acquisition module, used to obtain the trained deep neural network; A parameter adjustment value acquisition module, used for inputting the audio signal to be processed into the trained deep neural network for reverberation parameter adjustment training processing to obtain a filter parameter adjustment value; The parameter adjustment module is used to call the initial reverberation filter and perform parameter adjustment processing on the initial reverberation filter according to the filter parameter adjustment value to obtain the adaptive reverberation filter.

7. The reverberation processing device according to claim 6, characterized in that: The neural network acquisition module includes: A training data acquisition submodule, used to acquire training audio data and standard parameter adjustment values ​​corresponding to the training audio data; A neural network calling submodule, used to call an initial deep neural network, wherein the initial deep neural network includes a convolutional layer, a residual network module, an attention mechanism module, a pooling layer, a fully connected layer, and an output layer; A feature extraction submodule, used for performing feature extraction processing on the training audio data according to the convolution layer to obtain training audio features; A feature enhancement submodule, used for performing feature enhancement processing on the training audio features according to the residual network module to obtain training audio enhanced features; A feature screening submodule, used for performing feature screening processing on the training audio enhancement feature according to the attention mechanism module to obtain a training audio screening feature; A feature dimension reduction submodule, used for performing feature dimension reduction processing on the training audio screening features according to the pooling layer to obtain training audio dimension reduction features; A feature fusion submodule, used for performing feature fusion processing on the training audio features and the training audio dimensionality reduction features according to the fully connected layer to obtain training audio fusion features; An adjustment value prediction submodule, used for performing parameter adjustment value prediction processing on the training audio fusion feature according to the output layer to obtain a training parameter adjustment value; The convergence confirmation submodule is used to perform parameter adjustment processing on the initial deep neural network according to the training parameter adjustment value, the standard parameter adjustment value and the preset loss function until the parameters of the initial deep neural network meet the convergence conditions to obtain the trained deep neural network.

8. The reverberation processing device according to claim 6, characterized in that: The convolution module comprises: A fast Fourier transform submodule, configured to perform fast Fourier transform on the audio time domain representation of the audio signal to be processed of each channel and the filter time domain representation of the adaptive filter respectively if the audio signal to be processed is a multi-channel input audio signal; A parallel convolution submodule is used to perform parallel convolution operations on the audio time domain representation after fast Fourier transformation and the filter time domain representation to obtain parallel convolution operation results; An inverse fast Fourier transform result submodule is used to perform inverse fast Fourier transform on the parallel convolution operation results according to the original channels respectively to obtain inverse fast Fourier transform results of each channel; The synthesis submodule is used to perform synthesis processing on the inverse fast Fourier transform results of each channel to obtain the convolution frequency domain signal.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and the processor implements the steps of the reverberation processing method according to any one of claims 1 to 4 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the reverberation processing method according to any one of claims 1 to 4 are implemented.

Images