AI model training method and echo cancellation method

Abstract

The invention discloses an AI model training method and an echo cancellation method. The training method comprises the steps that first-stage training is carried out on a basic model, the basic model comprises an encoder, a decoder and an RNN model, a GCN-Complex model is adopted by the encoder, the GCN-Complex model is adopted by the decoder, and a DPRNN model is adopted by the RNN model; after the basic model is connected with an optimization model, second-stage training is carried out, parameter changes of the basic model are inhibited in the second-stage training process, and the optimization model comprises a multi-layer LSTM structure and a full connection layer; after the second-stage training is completed, an echo cancellation model is obtained; the echo cancellation model is used for performing streaming processing on the audio file needing echo cancellation. According to the method and the device, a good echo cancellation effect is achieved by streaming processing while the parameter quantity is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular, to an AI model training method and an echo cancellation method. BACKGROUND

[0002] This section is intended to provide background information to facilitate an understanding of the present disclosure. It is not admitted that any of the information provided in this section is prior art or relevant to the patentability of the claimed disclosure, including that which might be considered to be prior art by virtue of 37 C.F.R. § 1.102.

[0003] With the rapid development of communication technology, remote conferencing has become an indispensable part of people's daily work and life. In the process of remote conference double speaking, the near-end and the far-end use the loudspeaker and the microphone at the same time. The near-end microphone receives the far-end speaker's voice signal played by the near-end loudspeaker and the near-end speaker's voice signal at the same time. If the voice signal transmitted from the far-end is not effectively cancelled, the near-end microphone will transmit such voice signal to the far-end loudspeaker, and the far-end speaker will hear the echo of his previous speech. Echo cancellation is to eliminate the far-end speaker's voice signal played by the near-end loudspeaker, and only transmit the near-end speaker's voice signal received by the near-end microphone.

[0004] The generation of such echo must have three conditions: 1) loudspeaker: play the voice of the far-end (opponent); 2) microphone: pick up the voice of the near-end (your surroundings), including the voice of the near-end speaker and the voice played by the loudspeaker; 3) sound loop: the voice played by the loudspeaker through the air is picked up by the microphone and sent back to the far-end through the network, forming a closed electric-acoustic-electric loop.

[0005] If not handled, such echo will seriously interfere with the call, causing both parties to have to take turns speaking and being unable to communicate naturally, resulting in a poor experience. The task of the echo cancellation AEC (Acoustic Echo Cancellation) algorithm is to accurately identify and remove the echo signal from the loudspeaker in the signal collected by the microphone, and only keep the voice of the near-end speaker and the background noise.

[0006] Most of the prior art uses adaptive filters in traditional algorithms, such as normalized least mean square (NLMS), Kalman filter, etc., but the effect is poor.

[0007] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0008] The embodiments of the present application provide an AI model training method and an echo cancellation method to at least solve the technical problem of poor echo cancellation effect in the related art.

[0009] According to an aspect of the embodiments of the present application, an AI model training method is provided, comprising: performing first stage training on a basic model, wherein the basic model comprises an encoder, a decoder and an RNN model, the encoder adopts a GCRN-Complex model, the decoder adopts a GCRN-Complex model, and the RNN model adopts a DPRNN model; connecting the basic model with an optimization model after the first stage training, and performing second stage training, and suppressing parameter changes of the basic model during the second stage training, wherein the optimization model comprises a multi-layer LSTM structure and a full connection layer; and obtaining an echo cancellation model after the second stage training, wherein the echo cancellation model is used for processing an audio file that needs to be subjected to echo cancellation.

[0010] Optionally, in the basic model: the encoder comprises 5 conv layers, and the network structure of each conv layer comprises 2 conv2ds, 1 sigmoid, 1 batchnormal and 1 Elu; the decoder comprises 5 Transposedconv layers, and the network structure of each Transposedconv layer comprises 2 Transposedconv2ds, 1 sigmoid, 1 batchnormal and 1 Elu; and the RNN model comprises RowLstm and ColLstm two structures, and each structure comprises 2 Transposes, 1 BiLSTM, 1 Linear and 1 GroupNorm.

[0011] Optionally, in the basic model, the input_size in the RNN model is set to 64, the hidden_size is set to 96, and the output_size is set to 128.

[0012] Optionally, in the optimization model, the number of layers of the LSTM structure is set to 1.

[0013] According to another aspect of the embodiments of the present application, an echo cancellation method is also provided, comprising: obtaining a target audio that needs to be subjected to echo cancellation; inputting the target audio into the echo cancellation model for processing to obtain a clean audio.

[0014] Optionally, the target audio is input into the echo cancellation model for processing, including: processing the first sequence and saving the nth frame from the end of the output; concatenating the audio frame to be processed with the first sequence (if it is the first processing, the aforementioned first sequence; if it is the second processing, the second sequence, and so on) in the time dimension, and deleting the audio frame with the earliest sampling time in the first sequence to obtain the second sequence; processing the second sequence and saving the nth frame from the end of the output. This process is repeated to complete the streaming processing of the entire speech.

[0015] According to another aspect of the embodiments of this application, an AI model training apparatus is also provided, comprising: a first training module for performing a first stage of training on a base model, wherein the base model includes an encoder, a decoder, and an RNN model, the encoder adopting a GCRN-Complex model, the decoder adopting a GCRN-Complex model, and the RNN model adopting a DPRNN model; a second training module for: connecting the base model and an optimized model, performing a second stage of training, and suppressing parameter changes of the base model during the second stage of training, wherein the optimized model includes a multi-layer LSTM structure and a fully connected layer; and obtaining an echo cancellation model after completing the second stage of training, the echo cancellation model being used to process audio files that require echo cancellation.

[0016] Optionally, in the basic model: the encoder includes 5 conv layers, and the network structure of each conv layer includes 2 conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; the decoder includes 5 transposed conv layers, and the network structure of each transposed conv layer includes 2 transposed conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; the RNN model includes two structures, RowLstm and ColLstm, each of which includes 2 transposed, 1 BiLSTM, 1 linear, and 1 groupnorm.

[0017] Optionally, in the base model, the input_size is set to 64, the hidden_size is set to 96, and the output_size is set to 128 in the RNN model.

[0018] Optionally, in the optimization model, the number of layers in the LSTM structure is set to 1.

[0019] According to another aspect of the embodiments of this application, an AI model training apparatus is also provided, comprising: an acquisition module for acquiring target audio to be echo-cancelled; and a processing module for inputting the target audio into the echo-cancellation model for processing to obtain clean audio.

[0020] Optionally, the processing module is also used to process the first sequence, save the nth frame from the end of the output; concatenate an audio frame to be processed with the first sequence in the time dimension, and delete the audio frame with the earliest sampling time in the first sequence to obtain the second sequence, process the second sequence, save the nth frame from the end of the output, and so on, to complete the streaming processing of the entire speech.

[0021] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the storage medium including a stored program that executes the above-described method when the program is run.

[0022] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the above-described method through the computer program.

[0023] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the embodiments of the methods described above.

[0024] In this embodiment, the technical solution of this application is adopted to perform a first-stage training on the basic model. The basic model includes an encoder, a decoder, and an RNN model. The encoder adopts a GCRN-Complex model, the decoder adopts a GCRN-Complex model, and the RNN model adopts a DPRNN model. After connecting the basic model and the optimized model, a second-stage training is performed. During the second-stage training, the parameter changes of the basic model are suppressed. The optimized model includes a multi-layer LSTM structure and a fully connected layer. After completing the second-stage training, an echo cancellation model is obtained. The echo cancellation model is used to process audio files that need echo cancellation. This allows the powerful nonlinear modeling capability of neural networks to replace or assist traditional adaptive filtering, especially showing good results in dealing with nonlinear distortion and strong reverberation scenarios. It achieves good echo cancellation effect in streaming processing while reducing the number of parameters. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of an AI model training method according to an embodiment of this application; Figure 2 This is a schematic diagram of an optional echo cancellation model according to an embodiment of this application; Figure 3 This is a schematic diagram of an optional encoder structure according to an embodiment of this application; Figure 4 This is a schematic diagram of an optional Decoder structure according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional RNN structure according to an embodiment of this application; Figure 6 This is a schematic diagram of an AI model training device according to an embodiment of this application; Figure 7 This is a structural block diagram of a terminal device according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] Currently, the most common method in the market is echo cancellation using traditional adaptive filters, but adaptive filters have significant drawbacks. These mainly include the following: 1) Inadequate handling of nonlinear distortion The biggest weakness of traditional AEC is its inability to handle nonlinear distortion, as adaptive filters are essentially linear models. When hardware such as speakers and amplifiers produce nonlinear distortion (such as clipping and harmonics) due to excessive volume or poor quality, the relationship between the played signal and the original reference signal is no longer linear.

[0029] The algorithm cannot simulate this nonlinear transformation through linear combination, resulting in a large number of nonlinear echoes remaining. These echoes must be suppressed by the back-end nonlinear processing (NLP) module, but crude suppression will lead to problems such as deterioration of speech quality.

[0030] 2) The challenges of accuracy and robustness in dual-talk detection (DTD) DTD decisions rely on simple features such as energy and spectral correlation, making them prone to misjudgment in complex scenarios. Dual-talk detection may make incorrect judgments when there are sudden changes in background noise (such as the sound of a door closing or keyboard clicking) or when there is only noise at the near end, thus leading to a decrease in echo cancellation performance.

[0031] 3) Changes in echo path affect echo cancellation performance The performance of adaptive filters is closely related to the accuracy of state and observation estimations. Estimation errors can lead to unacceptable results, especially when the echo path changes abruptly, in which case the tracking performance of adaptive filters can degrade significantly.

[0032] AI technology has brought new ideas to AEC, especially its powerful nonlinear modeling capabilities. Its main problem is the high computational and power consumption. Even lightweight neural networks require far more multiply-accumulate operations and memory accesses than traditional adaptive filter algorithms, making pure AI solutions difficult to deploy in low-power, real-time-critical audio codec links. This solution, through its AI network architecture, processes speech signals in the frequency domain, effectively ensuring echo cancellation quality while significantly reducing hardware resource consumption.

[0033] According to one aspect of the embodiments of this application, a method embodiment for training an AI model is provided. Figure 1 This is a flowchart of an AI model training method according to an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps: Step S102: Perform the first stage training on the basic model. The basic model includes an encoder, a decoder, and an RNN model. The encoder uses the GCRN-Complex model, the decoder uses the GCRN-Complex model, and the RNN model uses the DPRNN model.

[0034] Optionally, in the base model: the encoder consists of 5 conv layers, each conv layer's network structure includes 2 conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; the decoder consists of 5 transposed conv layers, each transposed conv layer's network structure includes 2 transposed conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; the RNN model includes two structures, RowLStm and ColLStm, each structure including 2 transposed, 1 BiLSTM, 1 linear, and 1 GroupNorm, respectively. In the RNN model, input_size is set to 64, hidden_size to 96, and output_size to 128.

[0035] Step S104: After connecting the base model and the optimized model, the second stage of training is performed. During the second stage of training, the parameter changes of the base model are suppressed. The optimized model includes a multi-layer LSTM structure and a fully connected layer.

[0036] Optionally, in the optimization model, the number of layers in the LSTM structure is set to 1.

[0037] Step S106: After completing the second stage of training, an echo cancellation model is obtained. The echo cancellation model is used to process audio files that need echo cancellation.

[0038] The technical solution of this application involves a first-stage training of a basic model, which includes an encoder, a decoder, and an RNN model. The encoder and decoder employ a GCRN-Complex model, and the RNN model uses a DPRNN model. After connecting the basic model with the optimized model, a second-stage training is performed, during which parameter changes in the basic model are suppressed. The optimized model includes a multi-layer LSTM structure and fully connected layers. After completing the second-stage training, an echo cancellation model is obtained. This model is then used to process audio files requiring echo cancellation. By leveraging the powerful nonlinear modeling capabilities of neural networks, it can replace or assist traditional adaptive filtering, demonstrating excellent performance, especially in dealing with nonlinear distortion and strong reverberation scenarios. This achieves a reduction in the number of parameters while simultaneously achieving good echo cancellation in streaming processing.

[0039] According to another aspect of the embodiments of this application, a method embodiment for echo cancellation is provided. It includes: Step 1: Obtain the target audio for echo cancellation.

[0040] Step 2: Input the target audio into the echo cancellation model for processing to obtain clean audio.

[0041] Process the first sequence and save the nth frame from the end of the output; concatenate the audio frame to be processed with the first sequence (as described above) in the time dimension, and delete the audio frame with the earliest sampling time in the first sequence to obtain the second sequence. Process the second sequence and save the nth frame from the end of the output. Repeat this process to complete the streaming processing of the entire speech.

[0042] As an optional embodiment, the technical solution of this application is further described in detail below with reference to specific embodiments: refer to Figure 2 This system consists of two parts: a basic module and an optimization module. The basic module comprises an encoder, an RNN module, and a decoder. The encoder part of the decoder is based on the GCRN-Complex network model, and the RNN part is based on the DPRNN model. The optimization module contains a multi-layer LSTM module, and its design and connections are based on neural network models.

[0043] Generate a training set containing 20,000 data points, a validation set containing 1,000 data points, and 1,000 test sets of each of the following 5 types, corresponding to room1, room2, and room3 in Tables 1 and 2 below, with η2 = ∞ and room2 in scenarios with η2 = 0.1 and η2 = 0.5, while keeping SER = 3.5 dB and SNR = 10 dB consistent.

[0044] Step 1: Construct the basic echo cancellation model Net1, whose encoder and decoder use the GCRN-Complex model, and whose RNN part is based on the DPRNN model, with adjustments made to the specific network structure parameters.

[0045] The Encoder and Decoder network structure parameters are consistent with those in GCRN-Complex. In RNN, input_size, hidden_size, and output_size are set to 64, 96, and 128, respectively.

[0046] Step 2: Perform the first stage training of the echo cancellation model. Train the above-mentioned echo cancellation base model Net1 and output the AI ​​model training parameters.

[0047] Step 3: Construct the echo cancellation optimization model Net2, the main body of which is a multi-layer LSTM structure and fully connected layers. To compress the model as much as possible, the number of LSTM layers in the echo cancellation optimization model Net2 is set to 1. During streaming processing, the data selected to be retained is the third to last frame.

[0048] Step 4: Perform the second stage training of the echo cancellation model.

[0049] The echo cancellation base model Net1 and the echo cancellation optimization model Net2 are connected. When training the echo cancellation optimization model Net2, the parameter changes of the echo cancellation base model Net1 are suppressed in the training loss so that the parameters of the echo cancellation model do not change too much. The overall model parameters of the trained echo cancellation base model Net1 + echo cancellation optimization model Net2 are saved for inference.

[0050] Step 5: Use the overall output model to process the audio file that needs echo cancellation to obtain a clean near-end speech output.

[0051] Streaming processing takes 160 samples (one frame) of new data each time it comes in. This data is concatenated with the samples from previous frames, and the oldest 160 samples are removed. This ensures that the number of samples entering the model each time remains consistent, and is less than or equal to the number of samples used in training. Instead of saving the last frame for each model inference, the model saves the nth frame from the end. Although this introduces a delay of n*10ms, it is within an acceptable range and achieves better echo removal.

[0052] The main structures of Encoder, Decoder, and RNN are as follows: Figure 3 , Figure 4 , Figure 5 As shown.

[0053] 1) The Encoder layer contains 5 conv layers. The network structure of each conv layer contains two conv2d layers, one sigmoid layer, one batchnormal layer, and one Elu layer.

[0054] 2) The Decoder layer contains 5 Transposed conv layers. The network structure of each Transposed conv layer contains two Transposed conv2d, one sigmoid, one batchnormal, and one Elu.

[0055] 3) RNN contains two structures: RowLstm and ColLstm. Each structure contains two Transposes, one (Bi)LSTM, one Linear, and one GroupNorm.

[0056] In this scheme, a two-stage neural network architecture is designed for echo cancellation. Net2 uses a multi-layer LSTM two-stage model (the two-stage approach significantly improves network performance, with noticeable improvements in the ELE index). Multiple microphone signals are input into the neural network for processing together (by increasing the number of input channels; for example, if there are 2 microphones, the number of channels for the frequency domain microphone signal is 4, plus the 2 channels of the reference signal, for a total of 6 input channels. If there is only 1 microphone, the total number of input channels is 4), instead of processing each microphone signal individually. The signal is then synthesized using beamforming. The training scheme for the two-stage model involves training the first-stage neural network first, then suppressing the weights of the first stage to train the second-stage network, instead of combining the losses of the two stages via lambda connections and training only one stage. In real-time streaming inference scenarios, for each inference iteration, the model output selects the nth frame from the end as the audio data output, instead of the last frame.

[0057] To intuitively compare the performance of the models, this application selected the reference paper (H. Zhang and D. Wang, “Adeep learning approach to multi-channel and multi-microphone acoustic echocancellation,” in Proc. INTERSPEECH, 2021.) as the benchmark. The results of the MMAEC dual-lecture scenario and the CRN model are compared as follows: The dimensions of room1, room2, and room3 are 3x4x3m, 5x6x3m, and 11x14x3m, respectively. Nonlinear distortion is achieved using a scaled error function (SEF), where η2 is the SEF parameter. η2 = ∞ indicates a linear system with no distortion; the smaller η2 is, the more severe the distortion.

[0058] Table 1 SER=3.5 dB, SNR=10 dB, η2 = ∞ (linear system) .

[0059] Table 2 SER=3.5 dB, SNR=10 dB, Room2, η2 = 0.1, and η2 = 0.5 .

[0060] Table 3 Trainable parameters .

[0061] SER is the signal-to-return ratio, SNR is the signal-to-noise ratio, and η² is the parameter of the nonlinear distortion function SEF. For a detailed description, please refer to Part III, EXPERIMENTS (A Deep Learning Approach to Multi-Channel and Multi-Microphone Acoustic), in the paper. These three parameters are set when generating the data. After generating data with the corresponding parameters, inference is performed, and the inference result obtained is the result for that parameter. Neural network inference is a general process; given specific data, it will output the corresponding result.

[0062] In practice, CRN requires computation once for each microphone and twice for two microphones. However, in this application, the computation for the two microphones is placed in a single neural network, so the computational cost does not double.

[0063] The beneficial effects of this application are: using this network structure, compared with the network structure of the CRN model, the number of parameters is significantly reduced, making it suitable for edge deployment; using this network structure, echo cancellation achieves good results in real-time processing of streaming data.

[0064] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0065] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0066] According to another aspect of the embodiments of this application, an AI model training apparatus for implementing the above-described AI model training method is also provided. Figure 6 This is a schematic diagram of an AI model training device according to an embodiment of this application, such as... Figure 6 As shown, the device may include: The first training module 601 is used to perform the first stage training of the base model, wherein the base model includes an encoder, a decoder and an RNN model, the encoder adopts the GCRN-Complex model, the decoder adopts the GCRN-Complex model and the RNN model adopts the DPRNN model. The second training module 603 is used to: connect the base model and the optimized model, perform a second stage of training, and suppress parameter changes of the base model during the second stage of training, wherein the optimized model includes a multi-layer LSTM structure and a fully connected layer; after completing the second stage of training, an echo cancellation model is obtained, which is used to process audio files that need echo cancellation.

[0067] Optionally, in the basic model: the encoder includes 5 conv layers, and the network structure of each conv layer includes 2 conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; the decoder includes 5 transposed conv layers, and the network structure of each transposed conv layer includes 2 transposed conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; the RNN model includes two structures, RowLstm and ColLstm, each of which includes 2 transposed, 1 BiLSTM, 1 linear, and 1 groupnorm.

[0068] Optionally, in the base model, the input_size is set to 64, the hidden_size is set to 96, and the output_size is set to 128 in the RNN model.

[0069] Optionally, in the optimization model, the number of layers in the LSTM structure is set to 1.

[0070] The technical solution of this application involves a first-stage training of a basic model, which includes an encoder, a decoder, and an RNN model. The encoder and decoder employ a GCRN-Complex model, and the RNN model uses a DPRNN model. After connecting the basic model with the optimized model, a second-stage training is performed, during which parameter changes in the basic model are suppressed. The optimized model includes a multi-layer LSTM structure and fully connected layers. After completing the second-stage training, an echo cancellation model is obtained. This model is then used to process audio files requiring echo cancellation. By leveraging the powerful nonlinear modeling capabilities of neural networks, it can replace or assist traditional adaptive filtering, demonstrating excellent performance, especially in dealing with nonlinear distortion and strong reverberation scenarios. This achieves a reduction in the number of parameters while simultaneously achieving good echo cancellation in streaming processing.

[0071] According to another aspect of the embodiments of this application, an AI model training apparatus is also provided, comprising: an acquisition module for acquiring target audio to be echo-cancelled; and a processing module for inputting the target audio into the echo-cancellation model for processing to obtain clean audio. The processing module is further configured to process a first sequence, save the nth frame from the end of the output; concatenate the audio frame to be processed with the first sequence in the time dimension, and delete the audio frame with the earliest sampling time in the first sequence to obtain a second sequence; process the second sequence, save the nth frame from the end of the output, and so on, to complete the streaming processing of the entire speech.

[0072] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in a corresponding hardware environment and can be implemented in software or hardware.

[0073] According to another aspect of the embodiments of this application, a server or terminal for implementing the above-described method is also provided.

[0074] Figure 7 This is a structural block diagram of a terminal device according to an embodiment of this application, such as... Figure 7 As shown, the terminal may include: one or more (only one is shown in the figure) processors 201, memory 203, and transmission devices 205, such as... Figure 7 As shown, the terminal may also include input / output devices 207.

[0075] The memory 203 can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in this embodiment. The processor 201 executes various functional applications and data processing by running the software programs and modules stored in the memory 203, thereby implementing the aforementioned methods. The memory 203 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 203 may further include memory remotely located relative to the processor 201, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0076] The aforementioned transmission device 205 is used to receive or send data via a network, and can also be used for data transfer between the processor and memory. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 205 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 205 is a radio frequency (RF) module used for wireless communication with the Internet.

[0077] Specifically, memory 203 is used to store application programs.

[0078] The processor 201 can invoke the application program stored in the memory 203 via the transmission device 205 to perform the following steps: The first stage of training involves a base model, which includes an encoder, a decoder, and an RNN model. The encoder and decoder each employ a GCRN-Complex model, and the RNN model uses a DPRNN model. After connecting the base model with the optimized model, a second stage of training is performed. During this second stage, parameter variations in the base model are suppressed. The optimized model includes a multi-layer LSTM structure and fully connected layers. After completing the second stage of training, an echo cancellation model is obtained, which is used to process audio files requiring echo cancellation.

[0079] Processor 201 is also used to perform the following steps: Obtain the target audio to be echo-cancelled; input the target audio into the echo-cancellation model for processing to obtain clean audio.

[0080] The technical solution of this application involves a first-stage training of a basic model, which includes an encoder, a decoder, and an RNN model. The encoder and decoder employ a GCRN-Complex model, and the RNN model uses a DPRNN model. After connecting the basic model with the optimized model, a second-stage training is performed, during which parameter changes in the basic model are suppressed. The optimized model includes a multi-layer LSTM structure and fully connected layers. After completing the second-stage training, an echo cancellation model is obtained. This model is then used to process audio files requiring echo cancellation. By leveraging the powerful nonlinear modeling capabilities of neural networks, it can replace or assist traditional adaptive filtering, demonstrating excellent performance, especially in dealing with nonlinear distortion and strong reverberation scenarios. This solves the technical problem of poor echo cancellation performance in related technologies, achieving good echo cancellation results while reducing the number of parameters during streaming processing.

[0081] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0082] Those skilled in the art will understand that Figure 7 The structure shown is for illustrative purposes only. The terminal can be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal devices. Figure 7 This does not limit the structure of the aforementioned electronic device. For example, the terminal may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 7 The different configurations shown.

[0083] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0084] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to execute program code for the above-described method.

[0085] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0086] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: The first stage of training involves a base model, which includes an encoder, a decoder, and an RNN model. The encoder and decoder each employ a GCRN-Complex model, and the RNN model uses a DPRNN model. After connecting the base model with the optimized model, a second stage of training is performed. During this second stage, parameter variations in the base model are suppressed. The optimized model includes a multi-layer LSTM structure and fully connected layers. After completing the second stage of training, an echo cancellation model is obtained, which is used to process audio files requiring echo cancellation.

[0087] Optionally, the storage medium is also configured to store program code for performing the following steps: Obtain the target audio to be echo-cancelled; input the target audio into the echo-cancellation model for processing to obtain clean audio.

[0088] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0089] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0090] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0091] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0092] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An AI model training method, characterized in that, include: The first stage of training is performed on the base model, which includes an encoder, a decoder, and an RNN model. The encoder adopts the GCRN-Complex model, the decoder adopts the GCRN-Complex model, and the RNN model adopts the DPRNN model. After connecting the base model and the optimized model, a second stage of training is performed, and during the second stage of training, the parameter changes of the base model are suppressed. The optimized model includes a multi-layer LSTM structure and a fully connected layer. After completing the second stage of training, an echo cancellation model is obtained, which is used to process audio files that require echo cancellation.

2. The AI ​​model training method according to claim 1, characterized in that, In the aforementioned basic model: The encoder comprises 5 conv layers, and the network structure of each conv layer includes 2 conv2d, 1 sigmoid, 1 batchnormal, and 1 Elu; The decoder includes 5 Transposed conv layers, and the network structure of each Transposed conv layer includes 2 Transposed conv2d layers, 1 sigmoid layer, 1 batchnormal layer, and 1 Elu layer; The RNN model includes two structures: RowLstm and ColLstm. Each structure includes two Transpose, one BiLSTM, one Linear, and one GroupNorm.

3. The AI ​​model training method according to claim 1, characterized in that, In the basic model, the input_size is set to 64, the hidden_size is set to 96, and the output_size is set to 128.

4. The AI ​​model training method according to claim 1, characterized in that, In the optimization model, the number of layers in the LSTM structure is set to 1.

5. An echo cancellation method, characterized in that, An echo cancellation model trained using any one of the AI ​​model training methods of claims 1 to 4 is applied, wherein the echo cancellation method comprises: Obtain the target audio for echo cancellation; The target audio is input into the echo cancellation model for processing to obtain clean audio.

6. The method according to claim 5, characterized in that, The target audio is input into the echo cancellation model for processing, including: Process the first sequence and save the nth frame from the end of the output; concatenate the audio frame to be processed with the first sequence in the time dimension, and delete the audio frame with the earliest sampling time in the first sequence to obtain the second sequence. Process the second sequence and save the nth frame from the end of the output. Repeat this process to complete the streaming processing of the entire speech.

7. An AI model training device, characterized in that, include: The first training module is used to perform the first stage training of the base model, wherein the base model includes an encoder, a decoder and an RNN model, the encoder adopts the GCRN-Complex model, the decoder adopts the GCRN-Complex model and the RNN model adopts the DPRNN model. The second training module is used to: connect the base model and the optimized model, perform a second stage of training, and suppress parameter changes of the base model during the second stage of training. The optimized model includes a multi-layer LSTM structure and a fully connected layer. After completing the second stage of training, an echo cancellation model is obtained, which is used to process audio files that need echo cancellation.

8. An echo cancellation device, characterized in that, An echo cancellation device is applied to an echo cancellation model trained using any one of the AI ​​model training methods of claims 1 to 4, the echo cancellation device comprising: The acquisition module is used to acquire the target audio to be echo-cancelled. The processing module is used to input the target audio into the echo cancellation model for processing to obtain clean audio.

9. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 6 when it is run.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the method described in any one of claims 1 to 6 via the computer program.

Images