Echo cancellation model training method, echo cancellation method, device, medium and product

Abstract

The application discloses an echo cancellation model training method, an echo cancellation method, equipment, a medium and a product, relates to the technical field of machine learning, and the method comprises the following steps: acquiring training data, an echo cancellation model to be trained and a pre-constructed dynamic selection network, wherein the dynamic selection network comprises a fusion model and a plurality of pre-trained teacher models, the fusion model is used for fusing the outputs of the teacher models, and each teacher model at least comprises an echo cancellation task model and a human voice reservation task model; inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection network to obtain a fused echo cancellation result; calculating a knowledge distillation loss based on the student echo cancellation result and the fused echo cancellation result; and optimizing the echo cancellation model based on the knowledge distillation loss. The application realizes that the echo cancellation model can consider echo cancellation and human voice reservation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to an echo cancellation model training method, an echo cancellation method, equipment, a medium and a product. BACKGROUND

[0002] In a scenario where a terminal at one side communicates with a terminal at the other side, an acoustic echo is usually generated, that is, the sound played by a loudspeaker is picked up by a microphone and sent back to the terminal at the other side, so that the user of the terminal at the other side hears the echo signal of himself.

[0003] An echo cancellation model based on deep learning can be used for echo cancellation, however, the parameter quantity and the calculation complexity of these models are high, and the dependence on training data is also strong. In actual scenarios, in order to improve the applicability of the model, especially to enable the model to be deployed on a device with low calculation resources and low storage resources for echo cancellation, knowledge distillation is usually used to reduce the volume of the model while maintaining the performance.

[0004] However, the traditional knowledge distillation method usually uses a single teacher model to guide the student model, which cannot comprehensively capture and transfer diversified knowledge, which is particularly disadvantageous to maintaining the robustness of the model in a complex acoustic environment, resulting in a single echo cancellation performance of the student echo cancellation model obtained by distillation, which is difficult to balance echo cancellation and voice preservation. SUMMARY

[0005] The main purpose of the present application is to provide an echo cancellation model training method, an echo cancellation method, equipment, a medium and a product, which aims to solve the technical problem that the current echo cancellation model is difficult to balance echo cancellation and voice preservation.

[0006] To achieve the above purpose, the present application provides an echo cancellation model training method, which comprises the following steps:

[0007] Obtain training data, an echo cancellation model to be trained, and a pre-constructed dynamic selection network, wherein the dynamic selection network comprises a fusion model and a plurality of pre-trained teacher models, the fusion model is used to fuse the outputs of each teacher model, and each teacher model at least comprises an echo cancellation task model and a voice preservation task model;

[0008] Input the training data into the echo cancellation model to obtain a student echo cancellation result, and input the training data into the dynamic selection network to obtain a fusion echo cancellation result;

[0009] Calculate a knowledge distillation loss based on the student echo cancellation result and the fusion echo cancellation result;

[0010] optimizing the echo cancellation model based on the knowledge distillation loss, and returning to performing the step of inputting the training data into the echo cancellation model to obtain a student echo cancellation result and inputting the training data into the dynamic selection model to obtain a fused echo cancellation result until a preset training end condition is met, to obtain the trained echo cancellation model.

[0011] In an embodiment, the step of calculating the knowledge distillation loss based on the student echo cancellation result and the fused echo cancellation result comprises:

[0012] inputting the student echo cancellation result into a preset self-supervised learning model to obtain student deep features and inputting the fused echo cancellation result into the self-supervised learning model to obtain teacher deep features, wherein the self-supervised learning model comprises a large-scale self-supervised pre-training WavLM model of full-stack speech processing.

[0013] inputting the student deep features and the teacher deep features into a first preset loss function to obtain the knowledge distillation loss.

[0014] In an embodiment, after the step of calculating the knowledge distillation loss based on the student echo cancellation result and the fused echo cancellation result, the method further comprises:

[0015] obtaining a current iteration number of the echo cancellation model;

[0016] if the current iteration number meets a preset condition, freezing model parameters of the fusion model and performing the step of optimizing the echo cancellation model based on the knowledge distillation loss;

[0017] if the current iteration number does not meet the preset condition, freezing model parameters of the echo cancellation model and optimizing model parameters of the fusion model.

[0018] In an embodiment, the step of optimizing the model parameters of the fusion model comprises:

[0019] obtaining a second-order gradient of a second preset loss function and optimizing the fusion model based on the second-order gradient.

[0020] The second preset loss function is a loss function used when calculating a hard label loss of the echo cancellation model, and the hard label loss is a loss value between the student echo cancellation result and a training label corresponding to the training data.

[0021] In an embodiment, the preset condition is that the current iteration number is odd.

[0022] In an embodiment, the fusion model comprises an encoder, a self-attention network, a deep learning Mamba network based on sequence modeling, a decoder, and an output layer connected in sequence.

[0023] In addition, to achieve the above object, the present application also provides an echo cancellation method, comprising the following steps:

[0024] An original signal to be echo cancelled and a target echo cancellation model are obtained, and the original signal is input into the target echo cancellation model to obtain an echo cancellation result.

[0025] The target echo cancellation model is an echo cancellation model trained by using the echo cancellation model training method.

[0026] In addition, to achieve the above object, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the echo cancellation model training method and / or the echo cancellation method.

[0027] In addition, to achieve the above object, the present application also provides a readable storage medium, which is a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the echo cancellation model training method and / or the echo cancellation method.

[0028] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the echo cancellation model training method and / or the echo cancellation method.

[0029] The one or more technical solutions provided by the present application have at least the following technical effects:

[0030] The process involves acquiring training data, an echo cancellation model to be trained, and a pre-built dynamic selection network. The dynamic selection network includes a fusion model and multiple pre-trained teacher models. The fusion model is used to fuse the outputs of each teacher model, and each teacher model includes at least an echo cancellation task model and a voice preservation task model. The training data is input into the echo cancellation model to obtain student echo cancellation results, and then input into the dynamic selection network to obtain fused echo cancellation results. A knowledge distillation loss is calculated based on the student echo cancellation results and the fused echo cancellation results. The echo cancellation model is optimized based on the knowledge distillation loss. The process then returns to the steps of inputting the training data into the echo cancellation model to obtain student echo cancellation results and inputting the training data into the dynamic selection model to obtain fused echo cancellation results, until a preset training termination condition is met, resulting in a trained echo cancellation model. Thus, in this embodiment, the outputs of multiple pre-trained teacher models are fused using a fusion model. The knowledge distillation loss is calculated by combining the fused echo cancellation result with the student cancellation result output by the echo cancellation model. Based on this knowledge distillation loss, the echo cancellation model is optimized. In this case, the echo cancellation model also serves as the student model in the distillation process, thereby achieving the goal of multiple teacher models guiding the student model, rather than using a single teacher model to guide the student model. This allows the student model to comprehensively capture and learn the diverse knowledge of multiple teacher models. Furthermore, since the teacher model includes at least a voice preservation task model and an echo cancellation task model, the student model can comprehensively capture and learn the knowledge of the teacher model in both voice preservation and echo cancellation, allowing the student model to take into account both echo cancellation and voice preservation. In other words, the echo cancellation model can take into account both echo cancellation and voice preservation. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart illustrating the first embodiment of the echo cancellation model training method of this application;

[0034] Figure 2 This is a schematic diagram of the fusion model structure involved in an embodiment of the echo cancellation model training method of this application;

[0035] Figure 3 A schematic diagram of a teacher model structure involved in an embodiment of the echo cancellation model training method of the present application;

[0036] Figure 4 A schematic diagram of an echo cancellation model structure involved in an embodiment of the echo cancellation model training method of the present application;

[0037] Figure 5 A schematic diagram of a brief process of the echo cancellation model training method of the present application;

[0038] Figure 6 A schematic diagram of a device structure of the echo cancellation model training device of the present application;

[0039] Figure 7 A schematic diagram of a device structure of a hardware running environment involved in the echo cancellation model training method device in an embodiment of the present application.

[0040] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0041] In order to make the above purposes, features and advantages of the present application more apparent and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0042] Acoustic echo cancellation (AEC) technology plays a crucial role in audio communication, aiming to eliminate the coupling echo between the local loudspeaker and microphone of the far-end signal, thereby improving the communication quality. Current AEC methods can be divided into traditional adaptive filter algorithms and deep learning-based algorithms. In traditional adaptive filter methods, techniques such as least mean squares, normalized least mean squares, proportionally normalized least mean squares, and sub-band adaptive filters are mainly used. These methods adjust the filter weights to adapt to the changes in the echo path, with relatively low computational complexity, but have limitations in dealing with nonlinear distortion or long-term dependencies. Deep learning-based AEC methods use deep feedforward neural networks, convolutional neural networks, and recurrent neural networks to enhance echo cancellation performance, achieving excellent performance. However, these models have high parameter quantities and computational complexity, and are strongly dependent on training data. In addition, hybrid models combine linear filters with neural networks to balance robustness and adaptability.

[0043] In actual scenarios, especially on low-power embedded devices and mobile terminals, the deployment of AEC models is usually limited by computing resources and storage space. Deep neural networks, although perform well, are difficult to apply under low latency and low power requirements. Therefore, under the condition of limited computing, storage, and other resources, how to design an efficient AEC model is still a challenge that needs to be broken through.

[0044] In recent years, knowledge distillation (KD) technology has been widely applied in the field of deep learning to reduce the size of the model while maintaining performance. However, in the field of acoustic echo cancellation, knowledge distillation technology faces significant challenges in simultaneously ensuring echo removal and near-talker voice quality, especially in low-parameter and low-computing-resource environments. Traditional knowledge distillation methods usually use a single teacher model to guide the student model, which cannot fully capture and transfer diverse knowledge, which is particularly disadvantageous for maintaining robustness in complex acoustic environments. Therefore, an acoustic echo cancellation framework that effectively integrates the advantages of multiple teacher models and is suitable for low-parameter and low-computing-resource conditions is needed to achieve the best balance between echo cancellation and voice preservation.

[0045] Based on this, the main solution of the present application is: obtaining training data, an echo cancellation model to be trained, and a pre-constructed dynamic selection network, wherein the dynamic selection network includes a fusion model and a plurality of pre-trained teacher models, the fusion model is used to fuse the outputs of each teacher model, and each teacher model at least includes an echo cancellation task model and a voice preservation task model; inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection network to obtain a fused echo cancellation result; calculating a knowledge distillation loss based on the student echo cancellation result and the fused echo cancellation result; optimizing the echo cancellation model based on the knowledge distillation loss, returning to perform the steps of inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection model to obtain a fused echo cancellation result, until a pre-set training end condition is met, and a trained echo cancellation model is obtained.

[0046] The application fuses the output results of multiple pre-trained teacher models through a fusion model, calculates a knowledge distillation loss based on the fusion echo cancellation result obtained after fusion and the student cancellation result output by the echo cancellation model, and optimizes the echo cancellation model based on the knowledge distillation loss. At this time, the echo cancellation model is a student model in the distillation process, so as to achieve the purpose of guiding the student model by multiple teacher models, instead of using a single teacher model to guide the student model, so that the student model can comprehensively capture and learn the diversified knowledge of multiple teacher models. Since the teacher model at least includes a voice preservation task model and an echo cancellation task model, the student model can comprehensively capture and learn the knowledge of the teacher model in voice preservation and echo cancellation, so that the student model can balance echo cancellation and voice preservation, that is, the echo cancellation model can balance echo cancellation and voice preservation.

[0047] It should be noted that the execution subject of each embodiment of the echo cancellation model training method of the application can be a computing service device with data processing, model communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, such as a headset, AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, an AR helmet, a VR helmet, smart glasses, etc. The echo cancellation model training method of the application does not make specific limitations on this.

[0048] Based on this, the echo cancellation model training method of the first embodiment is proposed. Figure 1 As shown in the figure, the echo cancellation model training method includes the following steps S10-S50:

[0049] Step S10, obtaining training data, an echo cancellation model to be trained, and a pre-constructed dynamic selection network, wherein the dynamic selection network includes a fusion model and multiple pre-trained teacher models, the fusion model is used to fuse the outputs of each teacher model, and each teacher model at least includes an echo cancellation task model and a voice preservation task model;

[0050] At least two teacher models can be pre-trained on different training data sets in advance, so as to capture general features and patterns in the data. Each teacher model has different performance on different tasks, and the teacher model includes but is not limited to an echo cancellation task model and a voice preservation task model, such as a double-talking balance task model.

[0051] It should be noted that in the embodiment, defining the model as a certain task model means that the performance of the model on the task meets certain performance outstanding conditions, that is, the model performs outstandingly on the task, such as an echo cancellation task model, which means that the model performs outstandingly on the echo cancellation task and can well eliminate the echo in the signal, a human voice reservation task model, which means that the model performs outstandingly on the human voice reservation task and can well reserve the echo in the signal, and a double-speaking balance task model, which means that the model performs outstandingly on the double-speaking balance task and can well balance the echo cancellation and human voice reservation in the scenes of double speaking and single speaking.

[0052] Further, the relevant personnel can pre-set the condition for evaluating whether the performance of the model on a certain performance is outstanding. For example, in a specific embodiment, the performance of the model can be evaluated using an AECMOS (Acoustic Echo Control Mean Opinion Score) index to obtain an echo mos (Echo Mean Opinion Score) and a deg mos (Degradation Mean Opinion Score), if the echo mos is greater than 3 and less than or equal to 5, it is determined that the model performs outstandingly on the echo cancellation task, if the deg mos is greater than 3 and less than or equal to 5, it is determined that the model performs outstandingly on the human voice cancellation task, and if the echo mos is greater than 1 and less than or equal to 3 and the deg mos is greater than 1 and less than or equal to 3, it is determined that the model performs outstandingly on the double-speaking balance task.

[0053] Further, to pre-train the teacher model with outstanding performance of each performance, the relevant personnel can pre-train the teacher model using the corresponding training data set to obtain the teacher model with outstanding performance of each performance.

[0054] For the convenience of subsequent elaboration and description, exemplary, the pre-trained teacher model is taken as an example for the echo cancellation task model, the human voice reservation task model and the double speaking balance task model. In a specific embodiment, in order to pre-train the echo cancellation task model, the echo dominant scene can be simulated, the signal strength of the near-end speech is reduced, the near-end speech is mixed with different background noise, various noise environments are simulated, nonlinear distortion, reverberation effect and dynamic echo change are added to the mixed signal, and the noisy echo speech is obtained. The teacher model is pre-trained with the noisy echo speech as the input and the clean speech as the label to obtain the echo cancellation task model. In order to pre-train the human voice reservation task model, the near-end speech can be ensured to be dominant, the added reverberation and nonlinear distortion to the noise can be reduced, the signal with high speech fidelity can be generated preferentially, excessive delay and signal distortion can be avoided, and the model can focus on restoring the naturalness and clarity of the speech. That is, the teacher model can be pre-trained with the mixed signal with a signal-to-noise ratio greater than a certain threshold as the input and the clean speech as the label to obtain the human voice reservation task model. In order to pre-train the double speaking balance task model, the distribution of double speaking and single speaking scenes can be balanced, the mixed signal of speech and echo in various scenes such as silence, near-end single speaking, far-end single speaking and double speaking can be generated, the scene category can be labeled through voice activity detection, and the generalization ability of the diversified reverberation and noise enhancement dataset can be combined. That is, the teacher model can be pre-trained with the mixed signal in multiple scenes as the input and the clean speech and the scene category as the label to obtain the double speaking balance task model.

[0055] After pre-training the teacher models, the student model is trained using the teacher models in subsequent. In this embodiment, the student model is an echo cancellation model to be trained.

[0056] Considering that knowledge distillation uses the output of a teacher model to guide the student model, and multiple teacher models correspond to multiple outputs, in order to make the multiple outputs can be effectively used to guide the student model, the outputs of the multiple teacher models are fused by using a fusion model in this embodiment, so as to fuse the multiple outputs into one output, and then the principle of knowledge distillation is still used to guide the student model based on the one fused output.

[0057] The relevant personnel can pre-construct the fusion model. For example, in a specific embodiment, the fusion model is a MetaNet (Meta Networks, meta learning network).

[0058] It should be noted that the outputs of the teacher models are connected to the inputs of the fusion model, so as to receive the outputs of the teacher models through the fusion model and fuse the outputs of the teacher models.

[0059] Step S20, inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection network to obtain a fusion echo cancellation result;

[0060] After obtaining the training data, the training data are input into the echo cancellation model and the dynamic selection network respectively to obtain a student echo cancellation result representing an output of the student model and a fusion echo cancellation result representing an output of the teacher model.

[0061] It should be noted that the student echo cancellation result can be specifically a final output of the echo cancellation model, and the fusion echo cancellation result can be specifically a final output of the dynamic selection network, that is, a final output of the fusion model.

[0062] Step S30, calculating a knowledge distillation loss based on the student echo cancellation result and the fusion echo cancellation result;

[0063] The knowledge distillation loss is calculated based on the student echo cancellation result and the fusion echo cancellation result. Specifically, a loss function (hereinafter referred to as a first preset loss function) can be set in advance, and the student echo cancellation result and the fusion echo cancellation result are input into the first preset loss function to calculate the knowledge distillation loss.

[0064] It should be noted that the first preset loss function can be any loss function set in advance, such as a cross loss function, a logarithmic loss function, etc., and the present embodiment does not make a specific limitation thereto.

[0065] Step S40, optimizing the echo cancellation model based on the knowledge distillation loss.

[0066] If the preset training end condition is met, step S50 of obtaining the trained echo cancellation model is performed.

[0067] If the preset training end condition is not met, step S20 is returned to be performed until the preset training end condition is met.

[0068] It is easy to understand that during knowledge distillation, a knowledge distillation loss (also referred to as a soft label loss or a class probability distillation loss) between outputs of the teacher network and the student network is calculated, and a loss of the student network on a real label is also usually calculated, which is referred to as a hard label loss or a standard loss (Standard Loss). Specifically, the overall loss function of knowledge distillation is usually a combination of the two losses. Based on this, the present embodiment can also calculate a hard label loss between the student echo cancellation result and a training label, and optimize the echo cancellation model based on the knowledge distillation loss and the hard label loss, wherein the training label is a real label of the training data.

[0069] The echo cancellation model is optimized based on a knowledge distillation loss and a hard label loss. Specifically, the knowledge distillation loss and the hard label loss can be reduced as optimization objectives to update model parameters in the echo cancellation model. For example, a gradient descent algorithm can be used to calculate a gradient value corresponding to the model parameters in the echo cancellation model according to a sum value of the knowledge distillation loss and the hard label loss, and the model parameters in the echo cancellation model are updated according to the gradient value, so as to achieve the purpose of optimizing the echo cancellation model.

[0070] The training end condition can be a condition set in advance, such as reaching a predetermined number of iterations, a loss function value being lower than a predetermined threshold, running out of computing resources, reaching a time limit, an accuracy reaching a predetermined threshold, and the like. The present embodiment does not make a specific limitation on this.

[0071] The present embodiment fuses the output results of multiple pre-trained teacher models through a fusion model, calculates a knowledge distillation loss based on the fusion echo cancellation result obtained after fusion and the student echo cancellation result output by the echo cancellation model, and optimizes the echo cancellation model based on the knowledge distillation loss. At this time, the echo cancellation model is also a student model in the distillation process, so as to achieve the purpose of guiding the student model by multiple teacher models, instead of using a single teacher model to guide the student model. Thus, the student model can comprehensively capture and learn the diversified knowledge of multiple teacher models. Since the teacher model at least includes a voice preservation task model and an echo cancellation task model, the student model can comprehensively capture and learn the knowledge of the teacher model in voice preservation and echo cancellation, so that the student model can balance echo cancellation and voice preservation, that is, the echo cancellation model can balance echo cancellation and voice preservation. The echo cancellation model can maintain high echo cancellation capability and voice preservation effect under the condition of low calculation amount and low parameter amount. The user experience can be improved without significantly increasing the model parameter amount, especially in mobile devices and low-power scenarios. The adaptability of the echo cancellation model to complex acoustic environments is improved, and the robustness and generalization capability of the model are enhanced.

[0072] Based on the first embodiment of the present application, the same or similar contents as the above embodiment one can be referred to the above description, and will not be described hereinafter. On this basis, the step of calculating a knowledge distillation loss based on the student echo cancellation result and the fusion echo cancellation result includes:

[0073] Step A10, inputting the student echo cancellation result into a preset self-supervised learning model to obtain student deep features, and inputting the fusion echo cancellation result into the self-supervised learning model to obtain teacher deep features, wherein the self-supervised learning model includes a large-scale self-supervised pre-training WavLM model of full-stack speech processing;

[0074] It should be noted that the relevant personnel can also set other self-supervised learning models (Self-Supervised Learning, SSL) as the preset self-supervised learning model, and the present embodiment does not make specific limitations thereon. Exemplarily, the preset self-supervised learning model is taken as a WavLM (Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing) model, and the present application is described and explained in each embodiment.

[0075] Further, the WavLM model can be a frozen parameter model, that is, the model parameters of the WavLM model are not updated in the training process of the echo cancellation model.

[0076] The student echo cancellation result and the fusion echo cancellation result are respectively input into the WavLM model to respectively extract deep features in the echo cancellation result and the fusion echo cancellation result, and the deep features extracted from the student echo cancellation result are determined as student deep features, and the deep features extracted from the fusion echo cancellation result are determined as teacher deep features.

[0077] Further, the deep features (including student deep features and teacher deep features) can be specifically the output of one or more layers of the WavLM model. For example, in a specific embodiment, the deep features are the output of the N-10th to Nth layers of the WavLM model, where N is the number of the last layer of the WavLM model, or the total number of network layers of the WavLM model.

[0078] Further, the deep features can specifically include one or more of semantic information (such as the meaning of words or phrases in the audio), speaker features (such as voiceprints, speaker identity), emotional features (such as joy, anger, etc. emotions in the voice), and context information (such as the context of the voice, interrogative sentences, declarative sentences, etc.), and the actual output of the WavLM model is specifically used, and the present embodiment does not make specific limitations thereon.

[0079] Step A20, inputting the student deep features and the teacher deep features into a first preset loss function to obtain a knowledge distillation loss.

[0080] In this embodiment, the student echo cancellation result and the fusion echo cancellation result are input into the WavLM model, deep features are extracted using the WavLM model, and a knowledge distillation loss is calculated based on the extracted deep features. It can be understood that the deep features extracted by the WavLM model can better represent the depth information of the features, that is, more deeply reflect the difference between the echo cancellation model and the teacher model, and then the knowledge distillation loss is calculated based on the deep features to optimize the echo cancellation model, which can further improve the learning effect of the echo cancellation model.

[0081] Based on the first and / or second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as the above embodiments one and two can be referred to the above introduction, and the subsequent will not be repeated. On this basis, after the step of calculating the knowledge distillation loss based on the student echo cancellation result and the fusion echo cancellation result, the method further comprises:

[0082] Step B10, obtaining the current iteration number of the echo cancellation model;

[0083] Step B20, if the current iteration number meets a preset condition, freezing the model parameters of the fusion model, and executing the step of optimizing the echo cancellation model based on the knowledge distillation loss;

[0084] When the current iteration number meets the preset condition, the model parameters of the fusion model are frozen, that is, the model parameters of the fusion model are not updated, and the model parameters of the echo cancellation model are updated in this iteration.

[0085] The preset condition can be any condition set by relevant personnel based on actual needs, which is not limited here. In a preferred embodiment, the preset condition is that the current iteration number is odd, so as to alternately update the model parameters of the echo cancellation model and the fusion model, and ensure the balanced training and common learning optimization of the echo cancellation model and the fusion model.

[0086] Step B30, if the current iteration number does not meet the preset condition, freezing the model parameters of the echo cancellation model, and optimizing the model parameters of the fusion model.

[0087] When the current iteration number does not meet the preset condition, the model parameters of the echo cancellation model are frozen, that is, the model parameters of the echo cancellation model are not updated, and the model parameters of the fusion model are updated in this iteration.

[0088] The student echo cancellation result output by the echo cancellation model and the fusion echo cancellation result output by the dynamic selection network are used for knowledge distillation calculation, and then gradient back propagation is performed. If the fusion model is updated together with the echo cancellation model, the output result of the dynamic selection network may temporarily not reach the output result of the echo cancellation model with excellent performance. Based on this, in the embodiment, the echo cancellation model and the fusion model update the model parameters alternately, so that the training of the fusion model and the echo cancellation model is more stable and easier to converge.

[0089] Further, the model parameters of the fusion model are updated in the embodiment, so that the fusion model can automatically select the teacher output most suitable for the student model learning, dynamically adapt to different acoustic environments, effectively integrate the knowledge of multiple teacher models, and optimize the learning process of the student model, thereby realizing high-quality acoustic echo cancellation even in a resource-limited environment.

[0090] In a possible implementation, the step of optimizing the model parameters of the fusion model comprises:

[0091] In step C10, a second-order gradient of a second preset loss function is obtained, and the fusion model is optimized based on the second-order gradient. The second preset loss function is a loss function used when calculating the hard label loss of the echo cancellation model, and the hard label loss is a loss value between the student echo cancellation result and the training label corresponding to the training data.

[0092] Similar to the first preset loss function, the second preset loss function can be any loss function set in advance, such as a cross loss function or a logarithmic loss function, and the embodiment does not make a specific limitation on this. For example, in a specific implementation, the second preset loss function is a mean squared error (MSE, Mean Squared Error) loss. Let the student echo cancellation result be , and the training label be , the MSE loss function is defined as: , wherein the data amount of the training data, and are the corresponding values of the student echo prediction result and the training label of the i-th training data, respectively.

[0093] Let the second preset loss function be L MSE , the first-order gradient is calculated first, and then the second-order derivative is obtained.

[0094] ​In this embodiment, the fusion model is updated using a quadratic gradient based on a second preset loss function. This method of updating model parameters not only utilizes gradient information but also considers the Hessian matrix (second derivative) of the loss function, which reflects the curvature of the objective function. This allows the optimizer to adjust the step size and direction based on the "curvature" of the function when updating parameters. For example, in steep regions, the Hessian matrix can guide the optimizer to use a smaller step size, while in flat regions, a larger step size can be used to avoid excessively large or small parameter updates, thus making the training of the fusion model more stable and easier to converge.

[0095] Based on the first, second, and / or third embodiments of this application, in the fourth embodiment of this application, the content that is the same as or similar to the first, second, and third embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, refer to... Figure 2 As shown, the fusion model includes an encoder, a self-attention network, a sequence-modeling-based deep learning Mamba network, a decoder, and an output layer connected in sequence. Figure 2 (Not shown in the image).

[0096] An encoder is used to transform input data into a more compact and easier-to-process internal representation. The specific structure of the encoder can be pre-defined. For example, in one embodiment, the encoder includes a first convolutional layer, a second convolutional layer, and an input projection layer connected in sequence. The kernel size of the convolutional layers is (3, 5), the padding is (1, 2), and the number of output channels is the same as the input. Each convolutional layer is followed by a ReLU (Rectified Linear Unit) activation function to enhance the non-linear feature representation capability. The input projection layer inputs the convolutional features into the linear projection layer, with the feature dimensions being the same as the input.

[0097] Self-attention networks are used to capture long-range dependencies in sequences and extract high-level features from input data. The specific structure of the self-attention network can be pre-defined. For example, in one implementation, the self-attention network includes a position encoder and a Transformer encoder connected in sequence. The position encoder uses two-dimensional positional encoding to encode temporal positional information into features, enhancing the model's sensitivity to temporal and frequency variations. The positional encoding is generated using sine and cosine functions, while the time and frequency axes are encoded using different scales. The Transformer encoder is an encoder stack consisting of four sequentially connected encoder layers, each with eight attention heads. The encoder employs a self-attention mechanism to capture global dependencies in the sequence.

[0098] The Mamba network is used for predicting sequence data, and the relevant person can pre-set the network parameters of the Mamba network. For example, in an embodiment, the Mamba network is used as a mixer class, the convolution parameter d_conv = 4, the state dimension d_state = 16, and the expansion factor expand = 4, the network includes forward and backward blocks, and the forward and reverse feature representations are combined in a bidirectional model.

[0099] The decoder is used to generate the target sequence, and the relevant person can pre-set the specific structure of the decoder. For example, in an embodiment, the decoder includes an output projection layer and a deconvolution layer connected in sequence. The output projection layer projects the output of the Mamba network to the d_model dimension again, and the output is further optimized through a residual connection. The deconvolution layer includes two deconvolution operations, and the deconvolution kernel size is (3, 5) and the padding is (1, 2). The output after deconvolution is normalized by softmax to generate the final weight matrix for synthesizing the output signal.

[0100] The output layer is used to fuse the outputs of the teacher models based on the weight matrix output by the decoder to obtain the fused echo prediction result. Specifically, the teacher model output signal can be taken as input, and the teacher amplitude is weighted by the weight generated by the encoder, the decoder and the Mamba network. The weighted amplitude is combined with the phase information of the teacher model to form a complex signal in the frequency domain, and after summing in the teacher dimension, an inverse Fourier transform is performed to generate the final output audio signal.

[0101] For example, assuming that three pre-trained teacher models are teacher model 1, teacher model 2 and teacher model 3, the training data is input into the three teacher models to obtain output signal 1, output signal 2 and output signal 3, and the output signal 1, output signal 2 and output signal 3 are input into the fusion model, and the weight matrix generated by the encoder, the decoder and the Mamba network is

weight 1, weight 2, weight 3

[0102] Further, in an embodiment, with reference to Figure 3As shown, each teacher model consists of an encoder, a bottleneck layer, an alignment layer, and a decoder.

[0103] wherein the encoder comprises a near-end signal branch and a far-end signal branch, the near-end signal is input to the near-end signal branch, and the far-end signal is input to the far-end signal branch. The near-end branch comprises five encoding blocks, while the far-end signal branch comprises two encoding blocks followed by an alignment layer. The alignment layer uses a multi-head attention mechanism to align the near-end features and the far-end features in time. After alignment, the far-end and near-end features are concatenated together and input into the fifth encoding block.

[0104] Each encoding block consists of a down-sampling convolutional layer, a BatchNorm (BN) layer, an ELU (Exponential Linear Unit) activation function, and a Residual Block (RB). The first encoding block uses 16 filters, the second encoding block uses 64 filters, the third encoding block uses 32 filters, and the fourth to seventh encoding blocks each use 64 filters. The kernel size of the down-sampling convolutional layer is 4x3 with a stride of 1x2, which is mainly used to reduce the frequency dimension in the frequency axis. All convolution operations are causal convolutions, i.e., without look-ahead operation, to ensure that only past inputs are relied on.

[0105] In the alignment layer, the near-end features output by the fourth encoding block are and the far-end features output by the second encoding block are These features are processed through a point convolution layer to generate query (Q) and key (K) tensors, represented as and respectively, where h is the number of similar channels, t is the time, f is the frequency, and c is the feature channel. Then K is unfolded in the time axis to form a delay dimension, becoming where is the maximum echo delay (in time frames). Subsequently, a dot product is performed on the query and the unfolded key in the frequency axis, resulting in . The dot product result is input into a convolutional layer with a kernel size of 5x3, padding of 3x1, and a stride of 1. The convolutional layer uses one filter to combine h similar channels into one attention head. Finally, a softmax operation is performed on the delay axis to output the delay probability distribution . Then, the far-end features are weighted and summed along the time axis using the delay probability weights to obtain the aligned far-end features, which are input into the fifth encoding block.

[0106] The bottleneck layer is located between the encoder and the decoder, which is composed of a Mamba network and a linear projection layer connected in sequence. The input features are first flattened in the channel and frequency dimensions to obtain , which is then input into the Mamba network and then passed to the linear projection layer. The output of the linear projection layer is readjusted to . Compared with LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit), the Mamba network can have higher performance with similar parameter amount, so it is used as the bottleneck layer to improve the performance of the model. Using linear projection after the recurrent layer (i.e., the Mamba network) can reduce the number of hidden units, thereby improving the performance and stability of training.

[0107] The decoder is composed of five decoding blocks. Except for the fifth decoding block, the first decoding block to the fourth decoding block each includes a skip connection block, a residual block, a sub-pixel convolution block, batch normalization, and an ELU activation function. The last decoding block includes a skip connection block, a residual block, and a sub-pixel convolution block. The number of filters in the decoder only changes in the sub-pixel convolution block, and the tensor shape remains unchanged in the remaining modules. The number of filters in the sub-pixel convolution block in the first to fifth decoding blocks is 64, 64, 64, 32, and 32, respectively, each with a convolution kernel size of 4x3 and a step size of 1. All convolution operations are causal convolutions.

[0108] In addition, residual blocks are added in the encoding blocks and the decoding blocks to improve the capacity of the network while not hindering the flow of gradients. The residual block includes a convolution layer, batch normalization, and an ELU activation function, which is defined as follows: , where X and Y represent the input and output tensors, represents batch normalization, represents a two-dimensional convolution, is an activation function. The convolution layer has the same number of filters as the input, with a kernel size of 4x3 and a step size of 1, and causal padding ensures that the input shape remains unchanged.

[0109] Further, in a specific embodiment, as shown in Figure 4 , the echo cancellation model is composed of an encoder, a bottleneck layer, and a decoder connected in sequence. Compared with the teacher model, the alignment layer is removed, and the feature dimensions of the decoder and the encoder are reduced to reduce the computational load and complexity.

[0110] The basic structure of the encoder remains the same as that of the teacher model, but the third encoding block and the fourth encoding block each use 32 filters, and the fifth encoding block to the seventh encoding block each use 48 filters. The first encoding block uses 16 filters, and the second encoding block uses 32 filters.

[0111] The basic structure of the decoder remains the same as the teacher model, but the number of filters in the decoder only changes in the sub-pixel convolution block, and the remaining modules remain unchanged in tensor shape. The number of filters in the sub-pixel convolution block is 48, 48, 48, 32, and 32 in turn, and each convolution kernel size is 4x3 with a step size of 1. All convolution operations are causal convolution.

[0112] The echo cancellation model has a basic structure similar to the teacher model, but the number of model parameters is significantly reduced. For example, when the teacher model and the echo cancellation model use the above parameter design, the parameter amount of the teacher model is about 1.01M, and the parameter amount of the student model is about 0.26M (260K). The parameter amount is significantly reduced to facilitate the deployment of the echo cancellation model on devices with lower computing power for echo cancellation.

[0113] For example, to help understand the technical concept or technical principle of the echo cancellation model training method combined with the first embodiment, the second embodiment, and the third embodiment, please refer to Figure 5 As shown in the figure, the training method of the echo cancellation model includes:

[0114] Step one: input the training data into the pre-trained teacher model 1, teacher model 2, and teacher model 3 respectively, use the pre-trained teacher model to perform echo cancellation on the training data, and obtain the fusion echo cancellation result through the MetaNet fusion model.

[0115] Step two: input the training data into the echo cancellation model to be trained, that is, the student model, and output the student echo cancellation result.

[0116] Step three: take the clean speech as the training label, and input the student echo cancellation result estimated by the student model into the first loss function to calculate the hard label loss;

[0117] Step four: take the fusion echo cancellation result estimated by the MetaNet fusion model as the label, and input the student echo cancellation result estimated by the student model into the WavLM model, respectively, to obtain the student deep feature and the teacher deep feature, and based on their deep features and the second loss function calculate the knowledge distillation loss;

[0118] Step five: update the parameters of the student model according to the two losses obtained by backpropagation;

[0119] Step six: take the clean speech as the label, and update the parameters of the MetaNet fusion model with the second gradient of the first loss function .

[0120] It should be noted that only step five or step six is performed in different training rounds, and in an embodiment, steps five and six can be alternately performed in the manner that step five is performed in the first to 100th training rounds, step six is performed in the 101st training round, step five is performed in the 102nd to 201st training rounds, step six is performed in the 202nd training round, and so on, that is, the student model and the MetaNet fusion model are alternately updated at the alternation frequency of 100 times of student model update and 1 time of MetaNet fusion model update.

[0121] It should be noted that the above examples are only used to assist in understanding the present application and do not constitute a limitation on the echo cancellation model training method of the present application. Further forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0122] In addition, the present application also proposes an echo cancellation method, which comprises the following steps:

[0123] obtaining an original signal to be echo cancelled and a target echo cancellation model, and inputting the original signal into the target echo cancellation model to obtain an echo cancellation result;

[0124] The target echo cancellation model is an echo cancellation model trained by the echo cancellation model training method of any one of the above embodiments.

[0125] The execution subject of each embodiment of the echo cancellation method can be a computing service device with data processing, model communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, such as a headset, AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, an AR helmet, a VR helmet, smart glasses, etc. The present embodiment does not make a specific limitation on this. Further, the execution subject of the echo cancellation method and the echo cancellation model training method of the present application can be the same or different, which is not specifically limited herein.

[0126] It is easy to understand that after obtaining the original signal to be echo cancelled, the original signal is input into the target echo cancellation model to obtain the echo cancellation result, which can be the output of the target echo cancellation model.

[0127] In addition, the present application also proposes an echo cancellation model training device, which is shown in Figure 6 The echo cancellation model training device comprises:

[0128] An acquisition module 10 is configured to acquire training data, an echo cancellation model to be trained, and a pre-built dynamic selection network, wherein the dynamic selection network comprises a fusion model and a plurality of pre-trained teacher models, the fusion model is configured to fuse outputs of the teacher models, and each of the teacher models comprises at least an echo cancellation task model and a vocal preservation task model;

[0129] An input module 20 is configured to input the training data into the echo cancellation model to obtain a student echo cancellation result, and input the training data into the dynamic selection network to obtain a fusion echo cancellation result;

[0130] A calculation module 30 is configured to calculate a knowledge distillation loss based on the student echo cancellation result and the fusion echo cancellation result;

[0131] An optimization module 40 is configured to optimize the echo cancellation model based on the knowledge distillation loss until a preset training end condition is met, and obtain a trained echo cancellation model.

[0132] In an embodiment, the calculation module 30 is further configured to:

[0133] input the student echo cancellation result into a preset self-supervised learning model to obtain student deep features, and input the fusion echo cancellation result into the self-supervised learning model to obtain teacher deep features, wherein the self-supervised learning model comprises a large-scale self-supervised pre-training WavLM model of a full-stack speech processing;

[0134] input the student deep features and the teacher deep features into a first preset loss function to obtain the knowledge distillation loss.

[0135] In an embodiment, the optimization module 40 is further configured to:

[0136] acquire a current iteration number of the echo cancellation model;

[0137] if the current iteration number meets a preset condition, freeze model parameters of the fusion model, and optimize the echo cancellation model based on the knowledge distillation loss;

[0138] if the current iteration number does not meet the preset condition, freeze model parameters of the echo cancellation model, and optimize model parameters of the fusion model.

[0139] In an embodiment, the optimization module 40 is further configured to:

[0140] acquire a second-order gradient of a second preset loss function, and optimize the fusion model based on the second-order gradient;

[0141] The second preset loss function is a loss function used when calculating a hard label loss of the echo cancellation model, and the hard label loss is a loss value between the student echo cancellation result and a training label corresponding to the training data.

[0142] In an embodiment, the preset condition is that the current iteration number is an odd number.

[0143] In an embodiment, the fusion model comprises an encoder, a self-attention network, a deep learning Mamba network based on sequence modeling, a decoder and an output layer connected in sequence.

[0144] In addition, an electronic device is also provided in the embodiments of the present application, and the electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the echo cancellation model training method and / or the echo cancellation method.

[0145] Reference Figure 7 which shows a structural schematic diagram of an electronic device suitable for being used to implement the embodiments of the present application. The electronic device in the embodiments of the present application can also include, but is not limited to, mobile terminals such as earphones, AR glasses, VR glasses, AR helmets, VR helmets, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant: personal digital assistants), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Player: portable multimedia players) and the like, and fixed terminals such as digital TVs, desktop computers and the like. Figure 7 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0146] As Figure 7As shown, the electronic device can include a processing device 1001 (e.g., a central processor, a graphics processor, etc.) that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 1002 or loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the electronic device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or through wires to exchange data. Although the electronic device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0147] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a model through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0148] The electronic device provided by the embodiments of the present disclosure adopts the echo cancellation model training method and / or the echo cancellation method in the above embodiments, which can solve the technical problem that the current echo cancellation model is difficult to balance echo cancellation and voice reservation. Compared with the prior art, the electronic device provided by the present disclosure has the same beneficial effects as the echo cancellation model training method and / or the echo cancellation method provided by the above embodiments, and other technical features in the electronic device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0149] It should be understood that various parts of the present disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above description of embodiments, specific functional, structural, material or characteristic features are combined in a manner appropriate for the particular example or embodiment, but other examples or embodiments can combine different features in a different manner.

[0150] The above description is merely illustrative of the application and the application should not be limited thereto. Any variations and modifications of the application, which fall within the scope of the application, should be considered to be within the scope of the application. Therefore, the scope of the application should be determined by the scope of the claims.

[0151] In addition, to achieve the above object, the embodiments of the present application further provide a readable storage medium having computer readable program instructions (i.e., computer programs) stored thereon, the computer readable program instructions being used to execute the echo cancellation model training method and / or the echo cancellation method in the above embodiments.

[0152] The computer readable storage medium provided by the embodiments of the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0153] The above computer readable storage medium can be contained in an electronic device; or can exist separately without being assembled into an electronic device.

[0154] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to: obtain training data, an echo cancellation model to be trained, and a pre-constructed dynamic selection network, wherein the dynamic selection network comprises a fusion model and a plurality of pre-trained teacher models, the fusion model is used to fuse the outputs of each of the teacher models, and each of the teacher models at least includes an echo cancellation task model and a vocal preservation task model; input the training data into the echo cancellation model to obtain a student echo cancellation result, and input the training data into the dynamic selection network to obtain a fused echo cancellation result; calculate a knowledge distillation loss based on the student echo cancellation result and the fused echo cancellation result; optimize the echo cancellation model based on the knowledge distillation loss, return to perform the steps of inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection model to obtain a fused echo cancellation result, until a preset training end condition is met, and obtain a trained echo cancellation model.

[0155] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0156] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0157] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0158] The computer readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the above-mentioned echo cancellation model training method and / or echo cancellation method, and can solve the technical problem that the current echo cancellation model is difficult to balance echo cancellation and voice reservation. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the echo cancellation model training method and / or echo cancellation method provided by the above-mentioned embodiments, which will not be described here.

[0159] In addition, the embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned echo cancellation model training method and / or echo cancellation method.

[0160] The computer program product specific embodiments of the present application are basically the same as the above-mentioned echo cancellation model training and / or echo cancellation method embodiments, which will not be described here.

[0161] It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0162] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent advantages or disadvantages of the embodiments.

[0163] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software sensor, which is stored in a storage medium (such as a ROM / RAM, a magnetic disc, an optical disc) as described above, and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server or a model device, etc.) execute the methods described in the various embodiments of the present application.

[0164] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An echo cancellation model training method, characterized in that, The echo cancellation model training method comprises the following steps: obtaining training data, an echo cancellation model to be trained, and a pre-constructed dynamic selection network, wherein the dynamic selection network comprises a fusion model and a plurality of pre-trained teacher models, the fusion model is used to fuse the outputs of the teacher models, and each teacher model comprises at least an echo cancellation task model and a human voice reservation task model; inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection network to obtain a fused echo cancellation result; inputting the student echo cancellation result into a preset self-supervised learning model to obtain student deep features, and inputting the fused echo cancellation result into the self-supervised learning model to obtain teacher deep features, wherein the self-supervised learning model comprises a large-scale self-supervised pre-training WavLM model of a full-stack speech processing; inputting the student deep features and the teacher deep features into a first preset loss function to obtain a knowledge distillation loss; obtaining a current iteration number of the echo cancellation model; if the current iteration number meets a preset condition, freezing the model parameters of the fusion model, optimizing the echo cancellation model based on the knowledge distillation loss, returning to the step of inputting the training data into the echo cancellation model to obtain a student echo cancellation result, and inputting the training data into the dynamic selection network to obtain a fused echo cancellation result, and obtaining the trained echo cancellation model after a preset training end condition is met; if the current iteration number does not meet the preset condition, freezing the model parameters of the echo cancellation model, obtaining a second-order gradient of a second preset loss function, and optimizing the fusion model based on the second-order gradient; wherein the second preset loss function is a loss function used when calculating a hard label loss of the echo cancellation model, and the hard label loss is a loss value between the student echo cancellation result and a training label corresponding to the training data.

2. The echo cancellation model training method of claim 1, wherein, The preset condition is that the current iteration number is odd.

3. The echo cancellation model training method of any one of claims 1-2, wherein, The fusion model comprises an encoder, a self-attention network, a deep learning Mamba network based on sequence modeling, a decoder, and an output layer connected in sequence.

4. An echo cancellation method characterized by, The echo cancellation method comprises the following steps: obtaining an original signal to be echo cancelled and a target echo cancellation model, and inputting the original signal into the target echo cancellation model to obtain an echo cancellation result; wherein the target echo cancellation model is an echo cancellation model trained by the echo cancellation model training method of any one of claims 1 to 3.

5. An electronic device, comprising: The electronic device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the method of any one of claims 1 to 4.

6. A readable storage medium characterized by, The readable storage medium is a computer readable storage medium, and the readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the method of any one of claims 1 to 4.

7. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 4.

Images