Echo cancellation method, echo cancellation device and computer readable storage medium

Abstract

The present application discloses an echo cancellation method, an echo cancellation device and a computer-readable storage medium, the echo cancellation method comprising: obtaining a time domain received signal collected at a near end and a time domain reference signal sent at a far end; converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain respectively, to obtain a frequency domain received signal and a frequency domain reference signal; based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal, obtaining a frequency domain mask signal including a real mask and an imaginary mask; based on the frequency domain mask signal, acoustically masking the frequency domain received signal to obtain a frequency domain target signal; converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal. The echo cancellation method provided by the present application can ensure the effect of echo cancellation.

Description

Technical Field

[0001] The present application relates to the technical field of speech signal processing, and in particular to an echo cancellation method, an echo cancellation device and a computer-readable storage medium. Background Art

[0002] In video conferencing systems, acoustic coupling between microphones and speakers often generates acoustic echoes that affect voice interaction. Traditional echo cancellation methods include: first, using a finite-length linear filter to estimate the acoustic path from the speaker to the microphone in the acoustic environment to eliminate the linear echo, and then applying a gain value to the residual echo to remove the remaining nonlinear echo. In scenarios with low signal-to-echo ratio and strong nonlinear distortion, traditional methods cannot accurately estimate the ability of residual echoes, and thus easily obtain an erroneous gain value, which ultimately results in the video conferencing system being unable to suppress residual echoes and near-end voice distortion. Therefore, the traditional method of echo cancellation is not a good method. Summary of the invention

[0003] The present application provides an echo cancellation method, an echo cancellation device and a computer-readable storage medium, which can ensure the effect of echo cancellation.

[0004] A first aspect of an embodiment of the present application provides an echo cancellation method, the method comprising: acquiring a time domain received signal collected at a near end and a time domain reference signal sent at a far end; converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain, respectively, to obtain a frequency domain received signal and a frequency domain reference signal; based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal, obtaining a frequency domain mask signal including a real mask and an imaginary mask; based on the frequency domain mask signal, acoustically masking the frequency domain received signal to obtain a frequency domain target signal; converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal.

[0005] According to a second aspect of an embodiment of the present application, there is provided an echo cancellation device, which includes a processor, a memory and a communication circuit, wherein the processor is coupled to the memory and the communication circuit respectively, and program data is stored in the memory, and the processor executes the program data in the memory to implement: obtaining a time domain received signal collected at a near end and a time domain reference signal sent at a far end; converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain respectively to obtain a frequency domain received signal and a frequency domain reference signal; obtaining a frequency domain mask signal including a real mask and an imaginary mask based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal; acoustically masking the frequency domain received signal based on the frequency domain mask signal to obtain a frequency domain target signal; converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal.

[0006] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program can be executed by a processor to implement: obtaining a time domain received signal collected at a near end and a time domain reference signal sent at a far end; converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain, respectively, to obtain a frequency domain received signal and a frequency domain reference signal; based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal, obtaining a frequency domain mask signal including a real mask and an imaginary mask; based on the frequency domain mask signal, acoustically masking the frequency domain received signal to obtain a frequency domain target signal; converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal.

[0007] The beneficial effect is: the present application first obtains a frequency domain mask signal based on a frequency domain received signal and a frequency domain reference signal, and then uses the frequency domain mask signal to acoustically mask the frequency domain received signal, and the frequency domain mask signal includes a real mask and an imaginary mask. Therefore, in the process of obtaining the frequency domain target signal, the phase information and amplitude information of the frequency domain received signal are simultaneously considered. Compared with the prior art which only considers the amplitude information of the signal (the prior art sets the phase spectrum of the frequency domain target signal to be the same as the phase spectrum of the frequency domain received signal), the present application can also perform echo cancellation well in the case of low signal-to-echo ratio and strong nonlinear distortion of the intercom equipment, thereby ensuring the effect of echo cancellation. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flowchart of an implementation method of the echo cancellation method of the present application;

[0010] Figure 2 is a schematic diagram of the structure of the neural network pre-trained in this application;

[0011] Figure 3 It is a partial flow chart of another implementation method of the echo cancellation method of the present application;

[0012] Figure 4 It corresponds to Figure 3 A simple framework diagram of the method;

[0013] Figure 5 It is a structural schematic diagram of an embodiment of the echo cancellation device of the present application;

[0014] Figure 6 is a structural schematic diagram of another embodiment of the echo cancellation device of the present application;

[0015] Figure 7 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of an echo cancellation method of the present application, the method comprising:

[0018] S110: Acquire a time domain received signal collected by the near end and a time domain reference signal sent by the far end.

[0019] Specifically, the near end is the terminal that needs to perform echo cancellation on the voice signal it receives, and the far end is the terminal that is talking to the near end. For example, when terminal A and terminal B are talking, when the voice signal received by terminal A needs to be echo cancelled, terminal A is the near end and terminal B is the far end; when the voice signal received by terminal B needs to be echo cancelled, terminal B is the near end and terminal A is the far end.

[0020] The time domain received signal is the voice signal collected by the near end, and the time domain reference signal is the voice signal sent by the far end to the near end.

[0021] Among them, the purpose of the echo cancellation method of the present application is: after the near end collects the time domain reception signal, the time domain reference signal sent by the far end in the time domain reception signal is eliminated, so as to avoid the far end user still being able to hear the echo after the far end receives the voice signal sent by the near end. That is to say, for the near end, the time domain reference signal sent by the far end is an echo signal.

[0022] S120: Convert the time domain received signal and the time domain reference signal from the time domain to the frequency domain respectively to obtain a frequency domain received signal and a frequency domain reference signal.

[0023] Specifically, the time domain received signal and the time domain reference signal are respectively time-to-frequency converted to obtain a frequency domain received signal and a frequency domain reference signal.

[0024] In one application scenario, Fourier transform is performed on the time domain received signal and the time domain reference signal respectively to obtain the frequency domain received signal and the frequency domain reference signal.

[0025] In other application scenarios, other transformations such as Laplace transform may be used to transform the time domain received signal and the time domain reference signal from the time domain to the frequency domain.

[0026] When converting the signal from the time domain to the frequency domain, the same transformation can be performed on the time domain received signal and the time domain reference signal, for example, Fourier transform is performed on both the time domain received signal and the time domain reference signal, or different transformations can be performed, for example, Fourier transform is performed on the time domain received signal and Laplace transform is performed on the time domain reference signal.

[0027] In summary, the present application does not limit the process of how to convert the time domain received signal and the time domain reference signal from the time domain to the frequency domain.

[0028] S130: Based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal, obtain a frequency domain mask signal including a real mask and an imaginary mask.

[0029] Specifically, the frequency domain received signal and the frequency domain reference signal are analyzed to determine whether the frequency domain received signal contains an echo signal (ie, the frequency domain reference signal) at each frequency point, thereby obtaining a frequency domain mask signal. The frequency domain mask signal M can be expressed by the following formula:

[0030] M=M r +jM i , where M r is the real part mask, M i is the imaginary mask.

[0031] In one application scenario, the step of obtaining the frequency domain mask signal in step S130 specifically includes: obtaining the frequency domain mask signal based on at least one of the real part and the imaginary part of the frequency domain received signal and at least one of the real part and the imaginary part of the frequency domain reference signal.

[0032] Specifically, the frequency domain received signal D is expressed by the following formula:

[0033] D=D r +jD i , where D r is the real part of the received signal D in the frequency domain, D i is the imaginary part of the received signal D in the frequency domain.

[0034] The frequency domain reference signal X is expressed as follows:

[0035] X=X r +jX i , where X r is the real part of the frequency domain reference signal X, X i is the imaginary part of the frequency domain reference signal X.

[0036] Among them, based on D r , D i At least one of r , X i At least one of D r , D i , and X r , X i , get the frequency domain mask signal, or, based on D r and X r , X i , and obtain the frequency domain mask signal.

[0037] S140: Based on the frequency domain mask signal, acoustically mask the frequency domain received signal to obtain a frequency domain target signal.

[0038] Specifically, the frequency domain received signal is acoustically masked by using the frequency domain mask signal, and the frequency domain reference signal in the frequency domain received signal is removed, thereby obtaining the frequency domain target signal.

[0039] The process of acoustically masking the frequency domain received signal based on the frequency domain mask signal can be expressed by the following formula:

[0040] S=M×D=(M r +jM i )×(D r +jD i ), where S is the target signal in the frequency domain.

[0041] It can be seen from the above content that the frequency domain mask signal can be regarded as a mask filter to filter out the frequency domain reference signal in the frequency domain received signal.

[0042] S150: Convert the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal.

[0043] Specifically, the frequency domain target signal is converted from the frequency domain to the time domain, so that the time domain target signal can be obtained. The time domain target signal is the speech signal after the echo cancellation is performed on the time domain received signal.

[0044] In one application scenario, an inverse Fourier transform is performed on a frequency domain target signal to obtain a time domain target signal; in another application scenario, an inverse Laplace transform is performed on a frequency domain target signal to obtain a time domain target signal.

[0045] In the prior art, when performing echo cancellation, only the amplitude information of the signal is considered, and the phase spectrum of the frequency domain target signal is directly set to be the same as the phase spectrum of the frequency domain received signal. This method ignores the advantage of phase information estimation for echo cancellation and cannot completely suppress the echo.

[0046] In this embodiment, a frequency domain mask signal is first obtained based on the frequency domain received signal and the frequency domain reference signal, and then the frequency domain received signal is acoustically masked using the frequency domain mask signal. The frequency domain mask signal includes a real mask and an imaginary mask. Therefore, in the process of obtaining the frequency domain target signal, the phase information and amplitude information of the frequency domain received signal are simultaneously considered. Compared with the prior art which only considers the amplitude information of the signal, this embodiment can perform echo cancellation well even in the case of low signal-to-echo ratio and strong nonlinear distortion of the intercom equipment, thereby ensuring the effect of echo cancellation.

[0047] At the same time, in order to reduce the convergence time of the echo cancellation and reduce the steady-state error of the echo cancellation, this implementation method uses a pre-trained neural network to obtain a frequency domain mask signal.

[0048] Specifically, in one application scenario, step S130 includes: sending at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis to obtain a frequency domain mask signal.

[0049] Specifically, the neural network in this embodiment is a recurrent neural network, which is pre-trained to reach convergence. After receiving at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal, it can analyze whether the frequency domain received signal contains an echo signal (i.e., the frequency domain reference signal) at each frequency point, thereby obtaining a frequency domain mask signal.

[0050] Among them, the neural network can be RNN, GRU, LSTM, etc., and the number of layers and output dimensions of the neural network can be set according to the complexity of different application scenarios, and are not limited here.

[0051] In another application scenario, step S130 includes: sending at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis to obtain a real mask and an imaginary mask; based on the real mask and the imaginary mask, constructing a frequency domain mask signal.

[0052] Specifically, different from the above application scenarios, the pre-trained neural network outputs a real mask and an imaginary mask after receiving the characteristics of the frequency domain received signal and the frequency domain reference signal, and finally constructs a frequency domain mask signal based on the real mask and the imaginary mask.

[0053] Among them, at least one of the real part and the imaginary part of the frequency domain received signal and at least one of the real part and the imaginary part of the frequency domain reference signal can be input into a pre-trained neural network.

[0054] For example, the real part and imaginary part of the frequency domain received signal and the real part and imaginary part of the frequency domain reference signal are input into a pre-trained neural network, and then the neural network outputs a real mask and an imaginary mask, and finally a frequency domain mask signal is constructed based on the real mask and the imaginary mask.

[0055] Combination Figure 2 In this embodiment, the pre-trained neural network includes a first sub-neural network 10 and a second sub-neural network 20 connected in parallel. At this time, step S130 specifically includes: sending at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal to the first sub-neural network 10 and the second sub-neural network 20 for analysis to obtain a real mask and an imaginary mask respectively.

[0056] Specifically, at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal are input into the first sub-neural network 10 to obtain a real mask; at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal are input into the second sub-neural network 20 to obtain an imaginary mask.

[0057] For example, combined with Figure 2 , the real part D of the received signal D in the frequency domain r , the imaginary part D of the received signal D in the frequency domain i , the real part of the frequency domain reference signal X r and the imaginary part X of the frequency domain reference signal X i Composition feature sequence [X r X i D r D i] are respectively input into the first sub-neural network 10 and the second sub-neural network 20 to obtain the real mask M r , imaginary mask M i .

[0058] That is to say, the first sub-neural network 10 and the second sub-neural network 20 receive the same information but have different outputs.

[0059] The structures of the first sub-neural network 10 and the second sub-neural network 20 may be the same or different, and are not limited here. For example, in an application scenario, the first sub-neural network 10 that outputs a real mask includes fewer network layers and a lower hidden layer dimension than the second sub-neural network 20 that outputs an imaginary mask.

[0060] See also Figure 3 and Figure 4 The following describes the training method of the neural network in this embodiment. The training method of the neural network includes:

[0061] S201: Acquire a sample time domain received signal, a sample time domain reference signal, and a sample time domain real signal.

[0062] Specifically, the sample time-domain real signal is a speech signal that does not contain an echo signal.

[0063] S202: Convert the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain respectively to obtain a sample frequency domain received signal and a sample frequency domain reference signal.

[0064] Among them, step S202 is similar to step S120, and details can be found in the relevant content, which will not be repeated here.

[0065] S203: Send at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal to a neural network for analysis to predict a sample real mask and a sample imaginary mask, and construct a sample frequency domain mask signal based on the sample real mask and the sample imaginary mask.

[0066] Specifically, after constructing the basic framework of the neural network, at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal are sent to the neural network. Based on at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal, the neural network predicts whether the sample frequency domain received signal contains an echo signal (i.e., the sample frequency domain reference signal) at each frequency point, thereby outputting a sample real mask and a sample imaginary mask. Finally, based on the predicted sample real mask and sample imaginary mask, a sample frequency domain mask signal is constructed.

[0067] Among them, the neural network can use RNN, GRU, LSTM and other networks, and the number of layers and output dimensions of the neural network can be set according to the complexity of different application scenarios, and are not limited here.

[0068] S204: Based on the sample frequency domain mask signal, acoustically mask the sample frequency domain received signal to obtain a sample frequency domain target signal.

[0069] Specifically, step S204 is similar to the above step S140, and details can be found in the relevant content, which will not be repeated here.

[0070] S205: Convert the sample frequency domain target signal from the frequency domain to the time domain to obtain a sample time domain target signal.

[0071] Specifically, step S205 is similar to step S150, and details can be found in the relevant content, which will not be repeated here.

[0072] S206: Obtain a loss function value based on the sample time domain target signal and the sample time domain true signal.

[0073] Specifically, the mean absolute error (L1Loss), mean square error (L2Loss), etc. can be selected as the loss function, and then the sample time domain target signal and the sample time domain true signal are substituted into the loss function to obtain the loss function value.

[0074] S207: Based on the loss function value, update the parameters of the neural network.

[0075] Specifically, based on the loss function value, the parameters of the neural network are updated so that the neural network gradually converges.

[0076] S208: Determine whether the preset training stop condition is met.

[0077] If the judgment result is yes, the process ends; if the judgment result is no, the process returns to step S202.

[0078] The preset conditions for stopping training include: the number of training times for the neural network reaches a threshold number, or the loss function values ​​calculated twice are equal, or the loss function value is lower than a threshold.

[0079] Through the training method of this embodiment, it can be ensured that the real mask and the imaginary mask predicted by the trained neural network retain physical continuity, avoiding the problem of mismatch between the real mask and the imaginary mask.

[0080] In an application scenario, such as Figure 2 As shown, when the neural network includes a first sub-neural network 10 and a second sub-neural network 20 connected in parallel, step S203 predicts the sample real part mask and the sample imaginary part mask, including:

[0081] At least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal are sent to the first sub-neural network 10 and the second sub-neural network 20 for analysis to predict the sample real mask and the sample imaginary mask respectively.

[0082] Specifically, after receiving at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal, the first sub-neural network 10 predicts the sample real part mask; after receiving at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal, the second sub-neural network 20 predicts the sample imaginary part mask.

[0083] At this time, step S207 of updating the parameters of the neural network includes: based on the loss function value, simultaneously updating the parameters of the first sub-neural network 10 and the second sub-neural network 20.

[0084] Specifically, at this time, based on the loss function value, the parameters of the first sub-neural network 10 and the second sub-neural network 20 are updated simultaneously, which can ensure that the real mask predicted by the trained first sub-neural network 10 and the imaginary mask predicted by the second sub-neural network 20 retain physical continuity and avoid the problem of mismatch between the real mask and the imaginary mask.

[0085] See also Figure 5 , Figure 5 1 is a schematic diagram of the structure of an embodiment of the echo cancellation device of the present application. The echo cancellation device 200 includes a processor 210, a memory 220 and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230 respectively. The memory 220 stores program data. The processor 210 implements the steps in any of the above-mentioned implementation methods by executing the program data in the memory 220. The detailed steps can be found in the above-mentioned implementations and will not be repeated here.

[0086] The echo cancellation device 200 may be any device with voice signal processing capability, such as a computer or a mobile phone, and is not limited here.

[0087] See also Figure 6 , Figure 6 The echo cancellation device 300 includes a signal acquisition module 310 , a first conversion module 320 , a mask acquisition module 330 , an acoustic masking module 340 and a second conversion module 350 .

[0088] The signal acquisition module 310 is used to acquire a time domain received signal collected by the near end and a time domain reference signal sent by the far end.

[0089] The first conversion module 320 is connected to the signal acquisition module 310, and is used to convert the time domain received signal and the time domain reference signal from the time domain to the frequency domain to obtain the frequency domain received signal and the frequency domain reference signal.

[0090] The mask acquisition module 330 is connected to the first conversion module 320, and is used to obtain a frequency domain mask signal including a real mask and an imaginary mask based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal.

[0091] The acoustic masking module 340 is connected to the mask acquisition module 330 and is used to perform acoustic masking on the frequency domain received signal based on the frequency domain mask signal to obtain the frequency domain target signal.

[0092] The second conversion module 350 is connected to the acoustic masking module 340 and is used to convert the frequency domain target signal from the frequency domain to the time domain to obtain the time domain target signal.

[0093] In one implementation, the mask acquisition module 330 is specifically configured to obtain a frequency domain mask signal based on at least one of the real part and the imaginary part of the frequency domain received signal and at least one of the real part and the imaginary part of the frequency domain reference signal.

[0094] In one embodiment, the mask acquisition module 330 is specifically used to send at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis to obtain a frequency domain mask signal.

[0095] In one embodiment, the mask acquisition module 330 is specifically used to send at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis to obtain a real mask and an imaginary mask; based on the real mask and the imaginary mask, a frequency domain mask signal is constructed.

[0096] In one embodiment, the neural network includes a first sub-neural network and a second sub-neural network connected in parallel, and the mask acquisition module 330 is specifically used to send at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal to the first sub-neural network and the second sub-neural network for analysis to obtain a real mask and an imaginary mask, respectively.

[0097] In one embodiment, the echo cancellation device 300 further includes a network training module, which is used to obtain a sample time domain received signal, a sample time domain reference signal, and a sample time domain real signal before the signal acquisition module 310 obtains the time domain received signal collected by the near end and the time domain reference signal sent by the far end; respectively convert the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain to obtain a sample frequency domain received signal and a sample frequency domain reference signal; send at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal to a neural network for analysis to predict a sample real mask and a sample imaginary mask, and based on the sample real mask and the sample Imaginary part mask, construct a sample frequency domain mask signal; based on the sample frequency domain mask signal, acoustically mask the sample frequency domain received signal to obtain a sample frequency domain target signal; convert the sample frequency domain target signal from the frequency domain to the time domain to obtain a sample time domain target signal; based on the sample time domain target signal and the sample time domain true signal, obtain a loss function value; based on the loss function value, update the parameters of the neural network; repeatedly execute the steps of converting the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain to obtain the sample frequency domain received signal and the sample frequency domain reference signal to the step of updating the parameters of the neural network based on the loss function value, until the preset conditions for stopping training are met.

[0098] In one embodiment, the neural network includes a first sub-neural network and a second sub-neural network connected in parallel; the network training module is specifically used to send at least part of the characteristics of the sample frequency domain received signal and at least part of the characteristics of the sample frequency domain reference signal to the first sub-neural network and the second sub-neural network for analysis, and predict the sample real mask and the sample imaginary mask respectively; and, based on the loss function value, simultaneously update the parameters of the first sub-neural network and the second sub-neural network.

[0099] In one embodiment, the first conversion module 320 is specifically used to perform Fourier transform on the time domain received signal and the time domain reference signal, respectively, to obtain the frequency domain received signal and the frequency domain reference signal; and / or, the second conversion module 350 is specifically used to perform inverse Fourier transform on the frequency domain target signal to obtain the time domain target signal.

[0100] The echo cancellation device 300 may be any device with voice signal processing capability, such as a computer or a mobile phone, and is not limited here.

[0101] See also Figure 7 , Figure 7 The computer-readable storage medium 400 stores a computer program 410, which can be executed by a processor to implement the steps in any of the above methods.

[0102] Among them, the computer-readable storage medium 400 can specifically be a device that can store the computer program 410, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the computer program 410. The server can send the stored computer program 410 to other devices for execution, or it can also run the stored computer program 410 by itself.

[0103] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An echo cancellation method, characterized in that: The method comprises: Acquire a time domain received signal collected at a near end and a time domain reference signal sent at a far end; Respectively converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain to obtain a frequency domain received signal and a frequency domain reference signal; Based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal, a frequency domain mask signal including a real mask and an imaginary mask is obtained; based on the frequency domain mask signal, the frequency domain received signal is acoustically masked to obtain a frequency domain target signal; Converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal; The step of obtaining a frequency domain mask signal including a real mask and an imaginary mask based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal comprises: Sending at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis, outputting the real mask and the imaginary mask, and constructing the frequency domain mask signal based on the real mask and the imaginary mask; The neural network includes a first sub-neural network and a second sub-neural network connected in parallel; Before acquiring the time domain received signal collected by the near end and the time domain reference signal sent by the far end, the method further includes: Obtaining a sample time domain received signal, a sample time domain reference signal, and a sample time domain real signal; Respectively converting the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain to obtain a sample frequency domain received signal and a sample frequency domain reference signal; Sending at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal to the first sub-neural network and the second sub-neural network for analysis, and predicting the sample real mask and the sample imaginary mask respectively; The first sub-neural network that outputs the real part mask includes fewer network layers and a lower hidden layer dimension than the second sub-neural network that outputs the imaginary part mask.

2. The method according to claim 1, characterized in that: The step of obtaining a frequency domain mask signal including a real mask and an imaginary mask based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal comprises: The frequency domain mask signal is obtained based on at least one of the real part and the imaginary part of the frequency domain received signal and at least one of the real part and the imaginary part of the frequency domain reference signal.

3. The method according to claim 1, characterized in that The step of obtaining a frequency domain mask signal including a real mask and an imaginary mask based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal comprises: At least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal are sent to a pre-trained neural network for analysis to obtain the frequency domain mask signal.

4. The method according to claim 1, characterized in that: The neural network includes a first sub-neural network and a second sub-neural network connected in parallel; The step of sending at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis to obtain the real mask and the imaginary mask comprises: At least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal are sent to the first sub-neural network and the second sub-neural network for analysis to obtain the real mask and the imaginary mask respectively.

5. The method according to claim 1, characterized in that Before acquiring the time domain received signal collected by the near end and the time domain reference signal sent by the far end, the method further includes: Constructing a sample frequency domain mask signal based on the sample real mask and the sample imaginary mask; Based on the sample frequency domain mask signal, acoustically masking the sample frequency domain received signal to obtain a sample frequency domain target signal; Converting the sample frequency domain target signal from the frequency domain to the time domain to obtain a sample time domain target signal; Obtaining a loss function value based on the sample time domain target signal and the sample time domain true signal; Based on the loss function value, updating the parameters of the neural network; Repeat the steps of converting the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain to obtain a sample frequency domain received signal and a sample frequency domain reference signal, and updating the parameters of the neural network based on the loss function value, until a preset condition for stopping training is met.

6. The method according to claim 1, characterized in that The step of converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain to obtain a frequency domain received signal and a frequency domain reference signal comprises: Performing Fourier transform on the time domain received signal and the time domain reference signal respectively to obtain the frequency domain received signal and the frequency domain reference signal; And / or, the step of converting the frequency domain target signal from the frequency domain to the time domain to obtain the time domain target signal comprises: Performing an inverse Fourier transform on the frequency domain target signal to obtain the time domain target signal.

7. An echo cancellation device, characterized in that: The echo cancellation device includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit respectively. The memory stores program data. The processor executes the program data in the memory to implement: Acquire a time domain received signal collected at a near end and a time domain reference signal sent at a far end; Respectively converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain to obtain a frequency domain received signal and a frequency domain reference signal; Based on at least part of the characteristics of the frequency domain received signal and at least part of the characteristics of the frequency domain reference signal, obtaining a frequency domain mask signal including a real mask and an imaginary mask; Based on the frequency domain mask signal, acoustically masking the frequency domain received signal to obtain a frequency domain target signal; Converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal; The processor further implements, by executing the program data in the memory: Sending at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis, outputting the real mask and the imaginary mask, and constructing the frequency domain mask signal based on the real mask and the imaginary mask; The neural network includes a first sub-neural network and a second sub-neural network connected in parallel; Before acquiring the time domain received signal collected by the near end and the time domain reference signal sent by the far end, the method further includes: Obtaining a sample time domain received signal, a sample time domain reference signal, and a sample time domain real signal; Respectively converting the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain to obtain a sample frequency domain received signal and a sample frequency domain reference signal; Sending at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal to the first sub-neural network and the second sub-neural network for analysis, and predicting the sample real mask and the sample imaginary mask respectively; The first sub-neural network outputting the real part mask includes fewer network layers and lower hidden layer dimensions.

8. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, which can be executed by a processor to implement: Acquire a time domain received signal collected at a near end and a time domain reference signal sent at a far end; Respectively converting the time domain received signal and the time domain reference signal from the time domain to the frequency domain to obtain a frequency domain received signal and a frequency domain reference signal; Based on at least some characteristics of the frequency domain received signal and at least some characteristics of the frequency domain reference signal, obtaining a frequency domain mask signal including a real mask and an imaginary mask; Based on the frequency domain mask signal, acoustically masking the frequency domain received signal to obtain a frequency domain target signal; Converting the frequency domain target signal from the frequency domain to the time domain to obtain a time domain target signal; The computer program can be executed by a processor to further implement: Sending at least part of the features of the frequency domain received signal and at least part of the features of the frequency domain reference signal to a pre-trained neural network for analysis, outputting the real mask and the imaginary mask, and constructing the frequency domain mask signal based on the real mask and the imaginary mask; The neural network includes a first sub-neural network and a second sub-neural network connected in parallel; Before acquiring the time domain received signal collected by the near end and the time domain reference signal sent by the far end, the method further includes: Obtaining a sample time domain received signal, a sample time domain reference signal, and a sample time domain real signal; Respectively converting the sample time domain received signal and the sample time domain reference signal from the time domain to the frequency domain to obtain a sample frequency domain received signal and a sample frequency domain reference signal; Sending at least part of the features of the sample frequency domain received signal and at least part of the features of the sample frequency domain reference signal to the first sub-neural network and the second sub-neural network for analysis, and predicting the sample real mask and the sample imaginary mask respectively; The first sub-neural network outputting the real part mask includes fewer network layers and lower hidden layer dimensions.

Images