A method and device for processing acoustic echo

Abstract

The present application provides an acoustic echo processing method and device, the method comprising: obtaining a first correlation coefficient between a first residual signal and a first echo signal, the first residual signal and the first echo signal corresponding to a first speech signal; determining a second correlation coefficient based on the first correlation coefficient and an attenuation coefficient, the second correlation coefficient being less than or equal to the first correlation coefficient; performing residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal. In some scenarios, the over-estimation problem existing in the acoustic echo processing technology is improved to improve the acoustic echo processing effect.

Description

Technical Field

[0001] The present application relates to the field of acoustic echo, and in particular to an acoustic echo processing method and device. Background Art

[0002] In the process of voice communication, acoustic echo is an important factor that affects the call quality and user experience. The reason for acoustic echo is that the voice signal of the far-end caller in voice communication is played by the speaker of the terminal device used by the near-end caller, and then picked up by the microphone of the terminal device and transmitted to the far end, so that the far-end caller can always hear his own voice when speaking. Especially in the case of hands-free calls, the echo with too much energy will interfere with the semantic understanding of the far-end caller and seriously affect the call experience.

[0003] In the acoustic echo cancellation technology, it generally includes adaptive linear filtering and residual echo suppression steps. The adaptive linear filtering step is used to filter out most of the linear echoes to obtain a residual signal. The linear echo is most of the echo signals in the voice signal picked up by the near-end microphone that are linearly related to the voice signal of the far-end caller. The residual echo suppression step is used to estimate the residual echo component in the residual signal. Specifically, the residual echo in the residual signal is estimated by calculating the correlation coefficient between the residual signal and the linear echo.

[0004] However, in some scenarios, such as noisy scenarios, there is an over-estimation problem when using the above-mentioned acoustic echo cancellation technology to perform echo processing on the voice signal picked up by the near-end microphone, and the voice signal output by the echo processing is distorted. The acoustic echo processing effect needs to be improved. Summary of the invention

[0005] The present application provides an acoustic echo processing method and device thereof, which can improve the over-estimation problem existing in the acoustic echo processing technology and the distortion problem of the voice signal output by the echo processing in some scenarios, and improve the acoustic echo processing effect.

[0006] In a first aspect, the present application provides an acoustic echo processing method, the method comprising: obtaining a first correlation coefficient between a first residual signal and a first echo signal, the first residual signal and the first echo signal corresponding to a first speech signal; determining a second correlation coefficient based on the first correlation coefficient and an attenuation coefficient, the second correlation coefficient being less than or equal to the first correlation coefficient; performing residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal.

[0007] Exemplarily, the first correlation coefficient may be determined based on the product, quotient, difference, or power of the first correlation coefficient and the attenuation coefficient.

[0008] Exemplarily, the designers found that in a noisy scene or a double-talk scene, after the first voice signal was echo processed based on the adaptive linear filtering and residual echo suppression steps, the voice signal of the near-end caller appeared in the obtained voice signal and was distorted. Then, the acoustic echo processing method provided in the present application was used to attenuate the first correlation coefficient, that is, to determine a second correlation coefficient smaller than the first correlation coefficient based on the first correlation coefficient and the attenuation coefficient, and then to echo process the first residual echo signal based on the second correlation coefficient to obtain the above-mentioned second voice signal, so as to improve the problem of over-estimation of the first correlation coefficient that causes distortion of the voice signal of the near-end caller contained in the second voice signal, protect the voice signal of the near-end caller, and improve the acoustic echo processing effect.

[0009] Exemplarily, the designers found that in a non-noise scenario or a non-double-talk scenario, after the first voice signal is echo processed based on the adaptive linear filtering and residual echo suppression steps, the voice signal of the near-end talker in the obtained voice signal does not show distortion. In this case, the acoustic echo processing method provided in the present application can be used without attenuating the first correlation coefficient, that is, a second correlation coefficient equal to the first correlation coefficient is determined based on the first correlation coefficient and the attenuation coefficient, and then the first residual echo signal is echo processed based on the second correlation coefficient to obtain the above-mentioned second voice signal; or, the first residual echo signal is directly echo processed based on the first correlation coefficient to obtain the above-mentioned second voice signal, so as to minimize the echo signal in the second voice signal while ensuring that the second voice signal is not distorted.

[0010] In summary, by adopting the acoustic echo processing method provided in the present application, when it is necessary to attenuate the first correlation coefficient, based on the attenuation coefficient, a second correlation coefficient smaller than the first correlation coefficient can be obtained, and residual echo processing can be performed on the first residual signal based on the second correlation coefficient, so as to improve the problem of distortion of the near-end caller's voice signal contained in the second voice signal caused by the over-estimation problem of the first correlation coefficient being too large; in addition, when it is not necessary to attenuate the first correlation coefficient, based on the attenuation coefficient, a second correlation coefficient equal to the first correlation coefficient can be obtained, and residual echo processing can be performed on the first residual signal based on the second correlation coefficient, so as to reduce the echo signal in the second voice signal as much as possible while ensuring that the second voice signal is not distorted, thereby improving the acoustic echo processing effect.

[0011] In a possible implementation, determining the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient includes: determining the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient, and the attenuation coefficient is greater than 0 and less than or equal to 1.

[0012] In a possible implementation manner, when the speech scene corresponding to the first speech signal belongs to the first scene, the attenuation coefficient is greater than 0 and less than 1.

[0013] It can be understood that if the method for obtaining the second correlation coefficient is: determining the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient, then the attenuation coefficient is greater than 0 and less than 1, which is used to indicate the attenuation of the first correlation coefficient to obtain a second correlation coefficient less than the first correlation coefficient.

[0014] In a possible implementation manner, the first scene is a noise scene or a double-talk scene.

[0015] In a possible implementation, when a first value is less than a first threshold, the speech scene corresponding to the first speech signal belongs to the first scene, and the first value is determined based on energy of the first echo signal, energy of the first residual signal, and a first constant.

[0016] In one possible implementation, when the second numerical value corresponding to the first voice signal is less than a second threshold, the voice scene corresponding to the first voice signal belongs to the first scene, the second numerical value corresponding to the first voice signal is determined based on the energy of the first echo signal, the energy of the first residual signal, the first constant, the first voice signal, the second numerical value corresponding to the second voice signal, and the smoothing coefficient, and the second voice signal and the first voice signal belong to two adjacent frames of voice signals in the same segment of voice signals.

[0017] In a possible implementation, the second value corresponding to the first speech signal satisfies the following formula: β smooth (n) = β(n)*α+β smooth (n-1)*(1-α); wherein, the β smooth (n) is a second value determined according to the parameters of the first speech signal, and the β smooth (n-1) is a second value determined according to the parameters of the second speech signal, the first speech signal and the second speech signal belong to the nth frame signal and the n-1th frame signal in the same segment of speech signal respectively, the n is greater than or equal to 2, the β(n) is a first value, the first value is determined based on the energy of the first echo signal, the energy of the first residual signal, and a first constant, and the α is the smoothing coefficient.

[0018] It can be understood that if the first terminal obtains β after smoothing β(n) based on formula 11 smooth (n) Determine whether the first speech signal is in a large near-end interference scenario, because the β smooth(n) contains the scene features of the n frames of signals before the first speech signal, rather than only the scene features of the current frame signal, i.e., the first speech signal, so that based on the β smooth (n) The judgment result obtained by judging whether the first speech signal is in a large near-end interference scenario has high stability and accuracy, and can reduce the interference degree of unknown unstable factors on the judgment result.

[0019] In a possible implementation, when the value of n is 2, the β smooth (n-1) is taken as β(n-1), where β(n-1) is the product of the ratio of the energy of the second echo signal to the energy of the second residual signal and the first constant, and the second echo signal and the second residual signal correspond to the second speech signal.

[0020] In a possible implementation, when the amplitude of the target noise signal is greater than a third threshold, the speech scene corresponding to the first speech signal belongs to the first scene, and the target noise signal is used to indicate the noise intensity of the first speech signal.

[0021] In a possible implementation, the target noise signal is a noise signal obtained by performing noise estimation on the first speech signal; or, the target noise signal is a noise signal obtained by performing noise estimation on the second speech signal, and the second speech signal and the first speech signal belong to two adjacent frames of speech signals in the same segment of speech signals; or, the target noise signal is a noise signal obtained by performing noise estimation on the first residual signal; or, the target noise signal is a noise signal obtained by performing noise estimation on the second residual signal, the second residual signal corresponds to the second speech signal, and the second speech signal and the first speech signal belong to two adjacent frames of speech signals in the same segment of speech signals.

[0022] It is understandable that the first echo signal will contain some noise signals, which will reduce the noise signal contained in the first residual signal compared to the noise signal contained in the first voice signal, and the proportion of the noise signal in the first echo signal obtained by adaptive linear filtering of two different frame signals to the total noise signal will also be erratic. In other words, the complete first voice signal contains more complete noise scene features than the first residual signal, and the use of the noise estimation of the complete first voice signal to determine whether the scene where the first voice signal is located is in a large near-end interference scene is more representative, and the judgment result obtained is more accurate.

[0023] It is understandable that the first echo signal generally contains less noise signal, and most of the noise signal will remain in the first residual signal. The noise estimation of the first residual signal is also sufficient to represent the noise environment in which the first speech signal is located. Moreover, compared with the noise estimation of the first echo signal, the computational complexity of the noise estimation of the first residual signal is smaller, especially when the number of frame signals is large, which can save performance loss to a certain extent.

[0024] In a possible implementation, when the voice scene corresponding to the first voice signal belongs to the second scene, the attenuation coefficient takes a value of 1; determining the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient includes: determining the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient.

[0025] In a possible implementation manner, the first scene is a noise scene or a double-talk scene.

[0026] In a possible implementation, when the second value corresponding to the first speech signal is less than the second threshold value and the amplitude of the target noise signal is greater than the third threshold value, the speech scene corresponding to the first speech signal belongs to the first scene, and the target noise signal is used to indicate the noise intensity of the first speech signal; wherein the second value satisfies the formula β smooth (n) = β(n)*α+β smooth (n-1)*(1-α); β(n) is a first value, which is determined based on the energy of the first echo signal, the energy of the first residual signal, and a first constant, and β smooth (n) is used to represent the second value corresponding to the first speech signal, the β smooth (n-1) is used to represent a second value corresponding to a second speech signal, the first speech signal and the second speech signal belong to the nth frame signal and the n-1th frame signal in the same speech signal, respectively, n is greater than or equal to 2, α is the smoothing coefficient; and when n is 2, β smooth (n-1) is taken as β(n-1), where β(n-1) is the product of the ratio of the energy of the second echo signal to the energy of the second residual signal and the first constant.

[0027] In a possible implementation, each frame signal in the first speech signal includes M frequency points, where M is greater than or equal to 1, and the target noise signal includes M noise complex spectra corresponding one-to-one to the M frequency points, and the amplitude of the target noise signal is the sum of M modules corresponding to the M noise complex spectra.

[0028] In a possible implementation, performing residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal includes: obtaining a residual echo signal based on the product of the frequency spectrum of the first residual signal and the second correlation coefficient; performing noise estimation on the first residual signal to obtain a first noise signal; and removing the residual echo signal and the first noise signal in the first residual signal by frequency subtraction to obtain the second speech signal.

[0029] In a possible implementation manner, the first correlation coefficient satisfies the following formula: Wherein, the η(n) is used to represent the first correlation coefficient, the P ey (n) and the P yy (n) is related to the first speech signal, the P ey (n-1) and the P yy (n-1) is related to the second voice signal, the first voice signal and the second voice signal belong to the nth frame signal and the n-1th frame signal in the same voice signal, respectively, and n is greater than or equal to 2; the first voice signal and the second voice signal both include M frequency points, and M is greater than or equal to 1, and k is used to represent the kth frequency point among the M frequency points, and k is a positive integer greater than 1 and less than M, and R ey (n, k) is the cross power spectrum of the kth echo signal and the kth residual signal, the kth echo signal is the complex spectrum corresponding to the kth frequency point in the first echo signal, the kth residual signal is the complex spectrum corresponding to the kth frequency point in the first residual signal, is the sum of the cross-power spectra of the M frequency points of the first speech signal; β(n) is the product of the ratio of the energy of the first echo signal to the energy of the first residual signal and a first constant, the first constant being greater than 0 and less than 0.1; R yy (n, k) is the autopower spectrum of the kth echo signal, The sum of the M auto-power spectra corresponding to the M frequency points of the first speech signal is used to represent the first speech signal; wherein, when n is equal to 2, the P ey (n-1) is The P yy (n-1) is Said is the sum of the cross-power spectra of the residual signal and the echo signal corresponding to the M frequency points of the second speech signal, is the sum of the autopower spectra of the echo signal corresponding to the M frequency points of the second speech signal.

[0030] In a possible implementation, the method is applied to a scenario in which a first terminal and a second terminal are having a call. Before obtaining a first correlation coefficient between a first residual signal and a first echo signal, the method further includes: in the context of the call, obtaining a first reference signal and a first voice signal, wherein the first reference signal is a voice call signal received by the first terminal from the second terminal, and the first voice signal is a voice call signal picked up by a microphone of the first terminal after the first reference signal is played by the first terminal. Without considering a delay, the moment when the first terminal receives the first reference signal, the moment when the first terminal plays the first reference signal, and the moment when the microphone of the first terminal picks up the first voice signal belong to the same moment; based on the echo cancellation technology of the adaptive linear filtering, the first reference signal, and the first voice signal, obtaining the first echo signal and the first residual signal, wherein the first echo signal is a linear echo signal obtained from the first voice signal based on the echo cancellation technology of the adaptive linear filtering, and the first residual signal is other signals in the first voice signal except the first echo signal.

[0031] In a second aspect, an embodiment of the present application provides an acoustic echo processing device, which includes a unit for implementing the method shown in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0032] In a third aspect, an embodiment of the present application provides an acoustic echo processing device, which includes a processor and a memory, the processor and the memory are connected to each other, the memory is used to store instructions, and the processor is used to call the instructions, so that the method shown in the first aspect or any possible implementation of the first aspect is implemented.

[0033] In a fourth aspect, an embodiment of the present application provides a chip system, which includes one or more processors, and the processors are used to call computer instructions to implement the method shown in the first aspect or any possible implementation of the first aspect.

[0034] In the fifth aspect, an embodiment of the present application provides a module device, which includes a communication module, a power module, a storage module and a chip module, wherein: the power module is used to provide power to the module device; the storage module is used to store data and instructions; the communication module is used for internal communication of the module device, or for the module device to communicate with external devices; the chip module is used to execute the method shown in the first aspect or any possible implementation of the first aspect.

[0035] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the method shown in the first aspect or any possible implementation of the first aspect is implemented.

[0036] In a seventh aspect, an embodiment of the present application provides a computer program. When the computer program is executed, the method shown in the first aspect or any possible implementation of the first aspect is implemented.

[0037] It is understandable that the acoustic echo processing device, chip, module device, computer-readable storage medium, and computer program provided in the second aspect are used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of an acoustic echo processing method provided in an embodiment of the present application;

[0039] Figure 2 is a flow chart of another acoustic echo processing method provided in an embodiment of the present application;

[0040] Figure 3 is a flow chart of another acoustic echo processing method provided in an embodiment of the present application;

[0041] Figure 4 is a flow chart of another acoustic echo processing method provided in an embodiment of the present application;

[0042] Figure 5 It is a schematic diagram of functional modules for executing an acoustic echo processing method in a first terminal provided in an embodiment of the present application and signal flows between the modules;

[0043] Figure 6 is a schematic diagram of an acoustic echo processing device provided in an embodiment of the present application;

[0044] Figure 7 is a schematic diagram of another acoustic echo processing provided by an embodiment of the present application;

[0045] Figure 8 It is a schematic diagram of another acoustic echo processing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described below in conjunction with the accompanying drawings.

[0047] The terms "first" and "second" in the specification, claims and drawings of this application are only used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0048] The "embodiment" mentioned in this article means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It can be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0049] In the present application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three and more than three, and "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0050] like Figure 1 The flowchart shown is an acoustic echo processing method provided by the present application. Figure 1 As shown, the method includes:

[0051] S101: A first terminal obtains a first correlation coefficient between a first residual signal and a first echo signal.

[0052] In the embodiment of the present application, the first residual signal and the first echo signal correspond to the first speech signal.

[0053] Exemplarily, the first voice signal is a frame signal acquired by the first terminal based on a sound pickup device (such as a microphone) in a scenario where the first terminal and the second terminal are talking. The call scenario can be a call scenario of a phone application, or a voice call scenario or an audio and video call scenario of other social applications, for example, a voice call scenario or an audio and video call scenario in a WeChat application.

[0054] In the embodiment of the present application, it is assumed that the frame signal obtained by the first terminal based on the sound pickup device (such as a microphone) is recorded as a microphone signal, and the first voice signal is the nth frame signal in the microphone signal, where n is a positive integer.

[0055] In an embodiment of the present application, the first echo signal may include part or all of the linear echo signal in the first voice signal, and the first residual signal is other signals in the first voice signal except the first echo signal. Exemplarily, the first residual signal may include a linear echo component and / or a nonlinear echo component in the first voice signal, and the first residual signal also includes a near-end signal, which includes a voice signal of a near-end talker and / or a noise signal of the environment in which the first terminal is located.

[0056] The above-mentioned first echo signal is a linear echo signal obtained from the first voice signal by the echo cancellation method based on adaptive linear filtering of the first terminal and the first reference signal. The linear echo signal is part or all of the signal in the first voice signal that is linearly related to the reference signal. The first reference signal is a voice call signal sent by the second terminal to the first terminal and received by the first terminal. The first voice signal is a voice call signal picked up by the microphone of the first terminal after the first reference signal is played by the first terminal. Without considering the delay, the moment when the first terminal receives the first reference signal, the moment when the first terminal plays the first reference signal, and the moment when the microphone of the first terminal picks up the first voice signal belong to the same moment.

[0057] In one possible implementation, the method is applied to a scenario in which a first terminal and a second terminal are having a call. Before obtaining a first correlation coefficient between a first residual signal and a first echo signal, the method further includes: obtaining a first reference signal and a first voice signal in the context of the call, wherein the first reference signal is a voice call signal from the second terminal received by the first terminal, and the first voice signal is a voice call signal picked up by the near-end microphone after the first reference signal is played by the first terminal; obtaining the first echo signal and the first residual signal based on the adaptive linear filtering echo cancellation technology, the first reference signal, and the first voice signal, wherein the first echo signal is a linear echo signal obtained from the first voice signal based on the adaptive linear filtering echo cancellation technology, and the first residual signal is other signals in the first voice signal except the first echo signal.

[0058] Exemplarily, the above-mentioned adaptive linear filtering echo cancellation method can be an adaptive linear filtering echo cancellation technology based on time domain, for example, normalized least mean square adaptive linear filtering (NLMS) and recursive least square (RLS); or, the adaptive linear filtering echo cancellation method can also be an adaptive linear filtering echo cancellation technology based on frequency domain, for example, frequency domain adaptive linear filtering (FDAF).

[0059] In the embodiment of the present application, the first correlation coefficient is used to indicate the similarity between the first residual signal and the first echo signal. The greater the similarity between the first residual signal and the first echo signal, the more residual echo signal is contained in the first residual signal. The value of the first correlation coefficient is greater than 0 and less than 1. The closer the first correlation coefficient is to 0, the smaller the similarity between the first residual signal and the first echo signal is. The closer the first correlation coefficient is to 1, the greater the similarity between the first residual signal and the first echo signal is. The specific calculation method of the first correlation coefficient can refer to the relevant description in step S202 below, which will not be described in detail here.

[0060] In a possible implementation manner, the first correlation coefficient satisfies the following formula: Wherein, the η(n) is used to represent the first correlation coefficient, the P ey (n) and the P yy (n) is related to the first speech signal, the P ey (n-1) and the P yy (n-1) is related to the second voice signal, the first voice signal and the second voice signal belong to the nth frame signal and the n-1th frame signal in the same voice signal, respectively, and n is greater than or equal to 2; the first voice signal and the second voice signal both include M frequency points, and M is greater than or equal to 1, and k is used to represent the kth frequency point among the M frequency points, and k is a positive integer greater than 1 and less than M, and R ey (n, k) is the cross power spectrum of the kth echo signal and the kth residual signal, the kth echo signal is the complex spectrum corresponding to the kth frequency point in the first echo signal, the kth residual signal is the complex spectrum corresponding to the kth frequency point in the first residual signal, is the sum of the cross-power spectra of the M frequency points of the first speech signal; β(n) is the product of the ratio of the energy of the first echo signal to the energy of the first residual signal and a first constant, the first constant being greater than 0 and less than 0.1; R yy (n, k) is the autopower spectrum of the kth echo signal, The sum of the M auto-power spectra corresponding to the M frequency points of the first speech signal is used to represent the first speech signal; wherein, when n is equal to 2, the P ey (n-1) is The P yy (n-1) is Said is the sum of the cross-power spectra of the residual signal and the echo signal corresponding to the M frequency points of the second speech signal, is the sum of the auto-power spectra of the echo signal corresponding to the M frequency points of the second speech signal. For details, please refer to the relevant description of step S202 below, which will not be described in detail here.

[0061] S102: The first terminal determines a second correlation coefficient based on the first correlation coefficient and the attenuation coefficient.

[0062] In the embodiment of the present application, the second correlation coefficient is less than or equal to the first correlation coefficient. It is understandable that the attenuation coefficient may have different value ranges based on the specific manner in which the first terminal determines the second correlation coefficient using the first correlation coefficient and the attenuation coefficient.

[0063] Exemplarily, the first terminal determines the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient as follows: the first terminal determines the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient, and the value of the attenuation coefficient is greater than 0 and less than or equal to 1. When the attenuation coefficient is greater than 0 and less than 1, the second correlation coefficient is less than the first correlation coefficient, and when the attenuation coefficient is 1, the second correlation coefficient is equal to the first correlation coefficient. For example, the following description of Figure 2 (See step S204 for details) or Figure 4 The acoustic echo processing method shown (specifically referring to step S401 or step S402) is to determine the second correlation coefficient by multiplying the first correlation coefficient by the attenuation coefficient.

[0064] Exemplarily, the first terminal determines the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient as follows: the first terminal determines the second correlation coefficient based on the first correlation coefficient raised to the power of the attenuation coefficient, for example, the second correlation coefficient is equal to the first power, the second power, or the third power of the first correlation coefficient. The value of the attenuation coefficient may be a positive integer greater than or equal to 1 and less than or equal to 5. When the attenuation coefficient is 1, the second correlation coefficient is equal to the first correlation coefficient.

[0065] Exemplarily, the first terminal determines the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient as follows: the first terminal determines the second correlation coefficient based on the difference between the first correlation coefficient and the attenuation coefficient. Then the value of the attenuation coefficient can be greater than or equal to 0 and less than or equal to the fourth threshold value, for example, the fourth threshold value can be one-Lth of the first correlation coefficient, for example, the fourth threshold value can be one-half, one-third, or one-quarter of the first correlation coefficient. When the attenuation coefficient is equal to 0, the second correlation coefficient is equal to the first correlation coefficient, and when the attenuation coefficient is greater than 0 and less than or equal to the fourth threshold value, the second correlation coefficient is less than the first correlation coefficient.

[0066] Exemplarily, the first terminal may determine the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient as follows: the first terminal may determine the second correlation coefficient based on the quotient of the first correlation coefficient and the attenuation coefficient. Then the value of the attenuation coefficient may be greater than or equal to 1 and less than or equal to 5. When the attenuation coefficient is 1, the second correlation coefficient is equal to the first correlation coefficient, and when the attenuation coefficient is greater than 1 and less than 5 (for example, the value is 3), the second correlation coefficient is less than the first correlation coefficient.

[0067] It is understandable that, according to specific needs or designs, the first terminal can calculate the second correlation coefficient based on multiplication, division, subtraction, or power, all operations that can make the second correlation coefficient less than or equal to the first correlation coefficient. This document does not limit this.

[0068] In the embodiment of the present application, based on the specific method used by the first terminal to determine the second correlation coefficient, the specific value of the attenuation coefficient may have the following possibilities.

[0069] Exemplarily, when the first terminal determines the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient, if the voice scene corresponding to the first voice signal belongs to the first scene, the attenuation coefficient is greater than 0 and less than 1 (for example, the attenuation coefficient is 0.3). If the voice scene corresponding to the first voice signal belongs to the second scene, the attenuation coefficient is equal to 1.

[0070] Exemplarily, when the first terminal determines the second correlation coefficient based on the first correlation coefficient raised to the power of the attenuation coefficient, if the voice scene corresponding to the first voice signal belongs to the first scene, the attenuation coefficient is greater than 0 and less than or equal to 5 (for example, the attenuation coefficient is equal to 2). If the voice scene corresponding to the first voice signal belongs to the second scene, the attenuation coefficient is equal to 1.

[0071] Exemplarily, when the first terminal determines the second correlation coefficient based on the difference between the first correlation coefficient and the attenuation coefficient, if the voice scene corresponding to the first voice signal belongs to the first scene, the attenuation coefficient is greater than 0 and less than or equal to the fourth threshold value (for example, the attenuation coefficient is half of the first correlation coefficient). If the voice scene corresponding to the first voice signal belongs to the second scene, the attenuation coefficient is equal to 0.

[0072] In the embodiment of the present application, the first scenario can be understood as being used to indicate that there are other signals in the first residual signal other than the residual echo signal that are correlated with the first echo signal, that is, if the residual echo signal in the first residual signal is estimated based on the first correlation coefficient, there will be an over-estimation problem, and thus it is necessary to attenuate the first correlation coefficient and use the second correlation coefficient to perform residual echo processing on the first residual signal to improve the distortion effect caused to the first residual signal by the over-estimation.

[0073] Exemplarily, the first scenario is a loud noise scenario or a dual-talk scenario. In other descriptions herein, the first scenario is also referred to as a large near-end interference scenario. The second scenario is a scenario other than the first scenario. It can also be understood that the voice scene corresponding to the first voice signal belongs to the second scenario. Specifically, the voice scene corresponding to the first voice signal does not belong to the first scenario (that is, the voice scene corresponding to the first voice signal does not belong to the noise scenario or the dual-talk scenario).

[0074] The following implementation methods may be used to specifically determine the scene corresponding to the first voice signal.

[0075] In a possible implementation, when a first value is less than a first threshold, the speech scene corresponding to the first speech signal belongs to the first scene, and the first value is determined based on energy of the first echo signal, energy of the first residual signal, and a first constant.

[0076] In some descriptions of this document, the first value is also called the first smoothing coefficient. For example, the first value (β(n)) satisfies the formula Wherein, β0 is a first constant, and the value of β0 is greater than 0 and less than 0.1. For example, the value of β0 is greater than or equal to 0.02 and less than or equal to 0.05. Power(Y(n)) is the energy of the first echo signal, and Power(E(n)) is the energy of the first residual signal. The specific calculation method can refer to the relevant description of formula 5 to formula 7 in step S2022 below. The description of the first threshold can refer to the relevant description of judgment method 1.1 in step S203 below, which will not be described in detail here.

[0077] It is understandable that when the first speech signal contains more noise signals, since the adaptive linear filtering echo cancellation technology filters out less noise signals as linear echo signals into the first echo signal, most of the noise signals will remain in the first residual signal, that is, the energy value of the first residual signal changes greatly, while the energy value of the first echo signal changes less, then the smaller the value of the above β(n).

[0078] It is understandable that when the scene corresponding to the first voice signal belongs to a single-talk state, the first voice signal may contain a noise signal and an echo signal; and when the scene corresponding to the first voice signal belongs to a double-talk state, in addition to the noise signal and the echo signal, the first voice signal also contains the voice signal of the near-end caller of the first terminal. At this time, since the echo cancellation technology of adaptive linear filtering generally does not filter the voice signal of the near-end caller as a linear echo signal into the first echo signal, the voice signal of the near-end caller will be left in the first residual signal, that is, the energy value of the first residual signal changes greatly, while the energy value of the first echo signal changes less, the smaller the value of the above β(n).

[0079] Therefore, it can be determined that the current scene is a loud noise scene or a double talk scene based on the β(n) value being less than the first threshold.

[0080] In some other possible implementations, when the second numerical value corresponding to the first voice signal is less than a second threshold, the voice scene corresponding to the first voice signal belongs to the first scene, the second numerical value corresponding to the first voice signal is determined based on the energy of the first echo signal, the energy of the first residual signal, the first constant, the first voice signal, the second numerical value corresponding to the second voice signal, and the smoothing coefficient, and the second voice signal and the first voice signal belong to two adjacent frames of voice signals in the same segment of voice signals.

[0081] Exemplarily, in a possible implementation, the second value corresponding to the first speech signal satisfies the following formula β smooth (n) = β(n)*α+β smooth (n-1)*(1-α); wherein, the β smooth (n) is a second value determined according to the parameters of the first speech signal, and the β smooth (n-1) is a second value determined according to the parameters of the second speech signal, the first speech signal and the second speech signal belong to the nth frame signal and the n-1th frame signal in the same speech signal, respectively, n is greater than or equal to 2, β(n) is a first value, the first value is determined based on the energy of the first echo signal, the energy of the first residual signal, and a first constant, and α is the smoothing coefficient. When n is 2, β smooth(n-1) is β(n-1), and β(n-1) is the product of the ratio of the energy of the second echo signal to the energy of the second residual signal and the first constant, and the second echo signal and the second residual signal correspond to the second speech signal. Exemplarily, the specific calculation method and the description of the second threshold value can be related to the description of formula 11 of the judgment method 1.2.1 in step S203 below, which will not be described in detail here.

[0082] In the embodiment of the present application, if the first terminal obtains β after smoothing β(n), smooth (n) Determine whether the first speech signal is in a large near-end interference scenario, because the β smooth (n) contains the scene features of the n frames of signals before the first speech signal, rather than only the scene features of the current frame signal, i.e., the first speech signal, so that based on the β smooth (n) The judgment result obtained by judging whether the first speech signal is in a large near-end interference scenario has high stability and accuracy, and can reduce the interference degree of unknown unstable factors on the judgment result.

[0083] In some other possible implementations, the second voice signal and the first voice signal belong to two adjacent frames of voice signals in the same voice signal segment, which means that the second voice signal is the previous frame signal of the first voice signal, that is, the first voice signal belongs to the nth frame signal, and the second voice signal belongs to the n-1th frame signal. The first terminal can also determine the second value based on the judgment method 1.2.2 formula 12 in the following step S203, which is not limited in this document.

[0084] In some other possible implementations, the second voice signal and the first voice signal belong to two adjacent frames of voice signals in the same segment of voice signal, which means that the second voice signal is the next frame signal of the first voice signal, that is, the first voice signal belongs to the nth frame signal, and the second voice signal belongs to the n+1th frame signal. The specific calculation method of the corresponding second value and the description of the second threshold value can refer to the relevant description of formula 13 of judgment method 1.2.3 in step S203 below, which will not be described in detail here.

[0085] The beneficial effects of various different methods of determining whether the first speech signal is in the first scene based on the first value or the second value are as follows: Figure 2 There are related instructions in step S203 below, which will not be described in detail here.

[0086] In some other possible implementations, when the amplitude of the target noise signal is greater than a third threshold, the speech scene corresponding to the first speech signal belongs to the first scene, and the target noise signal is used to indicate the noise intensity of the first speech signal.

[0087] In a possible implementation, each frame signal in the first speech signal includes M frequency points, where M is greater than or equal to 1, and the target noise signal includes M noise complex spectra corresponding one-to-one to the M frequency points, and the amplitude of the target noise signal is the sum of M modules corresponding to the M noise complex spectra.

[0088] Exemplarily, the target noise signal is a noise signal obtained by performing noise estimation on the first speech signal. Specifically, the amplitude of the target noise signal is the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the first speech signal, and specific reference is made to the relevant description of formula 14 in judgment method 2.1 in step S203 below, which will not be described in detail here.

[0089] It is understandable that the first echo signal will contain some noise signals, which will reduce the noise signal contained in the first residual signal compared to the noise signal contained in the first voice signal, and the proportion of the noise signal in the first echo signal obtained by adaptive linear filtering of two different frame signals to the total noise signal will also be erratic. In other words, the complete first voice signal contains more complete noise scene features than the first residual signal, and the use of the noise estimation of the complete first voice signal to determine whether the scene where the first voice signal is located is in a large near-end interference scene is more representative, and the judgment result obtained is more accurate.

[0090] Exemplarily, the target noise signal is a noise signal obtained by performing noise estimation on the second speech signal, and the second speech signal and the first speech signal belong to two adjacent frames of speech signals in the same segment of speech signal. For example, the amplitude of the target noise signal is the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the second speech signal (the second speech signal is the previous frame signal of the first speech signal), specifically refer to the relevant description of formula 15 in judgment method 2.2.1 in step S203 below. For example, the amplitude of the target noise signal is the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the second speech signal (the second speech signal is the next frame signal of the first speech signal), specifically refer to the relevant description of formula 16 in judgment method 2.2.2 in step S203 below, which will not be described in detail here.

[0091] Exemplarily, the target noise signal is a noise signal obtained by performing noise estimation on the first residual signal. Specifically, the amplitude of the target noise signal is the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the first residual signal. For details, refer to the relevant description of formula 17 in judgment method 2.3 in step S203 below, which will not be described in detail here.

[0092] It is understandable that the first echo signal generally contains less noise signal, and most of the noise signal will remain in the first residual signal. The noise estimation of the first residual signal is also sufficient to represent the noise environment in which the first speech signal is located. Moreover, compared with the noise estimation of the first echo signal, the computational complexity of the noise estimation of the first residual signal is smaller, especially when the number of frame signals is large, which can save performance loss to a certain extent.

[0093] Exemplarily, the target noise signal is a noise signal obtained by performing noise estimation on the second residual signal, the second residual signal corresponds to the second speech signal, and the second speech signal and the first speech signal belong to two adjacent frames of speech signals in the same segment of speech signal. For example, the amplitude of the target noise signal is the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the second residual signal, specifically refer to the relevant description of formula 18 in judgment method 2.4.1 in step S203 below. For example, the amplitude of the target noise signal is the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the third residual signal, specifically refer to the relevant description of formula 19 in judgment method 2.4.2 in step S203 below.

[0094] In some other possible implementations, the conditions of the first threshold and the third threshold may be combined, or the conditions of the second threshold and the third threshold may be combined to determine whether the voice scene corresponding to the first voice signal belongs to the first scene.

[0095] Exemplarily, when the first value is smaller than the first threshold and the amplitude of the target noise signal is larger than the third threshold, the speech scene corresponding to the first speech signal belongs to the first scene.

[0096] Exemplarily, when the second value corresponding to the first speech signal is less than the second threshold and the amplitude of the target noise signal is greater than the third threshold, the speech scene corresponding to the first speech signal belongs to the first scene.

[0097] For example, when the second value corresponding to the first speech signal is less than the second threshold value and the amplitude of the target noise signal is greater than the third threshold value, the speech scene corresponding to the first speech signal belongs to the first scene, and the target noise signal is used to indicate the noise intensity of the first speech signal; wherein the second value satisfies the formula β smooth (n) = β(n)*α+β smooth (n-1)*(1-α); β(n) is a first value, which is determined based on the energy of the first echo signal, the energy of the first residual signal, and a first constant, and β smooth (n) is used to represent the second value corresponding to the first speech signal, the β smooth(n-1) is used to represent a second value corresponding to a second speech signal, the first speech signal and the second speech signal belong to the nth frame signal and the n-1th frame signal in the same speech signal, respectively, n is greater than or equal to 2, α is the smoothing coefficient; and when n is 2, β smooth (n-1) is taken as β(n-1), where β(n-1) is the product of the ratio of the energy of the second echo signal to the energy of the second residual signal and the first constant.

[0098] S103: The first terminal performs residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal.

[0099] In a possible implementation, the performing residual echo processing on the first residual signal based on the second correlation coefficient to obtain the second speech signal includes: obtaining a residual echo signal based on the product of the spectrum of the first residual signal and the second correlation coefficient; performing noise estimation on the first residual signal to obtain a first noise signal; and removing the residual echo signal and the first noise signal in the first residual signal by frequency subtraction to obtain the second speech signal. For details, please refer to the relevant description of steps S205 to S208 below, which will not be described in detail here.

[0100] It is understandable that the residual echo processing of the first residual signal based on the second correlation coefficient described in steps S205 to S208 below includes two steps of noise reduction and residual echo processing, also known as fusion noise reduction. In some other possible implementations, the first terminal may also first perform residual echo processing on the first residual signal, and then perform noise reduction processing on the second voice signal obtained after the echo processing to obtain a third voice signal, which is the voice signal obtained after the noise reduction and residual echo processing. The first terminal may send the third voice signal as the voice signal of the near-end caller to the second terminal. Alternatively, in some other possible implementations, the first terminal may first perform residual echo processing on the first residual signal to obtain a third voice signal, and then perform residual echo processing on the third voice signal obtained after the noise reduction processing to obtain a second voice signal, which is the voice signal obtained after the noise reduction and residual echo processing. The first terminal may send the second voice signal as the voice signal of the near-end caller to the second terminal.

[0101] The following combination Figure 2 Detailed description of this application Figure 1 The specific implementation method of the acoustic echo processing method provided. Figure 2 As shown, the method includes:

[0102] S201: A first terminal obtains a first echo signal and a first residual signal corresponding to a first voice signal.

[0103] For the meanings of the first speech signal, the first residual signal, the first echo signal, and the first correlation coefficient, reference may be made to other related descriptions herein, such as the related description in the above step S101.

[0104] S202: The first terminal determines a first correlation coefficient between a first echo signal and a first residual signal based on an echo cancellation technology of residual echo suppression.

[0105] In the embodiment of the present application, the first correlation coefficient is used to indicate the similarity between the first residual signal and the first echo signal.

[0106] For example, Figure 3 As shown, the first terminal determines the first correlation coefficient between the first echo signal and the first residual signal based on the echo cancellation technology of residual echo suppression, which may specifically include:

[0107] S2021. The first terminal calculates a first autopower spectrum of the first echo signal, and calculates a first cross-power spectrum of the first echo signal and the first residual signal.

[0108] In the embodiment of the present application, a frame signal includes M complex spectra corresponding to M frequency points, that is, the first echo signal includes complex spectra corresponding to M frequency points, and the first residual signal includes M complex spectra corresponding to M frequency points.

[0109] Exemplarily, the first auto power spectrum is the sum of M auto power spectra corresponding to M frequency points in the first echo signal, that is, the first auto power spectrum satisfies the following formula 1. The first cross power spectrum is the sum of M cross power spectra corresponding to M frequency points in the first residual signal, that is, the first cross power spectrum satisfies the following formula 2.

[0110]

[0111]

[0112] Wherein, k is used to represent the kth frequency point among M frequency points, and K is a positive integer greater than 1 and less than or equal to M. yy (n,k) is used to represent the autopower spectrum corresponding to the kth frequency point in the first echo signal, R ey (n, k) is used for the cross power spectrum corresponding to the kth frequency point in the first residual signal. yy (n,k) and the R ey (n, k) satisfies the following formula 3 and formula 4 respectively.

[0113] R yy (n,k)=real(Y(n,k)*Y(n,k)) * ) Formula 3

[0114] R ey (n,k)=real(E(n,k)*Y(n,k) * ) Formula 4

[0115] Wherein, Y(n,k) is the complex spectrum of the frequency point signal corresponding to the kth frequency point in the first echo signal, and Y(n,k) * is the conjugate spectrum of Y(n,k), E(n,k) is the complex spectrum of the frequency point signal corresponding to the kth frequency point in the first residual signal, and real represents the real part of the complex number, that is, R yy (n,k) is the pair E(n,k) and Y(n,k) * The real value obtained by taking the real part of the product is R ey (n,k) is E(n,k) and Y(n,k) * The real value obtained by taking the real part of the product of .

[0116] S2022. The first terminal calculates a first smoothing coefficient according to an energy ratio between the first echo signal and the first residual signal, and a first constant.

[0117] In the present application, the first smoothing coefficient satisfies the following formula 5.

[0118]

[0119] Wherein, β0 is a first constant, and the value of β0 is greater than 0 and less than 0.1. For example, the value of β0 is greater than or equal to 0.02 and less than or equal to 0.05. Power(Y(n)) is the energy of the first echo signal, and Power(E(n)) is the energy of the first residual signal.

[0120] In the embodiment of the present application, the energy of the first echo signal is the sum of the squares of the complex spectra corresponding to each of the M frequency points included in the first echo signal, that is, the above Power(Y(n)) satisfies the following formula 6. The energy of the first residual signal is the sum of the squares of the complex spectra corresponding to each of the M frequency points included in the first residual signal, that is, the above Power(E(n)) satisfies the following formula 7.

[0121]

[0122]

[0123] For the description of Y(n, k) and E(n, k), reference may be made to the relevant descriptions in Formula 3 and Formula 4 above, which will not be repeated here.

[0124] S2023. The first terminal calculates a value A based on the first smoothing coefficient, the first autopower spectrum, and related parameters of the second speech signal, and calculates a value B based on the first smoothing coefficient, the first cross-power spectrum, and related parameters of the second speech signal.

[0125] In the embodiment of the present application, the above parameter A satisfies the following formula 8, and the parameter B satisfies the following formula 9.

[0126]

[0127]

[0128] Among them, P ey (n) is used to represent parameters A, P yy (n) is used to represent the parameter B, and β(n) is the first smoothing coefficient mentioned above.

[0129] P ey (n-1), P yy (n-1) is a related parameter corresponding to the second voice signal, and the second voice signal is the previous frame signal of the first voice signal, that is, the second voice signal is the n-1th frame signal. yy (n-1) is determined based on the second autopower spectrum corresponding to the second echo signal, the energy ratio of the second echo signal to the second residual signal, and the first constant, and specifically calculates the same as P yy The calculation principle of P is the same as that of P, which will not be described in detail. ey (n-1) is determined based on the second cross power spectrum corresponding to the second echo signal and the second residual signal, the energy ratio of the second echo signal to the second residual signal, and the first constant, and specifically calculates the value corresponding to P ey The calculation principle of (n) is the same and will not be described in detail.

[0130] Among them, when n is equal to 2, P ey (n-1) is The P yy (n-1) is is the sum of the cross power spectra of the residual signal and the echo signal corresponding to the M frequency points of the second speech signal, is the sum of the autopower spectra of the echo signal corresponding to the M frequency points of the second speech signal.

[0131] In an embodiment of the present application, the second echo signal may include part or all of the linear echo signal in the second voice signal, and the second residual signal is other signals in the second voice signal except the second echo signal. Exemplarily, the second residual signal may include a linear echo component and / or a nonlinear echo component in the second voice signal, and the second residual signal also includes a near-end signal corresponding to the n-1th frame signal, and the near-end signal includes a voice signal of the near-end talker and / or a noise signal of the environment in which the first terminal is located.

[0132] The second echo signal is a linear echo signal obtained from the second voice signal by the echo cancellation method based on adaptive linear filtering of the first terminal and the second reference signal. The linear echo signal is part or all of the signal in the second voice signal that is linearly related to the second reference signal. The second reference signal is a voice call signal sent by the second terminal to the first terminal and received by the first terminal. The second voice signal is a voice call signal picked up by the microphone of the first terminal after the second reference signal is played by the first terminal. Without considering the delay, the moment when the first terminal receives the second reference signal, the moment when the first terminal plays the second reference signal, and the moment when the microphone of the first terminal picks up the second voice signal belong to the same moment.

[0133] It is understandable that the above S2022 and S2023 can be executed simultaneously or sequentially, and this document does not limit the order of their execution.

[0134] S2024. The first terminal determines the first correlation coefficient based on the ratio of the value A to the value B.

[0135] In the embodiment of the present application, the first correlation coefficient satisfies the following formula 10, where η(n) is used to represent the first correlation coefficient.

[0136]

[0137] In the embodiment of the present application, the above steps S2021 and S2022 can be executed simultaneously or sequentially, and the order of their execution is not limited herein.

[0138] S203: The first terminal determines whether the voice scene corresponding to the first voice signal belongs to a large near-end interference scene.

[0139] In the embodiment of the present application, the large near-end interference scenario may be a large noise scenario and / or a double-talk scenario, wherein the large noise scenario refers to a scenario where the signal strength is higher than a certain threshold, and the double-talk scenario refers to a scenario where both the caller of the first terminal and the caller of the second terminal speak at the same time.

[0140] In an embodiment of the present application, the first terminal may determine whether the voice scene corresponding to the first voice signal belongs to a large near-end interference scene by the following two methods (method 1 and method 2).

[0141] Method 1:

[0142] Based on the first smoothing coefficient, ie, β(n), it is determined whether the speech scene corresponding to the first speech signal belongs to a large near-end interference scene.

[0143] (Judgment method 1.1) Exemplarily, when it is determined that the above β(n) is less than the first threshold, it is determined that the speech scene corresponding to the first speech signal belongs to the large near-end interference scene. Exemplarily, the value of the first threshold is greater than 0 and less than or equal to 0.5, for example, the first threshold is 0.1 or 0.2.

[0144] (Judgment method 1.2) Exemplarily, when it is determined that the second value corresponding to the first speech signal is less than the second threshold, it is determined that the speech scene corresponding to the first speech signal belongs to the large near-end interference scene, and the second value corresponding to the first speech signal is related to the β(n). Exemplarily, the value of the second threshold is greater than 0 and less than or equal to 0.5, for example, the second threshold is 0.1 or 0.2.

[0145] (Judgment method 1.2.1) Exemplarily, the second value corresponding to the first speech signal satisfies the following formula 11, where β smooth (n) is the second value corresponding to the first speech signal, β smooth (n-1) is the second value corresponding to the second voice signal, and the second voice signal is the previous frame signal of the first voice signal (that is, the n-1th frame signal), and the β smooth (n-1) is related to β(n-1).

[0146] β smooth (n) = β(n)*α+β smooth (n-1)*(1-α) Formula 11

[0147] When n is 2, β smooth (n-1) takes the value of β(n-1), which is the product of the ratio of the energy of the second echo signal to the energy of the second residual signal and the first constant, and the second echo signal and the second residual signal correspond to the second speech signal.

[0148] It can be understood that, based on the fact that the β(n) value is less than the first threshold, it can be determined that the current scene is a loud noise scene or a double-talk scene, and the β(n) value obtained by smoothing the calculation is smooth (n), based on this β smooth (n) is less than the second threshold value, which can also be used to determine that the current scene is a loud noise scene or a double-talk scene.

[0149] In the embodiment of the present application, if the first terminal obtains β after smoothing β(n) based on formula 11, smooth (n) Determine whether the first speech signal is in a large near-end interference scenario, because the β smooth (n) contains the scene features of the n frames of signals before the first speech signal, rather than only the scene features of the current frame signal, i.e., the first speech signal, so that based on the β smooth (n) The judgment result obtained by judging whether the first speech signal is in a large near-end interference scenario has high stability and accuracy, and can reduce the interference degree of unknown unstable factors on the judgment result.

[0150] (Judgment method 1.2.2) Exemplarily, the second value corresponding to the first speech signal satisfies the following formula 12, where β smooth (n) is the second numerical value corresponding to the first speech signal, β(n-1) is the second smoothing coefficient corresponding to the second speech signal, and the calculation method of the second smoothing coefficient is consistent with the calculation method of the first smoothing coefficient, which will not be described in detail here.

[0151] β smooth (n)=β(n)*α+β(n-1)*(1-α) Formula 12

[0152] When the first speech signal is the first frame signal, β smooth (n) = β(n).

[0153] It can be understood that, based on the fact that the β(n) value is less than the first threshold, it can be determined that the current scene is a loud noise scene or a double-talk scene, and the β(n) and β(n-1) obtained by smoothing the calculation are smooth (n), based on this β smooth (n) is less than the second threshold value, which can also be used to determine that the current scene is a loud noise scene or a double-talk scene.

[0154] In the embodiment of the present application, if the first terminal obtains β after smoothing β(n) based on formula 12, smooth (n) Determine whether the first speech signal is in a large near-end interference scenario, because the β smooth (n) includes the scene features of the first speech signal and a frame of speech signal before the first speech signal, that is, the second speech signal, rather than only including the scene features of the current frame signal, that is, the first speech signal, so that based on the β smooth (n) The judgment result obtained by judging whether the first speech signal is in a large near-end interference scenario has good stability and accuracy, and can reduce the interference degree of unknown unstable factors on the judgment result to a certain extent.

[0155] (Judgment method 1.2.3) Exemplarily, when the first voice signal is not the last frame signal, the second value corresponding to the first voice signal satisfies the following formula 12, where β smooth (n) is the second value corresponding to the first speech signal, β smooth (n+1) is the second value corresponding to the third voice signal, and the third voice signal is the next frame signal of the first voice signal (that is, the n+1th frame signal). smooth (n+1) is related to β(n+1).

[0156] β smooth (n)=β(n)*α+β(n+1)*(1-α) Formula 13

[0157] When the first speech signal is the last frame signal, β smooth (n) = β(n).

[0158] It can be understood that, based on the fact that the β(n) value is less than the first threshold, it can be determined that the current scene is a loud noise scene or a double-talk scene, and the β(n) value obtained by smoothing the calculation is smooth (n), based on this β smooth (n) is less than the second threshold value, which can also be used to determine that the current scene is a loud noise scene or a double-talk scene.

[0159] In the embodiment of the present application, if the first terminal obtains β after smoothing β(n) based on formula 13, smooth (n) Determine whether the first speech signal is in a large near-end interference scenario, because the β smooth (n) includes the scene features of the first speech signal and a frame of speech signal after the first speech signal, that is, the third speech signal, rather than only including the scene features of the current frame signal, that is, the first speech signal, so that based on the β smooth (n) The judgment result obtained by judging whether the first speech signal is in a large near-end interference scenario has good stability and accuracy, and can reduce the interference degree of unknown unstable factors on the judgment result to a certain extent.

[0160] Method 2:

[0161] Based on the noise intensity of the noise environment in which the first speech signal is located, it is determined whether the speech scene corresponding to the first speech signal belongs to a large near-end interference scene.

[0162] (Judgment method 2.1) Exemplarily, when it is determined that the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the first speech signal is greater than the third threshold, it is determined that the speech scene corresponding to the first speech signal belongs to the large near-end interference scene, and the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the first speech signal satisfies the following formula 14. Exemplarily, the value of the third threshold is greater than 0 and less than or equal to 0.5, for example, the value of the second threshold is 0.1 or 0.2.

[0163]

[0164] Among them, S1(n,k) is used to represent the amplitude spectrum corresponding to the kth frequency point in the first speech signal. It can be understood that the amplitude spectrum can also be understood as the amplitude of the frequency point signal, and the amplitude spectrum is a real number.

[0165] (2.2) Exemplarily, based on the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectra of the adjacent speech signals of the first speech signal, it is determined whether the speech scene corresponding to the first speech signal belongs to a large near-end interference scene.

[0166] (Judgment method 2.2.1) For example, when it is determined that the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the second speech signal is greater than the third threshold, it is determined that the speech scene corresponding to the first speech signal belongs to a large near-end interference scene, and the second speech signal is the previous frame signal of the first speech signal, that is, the second speech signal is the n-1th frame signal, and the noise estimation amplitude spectrum of the second speech signal satisfies the following formula 15.

[0167]

[0168] Among them, S1(n-1,k) is used to represent the amplitude spectrum corresponding to the kth frequency point in the noise estimation amplitude spectrum of the second speech signal.

[0169] (Judgment method 2.2.2) For example, when it is determined that the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the third speech signal is greater than the third threshold, it is determined that the speech scene corresponding to the first speech signal belongs to a large near-end interference scene, and the third speech signal is the next frame signal of the first speech signal, that is, the third speech signal is the n+1th frame signal, and the noise estimation amplitude spectrum of the second speech signal satisfies the following formula 16.

[0170]

[0171] Among them, S1(n+1,k) is used to represent the amplitude spectrum corresponding to the kth frequency point in the noise estimation amplitude spectrum of the second speech signal.

[0172] (Judgment method 2.3) Exemplarily, when it is determined that the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the first residual signal is greater than the third threshold, it is determined that the speech scene corresponding to the first speech signal belongs to a large near-end interference scene, and the noise estimation amplitude spectrum of the first residual signal satisfies the following formula 17.

[0173]

[0174] Among them, S2(n,k) is used to represent the amplitude spectrum corresponding to the kth frequency point in the noise estimation amplitude spectrum of the first residual signal.

[0175] (2.4) Exemplarily, based on the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the residual signal corresponding to the adjacent speech signals of the first speech signal, it is determined whether the speech scene corresponding to the first speech signal belongs to a large near-end interference scene.

[0176] (Judgment method 2.4.1) For example, when it is determined that the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the second residual signal is greater than the third threshold, it is determined that the voice scene corresponding to the first voice signal belongs to a large near-end interference scene, and the second residual signal is the residual signal corresponding to the second voice signal, and the second voice signal is the n-1th frame signal, and the noise estimation amplitude spectrum of the second residual signal satisfies the following formula 18.

[0177]

[0178] Among them, S2(n-1,k) is used to represent the amplitude spectrum corresponding to the kth frequency point in the noise estimation amplitude spectrum of the second residual signal.

[0179] (Judgment method 2.4.2) For example, when it is determined that the sum of the amplitude spectra corresponding to each frequency point in the noise estimation amplitude spectrum of the third residual signal is greater than the third threshold, it is determined that the voice scene corresponding to the first voice signal belongs to a large near-end interference scene, and the third residual signal is the residual signal corresponding to the third voice signal, and the third voice signal is the n+1th frame signal, and the noise estimation amplitude spectrum of the third residual signal satisfies the following formula 19.

[0180]

[0181] Among them, S2(n+1,k) is used to represent the amplitude spectrum corresponding to the kth frequency point in the noise estimation amplitude spectrum of the third residual signal.

[0182] Exemplarily, the noise estimation method used to obtain the above noise estimation amplitude spectrum may be a minimum tracking method. It is understandable that, in addition to the minimum tracking method, any other noise reduction method that can obtain the noise estimation amplitude spectrum is feasible.

[0183] In an embodiment of the present application, the first terminal can determine that the voice scene corresponding to the first voice signal belongs to a large near-end interference scenario by β(n) satisfying the conditions in one of the judgment methods in method 1, or by the noise intensity of the noise environment in which the first voice signal is located satisfying the conditions in one of the judgment methods in method 2.

[0184] In some other possible implementation methods, the first terminal may also combine method 1 and method 2, and determine that the voice scene corresponding to the first voice signal belongs to a large near-end interference scene when β(n) satisfies the conditions in one of the judgment methods in method 1 and the noise intensity of the noise environment in which the first voice signal is located satisfies the conditions in one of the judgment methods in method 2.

[0185] It is understandable that step S203 can be executed simultaneously with S201 or S202 or executed sequentially, and the order of execution is not limited herein.

[0186] When the first terminal determines that the voice scene corresponding to the first voice signal belongs to the large near-end interference scene, step S204 is executed. When the first terminal determines that the voice scene corresponding to the first voice signal does not belong to the large near-end interference scene, step S206 is executed.

[0187] S204: The first terminal corrects the first correlation coefficient based on the product of the attenuation coefficient A and the first correlation coefficient to obtain a second correlation coefficient.

[0188] In an embodiment of the present application, the attenuation coefficient A is greater than 0 and less than 1, for example, the attenuation coefficient A is 0.3, and the second correlation coefficient is equal to the product of the attenuation coefficient A and the first correlation coefficient.

[0189] In the embodiment of the present application, when it is determined that the current state is a large near-end interference state, the first correlation coefficient η(n) is attenuated to prevent the residual echo from being over-estimated. Exemplarily, the second correlation coefficient satisfies the following formula 20, where η′(n) is the second correlation coefficient and G is the attenuation coefficient A.

[0190] η′(n)=η(n)*G Formula 20

[0191] In some other acoustic echo processing methods, the processing methods corresponding to the first voice signal being in a large near-end interference scenario or not being in a large near-end interference scenario are the same, and the first correlation coefficient is used to estimate the residual echo signal in the first residual signal. However, technical personnel have found that when the first voice signal is in a large near-end interference scenario, the first correlation coefficient is used to estimate the residual echo signal in the first residual signal, and the residual echo signal is removed from the first residual signal to finally obtain the second voice signal, and the voice signal of the near-end caller contained in the second voice signal is distorted. This indicates that there is an over-estimation problem in estimating the residual echo signal in the first residual signal using the first correlation coefficient, that is, when estimating the first correlation coefficient, the estimated value of the first correlation coefficient is too large, resulting in the residual echo signal calculated based on the first correlation coefficient containing the voice signal of the near-end caller.

[0192] However, by using the acoustic echo processing method provided in the present application, when the first voice signal is in a large near-end interference scenario, the first correlation coefficient is attenuated, thereby improving the problem of over-estimation of the first correlation coefficient in the large near-end interference scenario, which causes distortion of the near-end caller's voice signal, and improving the effect of the acoustic echo processing method.

[0193] S205: The first terminal estimates a residual echo signal in the first residual signal based on the second correlation coefficient.

[0194] In an embodiment of the present application, if the speech scene corresponding to the first speech signal belongs to a large near-end interference scene, the first terminal estimates the residual echo signal in the first residual signal based on the second correlation coefficient.

[0195] It can be understood that the residual echo signal includes a complex frequency spectrum corresponding to M frequency points.

[0196] Exemplarily, the complex spectrum RES(n,k) of the echo signal corresponding to the kth frequency point in the residual echo signal satisfies the following formula 21, where η′(n) is the above-mentioned second correlation coefficient, and Y(n,k) is the complex spectrum of the frequency point signal corresponding to the kth frequency point in the first echo signal.

[0197] RES(n,k)=η′(n)*Y(n,k) Formula 21

[0198] That is, the residual echo signal is

[0199] S206: The first terminal estimates a residual echo signal in the first residual signal based on the first correlation coefficient.

[0200] In an embodiment of the present application, if the speech scene corresponding to the first speech signal does not belong to a large near-end interference scene, the first terminal estimates the residual echo signal in the first residual signal based on the first correlation coefficient.

[0201] Exemplarily, the complex spectrum RES(n,k) of the echo signal corresponding to the kth frequency point in the residual echo signal satisfies the following formula 22, where η(n) is the above-mentioned first correlation coefficient, and Y(n,k) is the complex spectrum of the frequency point signal corresponding to the kth frequency point in the first echo signal.

[0202] RES(n,k)=η(n)*Y(n,k) Formula 22

[0203] S207: The first terminal calculates a noise signal in the first residual signal.

[0204] Exemplarily, the first terminal calculates the noise signal in the first residual signal based on the minimum tracking method. For example, the amplitude spectrum corresponding to the kth frequency point in the noise estimation amplitude spectrum of the noise signal is recorded as S(n,k), and the noise estimation amplitude spectrum of the noise signal is

[0205] It is understandable that step S207 can be executed simultaneously with step S205 or executed sequentially, and the order of execution is not limited herein.

[0206] S208: The first terminal performs noise reduction and residual echo processing on the first residual signal by spectral subtraction based on the residual echo signal and the noise signal to obtain a second speech signal.

[0207] Exemplarily, the amplitude value of the kth frequency point in the residual echo signal and the amplitude value of the kth frequency point in the noise signal are combined based on the following formula 23. Wherein, abs is used to represent the modulus of RES(n,k), that is, the amplitude of RES(n,k), and MER(n,k) is the sum of the amplitude values ​​of the noise signal at the kth frequency point and the residual echo signal.

[0208] MER(n,k)=S(n,k)+abs(RES(n,k)) Formula 23

[0209] Alternatively, the amplitude value of the kth frequency point in the residual echo signal and the amplitude value of the kth frequency point in the noise signal may be combined based on the following formula 24. Wherein, MER(n,k) is the larger amplitude value of the amplitude value of the kth frequency point in the residual echo signal and the amplitude value of the kth frequency point in the noise signal.

[0210] MER(n,k)=max(S(n,k),abs(RES(n,k))) Formula 24

[0211] Exemplarily, the first terminal removes the signal MER(n,k) obtained by combining the noise signal and the residual echo signal from the first residual echo signal in a spectral manner based on the following formula 25, and uses the phase of the first residual signal before spectral subtraction to restore the real spectrum obtained after spectral subtraction to a complex spectrum. Among them, OUT(n,k) is used to represent the complex spectrum corresponding to the kth frequency point in the complex spectrum obtained after spectral subtraction of the first residual echo signal, and phase(E(n,k)) is used to represent the phase of the first residual signal before spectral subtraction.

[0212] OUT(n,k)=(abs(E(n,k))-MER(n,k))*phase(E(n,k)) Formula 25

[0213] It is understandable that the final second speech signal can be expressed as

[0214] Combination Figure 4 This application also provides another Figure 1 The specific implementation method of the acoustic echo processing method shown, wherein the specific value of the attenuation coefficient can be determined based on whether the speech scene corresponding to the first speech signal belongs to a large near-end interference scene; when the speech scene corresponding to the first speech signal does not belong to a large near-end interference scene, the residual echo signal in the first residual signal can also be estimated based on the second correlation coefficient, wherein the attenuation coefficient is 1, and the second correlation coefficient is the product of the first correlation coefficient and the attenuation coefficient.

[0215] For example, Figure 4 As shown, the above steps S204 to S206 can be replaced by Figure 4 Steps S401 to S403 are shown. Figure 4 As shown, after step S203, the first terminal executes steps S401 to S403, which specifically include:

[0216] After the first terminal determines in step S203 that the voice scene corresponding to the first voice signal belongs to the large near-end interference scene, step S401 is executed. After the first terminal determines in step S203 that the voice scene corresponding to the first voice signal does not belong to the large near-end interference scene, step S402 is executed.

[0217] S401: The first terminal determines a second correlation coefficient based on a product of an attenuation coefficient B and a first correlation coefficient.

[0218] In the embodiment of the present application, the attenuation coefficient B is greater than 0 and less than 1, for example, the attenuation coefficient B is 0.3. The attenuation coefficient B may be equal to or different from the attenuation coefficient A, which is not limited herein.

[0219] S402: The first terminal determines a second correlation coefficient based on a product of the attenuation coefficient C and the first correlation coefficient.

[0220] In the embodiment of the present application, the attenuation coefficient C is 1, which can also be understood as no attenuation operation is performed on the first correlation coefficient.

[0221] The meaning of the first correlation coefficient and its calculation method can be referred to other related descriptions in this document, such as the related description of step S202 above, which will not be described in detail here.

[0222] S403: The first terminal estimates a residual echo signal in the first residual signal based on the second correlation coefficient.

[0223] The specific calculation method of estimating the residual echo signal in the first residual signal by the first terminal based on the second correlation coefficient can refer to other relevant descriptions of this document, such as the relevant description of step S205 above, which will not be described in detail here.

[0224] After step S403, the above steps S207 to S208 are executed.

[0225] Exemplarily, the first terminal is used to execute Figures 1 to 4 The functional modules of the acoustic echo processing method shown in any one of the items and the signal flow between the modules can be as follows: Figure 5 As shown, specifically including:

[0226] The microphone 51 is used to pick up a first voice signal 502 and send the first voice signal 502 to an adaptive linear filter (AF) 52 and a subtractor 53. The first voice signal 502 may include one or more of a voice signal of a near-end caller, a noise signal, or an echo signal.

[0227] The adaptive linear filter (AF) 52 is used to obtain part or all of the linear echo signal in the first speech signal 502 that is linearly related to the first reference signal 501 after receiving the first reference signal 501 and the first speech signal 502, obtain the first echo signal 503, and send the first echo signal 503 to the subtractor 53 and the residual echo estimation module 54.

[0228] The subtractor 53 is used to remove the first echo signal 503 from the first speech signal 502 after receiving the first speech signal 502 and the first echo signal 503, to obtain a first residual signal 504, and send the first residual signal 504 to the residual echo estimation module 54, the adaptive filter (AF) 52, and the residual echo suppression module 55.

[0229] The above-mentioned adaptive filter (AF) 52 is also used to update the filter coefficient based on the first residual signal 504 after receiving it, and filter the third voice signal based on the updated filter coefficient, and the third voice signal is the next frame signal of the first voice signal.

[0230] The residual echo estimation module 54 is used to estimate the correlation coefficient between the first echo signal 503 and the first residual signal 504 after receiving the first echo signal 503 and the first residual signal 504, and obtain the residual echo signal 505 in the first residual signal 504 based on the correlation coefficient.

[0231] Exemplarily, the residual echo estimation module 54 is specifically used to perform the following method steps:

[0232] S541 , the residual echo estimation module 54 calculates a first correlation coefficient.

[0233] That is, the residual echo estimation module 54 calculates a first correlation coefficient for indicating the similarity between the first echo signal 503 and the first residual signal 504. The specific calculation method may refer to other relevant descriptions herein, such as the relevant description of the above step S202, which will not be described in detail here.

[0234] S542: The residual echo estimation module 54 performs large near-end interference detection.

[0235] That is, the residual echo estimation module 54 determines whether the first speech signal 502 (or the first residual signal 504) is in a large near-end interference scenario. The specific determination method can refer to other relevant descriptions of this document, such as the relevant description of the above step S203, which will not be described in detail here.

[0236] It is understandable that if the voice signal used by the residual echo estimation module 54 for large near-end interference detection is the first voice signal 502, then the microphone 51 also needs to send the above-mentioned first voice signal 502 to the residual echo estimation module 54. After receiving the first voice signal 502, the residual echo estimation module 54 determines whether the voice scene corresponding to the first voice signal belongs to the large near-end interference scene based on the first voice signal 502. If the voice signal used by the residual echo estimation module 54 for large near-end interference detection is the first residual signal 504, then the microphone 51 does not need to send the above-mentioned first voice signal 502 to the residual echo estimation module 54.

[0237] When it is determined that the scenario belongs to the large near-end interference scenario, the residual echo estimation module 54 executes step S543 ; when it is determined that the scenario does not belong to the large near-end interference scenario, the residual echo estimation module 54 executes step S545 .

[0238] S543, the residual echo estimation module 54 corrects the first correlation coefficient to obtain a second correlation coefficient.

[0239] Exemplarily, the residual echo estimation module 54 determines the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient (such as the above-mentioned attenuation coefficient A or attenuation coefficient B). For details, please refer to other relevant instructions in this document, such as the above-mentioned step S204 or the above-mentioned step S501, which will not be described in detail here.

[0240] S544 , the residual echo estimation module 54 calculates the residual echo signal 505 in the first residual signal based on the second correlation coefficient.

[0241] When it is determined that the scenario belongs to a large near-end interference scenario, the residual echo estimation module 54 calculates the residual echo signal in the first residual signal based on the second correlation coefficient. For details, please refer to other relevant descriptions in this document, such as the above step S205, which will not be described in detail here.

[0242] S545 , the residual echo estimation module 54 calculates the residual echo signal 505 based on the first correlation coefficient.

[0243] When it is determined that it does not belong to the large near-end interference scenario, the residual echo estimation module 54 calculates the residual echo signal in the first residual signal based on the first correlation coefficient. For details, please refer to other relevant instructions in this article, such as the above-mentioned step S206, which will not be described in detail here. After executing steps S541 to S545, the residual echo estimation module 54 can obtain the residual echo signal 505 and input the residual echo signal 505 into the residual echo suppression module 55.

[0244] The residual echo suppression module 55 is used to perform residual echo processing on the first residual signal by using spectral subtraction after receiving the first residual signal 504 and the residual echo signal 505 to obtain a second speech signal 506 .

[0245] Exemplarily, in a possible implementation, the residual echo suppression module 55 is further configured to receive a noise signal corresponding to the first residual signal 504 sent by the noise reduction module, and perform noise reduction and residual echo processing on the first residual signal using a spectral subtraction method to obtain a second speech signal 506. For details, please refer to other relevant descriptions herein, such as the above-mentioned step S208, which will not be described in detail here.

[0246] The acoustic echo processing device provided by an embodiment of the present invention will be introduced below.

[0247] See also Figure 6 , which is a schematic diagram of the structure of an acoustic echo processing device according to an embodiment of the present invention. Figure 6 As shown, the acoustic echo processing device of the embodiment of the present invention may include:

[0248] A first processing unit 601 is configured to obtain a first correlation coefficient between a first residual signal and a first echo signal, where the first residual signal and the first echo signal correspond to a first speech signal;

[0249] The first processing unit 601 is further configured to determine a second correlation coefficient based on the first correlation coefficient and the attenuation coefficient, wherein the second correlation coefficient is less than or equal to the first correlation coefficient;

[0250] The second processing unit 602 is configured to perform residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal.

[0251] Exemplarily, the first processing unit 601 may include an acquisition unit 6011 and a determination unit 6012, wherein the acquisition unit 6011 is used to acquire a first correlation coefficient between a first residual signal and a first echo signal, and the first residual signal and the first echo signal correspond to a first speech signal; and the determination unit 6012 is used to determine a second correlation coefficient based on the first correlation coefficient and the attenuation coefficient, and the second correlation coefficient is less than or equal to the first correlation coefficient.

[0252] In a possible implementation, the first processing unit 601 is specifically configured to determine the second correlation coefficient based on the product of the first correlation coefficient and the attenuation coefficient, where the attenuation coefficient is greater than 0 and less than or equal to 1.

[0253] The meanings of the first speech signal, the first residual signal, the first echo signal, the first correlation coefficient, and the second correlation coefficient can also be referred to other relevant descriptions in this document and will not be repeated here.

[0254] It should be noted that the specific implementation process can be found in Figures 1 to 4 The specific description of the method embodiment shown is not repeated here.

[0255] Exemplarily, the first processing unit 601 may be used to execute Figure 5 The tasks of the adaptive filter (AF) 52, the subtractor 53, and the residual echo estimation module 54 shown in the figure; the second processing unit 602 can be used to perform Figure 5 The task of the residual echo suppression module 55 is shown.

[0256] In the embodiment of the present application, the acoustic echo processing device may be a functional module in the first terminal, or the acoustic echo processing device may be the first terminal, and a voice communication connection may be established between the acoustic echo processing device and the second terminal.

[0257] In one possible implementation, Figure 6In the acoustic echo processing device shown, the first processing unit 601 and the second processing unit 602 can be implemented with one or more processors. In addition, the acoustic echo processing device can also include a receiver and a transmitter. The receiver can be used to receive the first voice signal sent by the second terminal based on the voice communication connection, and the transmitter can send the second voice signal after the acoustic echo processing and noise reduction processing to the second terminal based on the communication connection. In the embodiment of the present application, the processor and the transceiver can be coupled, etc., and the embodiment of the present application does not limit the connection method between the processor and the transceiver.

[0258] like Figure 7 As shown, the acoustic echo processing device 70 includes one or more processors 720 and a transceiver 710 .

[0259] Exemplarily, when the acoustic echo processing device 70 conducts a voice call with the second terminal, the transceiver 710 is configured to receive a first voice signal from the second terminal.

[0260] Processor 720 is used to determine a first correlation coefficient between the first echo signal and the first residual signal according to the first speech signal, determine a second correlation coefficient based on the first correlation coefficient and the attenuation coefficient, and perform residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal.

[0261] For the meanings of the first speech signal, the first residual signal, the first echo signal, the first correlation coefficient, and the second correlation coefficient, please refer to other relevant descriptions in this document and will not be elaborated here.

[0262] It should be noted that the specific implementation process can be found in Figures 1 to 4 The specific description of the illustrated embodiment will not be repeated here.

[0263] exist Figure 7 In various implementations of the acoustic echo processing apparatus shown, the transceiver may include a receiver and a transmitter, wherein the receiver is used to perform a receiving function (or operation) and the transmitter is used to perform a transmitting function (or operation), and the transceiver is used to communicate with other devices / apparatuses through a transmission medium.

[0264] Optionally, the acoustic echo processing device 70 may further include one or more memories 730 for storing program instructions and / or data, etc. The memory 730 is coupled to the processor 720. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 720 may operate in coordination with the memory 730. The processor 720 may execute program instructions stored in the memory 730. Optionally, at least one of the one or more memories may be included in the processor.

[0265] The specific connection medium between the transceiver 710, the processor 720 and the memory 730 is not limited in the embodiment of the present application. Figure 7 The memory 730, the processor 720 and the transceiver 710 are connected via a bus 740. Figure 7 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0266] In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor, etc.

[0267] In the embodiment of the present application, the memory may include, but is not limited to, non-volatile memory such as a hard disk drive (HDD) or a solid-state drive (SSD), a random access memory (RAM), an erasable programmable read-only memory (EPROM), a read-only memory (ROM) or a portable read-only memory (CD-ROM), etc. The memory is any storage medium that can be used to carry or store program codes in the form of instructions or data structures and can be read and / or written by a computer (such as the acoustic echo processing device shown in the present application), but is not limited thereto. The memory in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0268] Exemplarily, the processor 720 is mainly used to process the communication protocol and communication data, and to control the entire acoustic echo processing device, execute the software program, and process the data of the software program. The memory 730 is mainly used to store the software program and data. The transceiver 710 may include a control circuit and an antenna. The control circuit is mainly used to convert the baseband signal and the radio frequency signal and process the radio frequency signal. The antenna is mainly used to transmit and receive radio frequency signals in the form of electromagnetic waves.

[0269] When the acoustic echo processing device is turned on, the processor 720 can read the software program in the memory 730, interpret and execute the instructions of the software program, and process the data of the software program. When data needs to be sent wirelessly, the processor 720 performs baseband processing on the data to be sent, and outputs the baseband signal to the RF circuit. The RF circuit performs RF processing on the baseband signal and then sends the RF signal outward in the form of electromagnetic waves through the antenna. When data is sent to the acoustic echo processing device, the RF circuit receives the RF signal through the antenna, converts the RF signal into a baseband signal, and outputs the baseband signal to the processor 720. The processor 720 converts the baseband signal into data and processes the data.

[0270] In another implementation, the RF circuit and antenna may be arranged independently of the processor performing baseband processing. For example, in a distributed scenario, the RF circuit and antenna may be arranged remotely from the acoustic echo processing device.

[0271] It is understandable that the acoustic echo processing device shown in the embodiment of the present application may also have Figure 7The embodiments of the present application do not limit the number of components and the like. The methods executed by the processor and the transceiver shown above are only examples, and the specific steps executed by the processor and the transceiver can refer to the methods described above.

[0272] Exemplarily, the transceiver 710 is used to obtain Figure 5 The first reference signal 501 shown, the processor 720, can be used to perform Figure 5 The tasks of the adaptive filter (AF) 52, the subtractor 53, the residual echo estimation module 54, and the residual echo suppression module 55 are shown.

[0273] like Figure 8 As shown, Figure 8 800 can execute the relevant steps of the acoustic echo processing device in the aforementioned method embodiment, and the module device 800 includes: a communication module 801, a power module 802, a storage module 803 and a chip module 804. Among them, the power module 802 is used to provide power to the module device; the storage module 803 is used to store data and instructions; the communication module 801 is used for internal communication of the module device, or for the module device to communicate with external devices; the chip module 804 can execute the above Figures 1 to 4 The method shown, and the steps performed by the relevant implementation method.

[0274] Understandably, for the specific description of the chip module, you can also refer to Figure 6 or Figure 7 etc., which will not be elaborated here.

[0275] Exemplarily, the communication module 801 is used to obtain Figure 5 As shown in the first reference signal 501, the chip module 804 can be used to perform Figure 5 The tasks of the adaptive filter (AF) 52, the subtractor 53, the residual echo estimation module 54, and the residual echo suppression module 55 are shown.

[0276] In addition, the present application also provides a computer program, which is used to implement the method provided by the present application.

[0277] The present application also provides a computer-readable storage medium, in which a computer code is stored. When the computer code is executed on a computer, the computer executes the method provided by the present application.

[0278] The present application also provides a computer program product, which includes a computer code or a computer program. When the computer code or the computer program runs on a computer, the method provided by the present application is executed.

[0279] The present application also provides a chip system, which includes one or more processors, and the processors are used to call computer instructions so that the method provided by the present application is executed.

[0280] The computer-readable storage medium may be an internal storage unit of the acoustic echo processing device described in any of the aforementioned embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the acoustic echo processing device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the device. Further, the computer-readable storage medium may also include both an internal storage unit of the acoustic echo processing device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the acoustic echo processing device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0281] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means.

[0282] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0283] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0284] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0285] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist independently, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0286] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, etc.) to perform some steps of the method described in each embodiment of the present invention.

[0287] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0288] What is disclosed above is only a preferred embodiment of the present application, and it certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiment and equivalent changes made according to the claims of the present application are still within the scope covered by the application.

Claims

1. An acoustic echo processing method, characterized in that: The method comprises: Acquire a first correlation coefficient between a first residual signal and a first echo signal, wherein the first residual signal and the first echo signal correspond to a first speech signal; Determining a second correlation coefficient based on the first correlation coefficient and the attenuation coefficient, the second correlation coefficient is less than or equal to the first correlation coefficient; wherein, when the speech scene corresponding to the first speech signal belongs to the first scene, the attenuation coefficient is greater than 0 and less than 1; whether the speech scene corresponding to the first speech signal belongs to the first scene is determined based on a first value, the first value is determined based on the energy of the first echo signal, the energy of the first residual signal, and a first constant; Perform residual echo processing on the first residual signal based on the second correlation coefficient to obtain a second speech signal.

2. The method according to claim 1, characterized in that Determining the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient includes: The second correlation coefficient is determined based on a product of the first correlation coefficient and the attenuation coefficient.

3. The method according to claim 1, characterized in that When the first value is less than the first threshold, the voice scene corresponding to the first voice signal belongs to the first scene.

4. The method according to claim 1, characterized in that: When the second numerical value corresponding to the first voice signal is less than the second threshold, the voice scene corresponding to the first voice signal belongs to the first scene, the second numerical value corresponding to the first voice signal is determined based on the first numerical value, the first voice signal, the second numerical value corresponding to the second voice signal, and the smoothing coefficient, and the second voice signal and the first voice signal belong to two adjacent frames of voice signals in the same segment of voice signals.

5. The method according to claim 4, characterized in that The second value corresponding to the first speech signal satisfies the following formula: ； Among them, the is a second value determined according to the parameter of the first speech signal, is a second value determined according to a parameter of the second speech signal, the first speech signal and the second speech signal respectively belong to the nth frame signal and the n-1th frame signal in the same segment of speech signal, the n is greater than or equal to 2, the is a first value, the first value is determined based on the energy of the first echo signal, the energy of the first residual signal, and a first constant, is the smoothing coefficient.

6. The method according to claim 5, characterized in that When the value of n is 2, The value is , is the product of the ratio of the energy of the second echo signal to the energy of the second residual signal and the first constant, and the second echo signal and the second residual signal correspond to the second speech signal.

7. The method according to any one of claims 1 to 6, characterized in that: When the amplitude of the target noise signal is greater than the third threshold, the speech scene corresponding to the first speech signal belongs to the first scene, and the target noise signal is used to indicate the noise intensity of the first speech signal.

8. The method according to claim 7, characterized in that The target noise signal is a noise signal obtained by performing noise estimation on the first speech signal; or, The target noise signal is a noise signal obtained by performing noise estimation on a second speech signal, wherein the second speech signal and the first speech signal belong to two adjacent frames of speech signals in the same segment of speech signals; or, The target noise signal is a noise signal obtained by performing noise estimation on the first residual signal; or, The target noise signal is a noise signal obtained by performing noise estimation on the second residual signal, the second residual signal corresponds to the second speech signal, and the second speech signal and the first speech signal belong to two adjacent frames of speech signals in the same segment of speech signals.

9. The method according to claim 1, characterized in that: When the speech scene corresponding to the first speech signal belongs to the second scene, the attenuation coefficient is 1; Determining the second correlation coefficient based on the first correlation coefficient and the attenuation coefficient includes: The second correlation coefficient is determined based on a product of the first correlation coefficient and the attenuation coefficient.

10. The method according to any one of claims 1 to 6, characterized in that: The first scene is a noise scene or a double-talk scene.

11. An acoustic echo processing device, characterized in that: The apparatus comprises means for executing the method according to any one of claims 1 to 10.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, where the computer program codes include computer instructions. The one or more processors call the computer instructions so that the method according to any one of claims 1 to 10 is executed.

13. A chip system, characterized in that: The chip system includes one or more processors, and the processor is used to call computer instructions so that the method according to any one of claims 1 to 10 is executed.

14. A module device, characterized in that: The module device includes a communication module, a power module, a storage module and a chip module, wherein: The power module is used to provide electrical energy to the module device; The storage module is used to store data and instructions; The communication module is used for internal communication of the module device, or for communication between the module device and an external device; The chip module is used to execute the method according to any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions, and when the instructions are executed, the method of any one of claims 1 to 10 is performed.

Images