Echo cancellation method and related device, equipment and storage medium
By acquiring and detecting the signal of the call device, and sensing the vibration of the diaphragm with an acceleration sensor, the effective echo is achieved without adding additional components, and the cost and function caused by adding components in the prior art is solved. Consumption problem.
Patent Information
- Application Number
- CN202111498803.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-09
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-12-09
AI Technical Summary
The existing echo cancellation technology requires the addition of additional components, resulting in increased hardware cost and power consumption, and there are uncertainties that cannot effectively achieve echo cancellation without adding additional components.
By obtaining the near-end signals, far-end signals and sensing signals of the call device, detecting the call status, and echo cancellation based on these signals, using the built-in acceleration sensor to sense the diaphragm vibration of the speaker circuit, and indirectly obtaining Nonlinear echo components, thereby improving echo cancellation effect.
Without adding additional components, the original components of the call device are used to achieve echo cancellation, which improves the echo cancellation effect and avoids the increase in hardware costs and power consumption and the emergence of uncertain factors.
Smart Images

Figure CN114299979B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to an echo cancellation method and related devices, equipment and storage media. Background Art
[0002] With the rapid development of electronic information technology, audio calls have been more and more widely used in many scenarios such as remote conferences and instant messaging. During a call, the far-end signal played by the speaker circuit is picked up by the pickup circuit and sent back to the far end, forming an echo. If the echo is too loud, it will affect the normal call. Therefore, echo cancellation technology has become one of the indispensable technologies in audio calls.
[0003] At present, the existing echo cancellation technology mainly adds additional components to the original circuit to assist in echo cancellation in order to improve the echo cancellation effect. However, adding additional components often brings new uncertainties on the one hand, and also increases additional hardware costs and power consumption on the other hand. In view of this, how to achieve echo cancellation using the original components of the call device without adding additional components has become an urgent problem to be solved. Summary of the invention
[0004] The main technical problem solved by the present application is to provide an echo cancellation method and related devices, equipment and storage media, which can achieve echo cancellation by using the original components of the communication device without adding additional components.
[0005] In order to solve the above technical problems, the first aspect of the present application provides an echo cancellation method, including: obtaining a near-end signal, a far-end signal and a sensing signal of an acceleration sensor; wherein the near-end signal is collected by a pickup circuit of a call device, and the far-end signal is played by a speaker circuit of the call device, and the acceleration sensor, the pickup circuit and the speaker circuit belong to the same call device; detecting the near-end signal to obtain a call status; wherein the call status includes any one of the following: a near-end single-talk status, a far-end single-talk status, and a near-end and far-end dual-talk status; based on the call status, using the far-end signal, the sensing signal and the near-end signal to perform echo cancellation to obtain an optimized signal.
[0006] In order to solve the above technical problems, the second aspect of the present application provides an echo cancellation device, including: a signal acquisition module, a state detection module and an echo cancellation module, the signal acquisition module is used to obtain a near-end signal, a far-end signal and a sensing signal of an acceleration sensor; wherein the near-end signal is collected by a pickup circuit of a call device, and the far-end signal is played by a speaker circuit of the call device, and the acceleration sensor, the pickup circuit and the speaker circuit belong to the same call device; the state detection module is used to detect the near-end signal to obtain a call state; wherein the call state includes any one of the following: a near-end single-talk state, a far-end single-talk state, and a far-end and near-end dual-talk state; the echo cancellation module is used to perform echo cancellation based on the call state using the far-end signal, the sensing signal and the near-end signal to obtain an optimized signal.
[0007] In order to solve the above technical problems, the third aspect of the present application provides a call device, including a speaker circuit, a pickup circuit, a communication circuit, an acceleration sensor, a memory and a processor. The speaker circuit, the pickup circuit, the communication circuit, the acceleration sensor and the memory are coupled to the processor, and program instructions are stored in the memory. The processor is used to execute the program instructions to implement the echo elimination method in the above first aspect.
[0008] In order to solve the above technical problem, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the echo cancellation method in the above first aspect.
[0009] The above scheme obtains the near-end signal, the far-end signal and the sensing signal of the acceleration sensor, and the near-end signal is collected by the pickup circuit of the call device, and the far-end signal is played by the speaker circuit of the call device. The acceleration sensor, the pickup circuit and the speaker circuit belong to the call device, and the near-end signal is detected to obtain the call state, and the call state includes any of the following: near-end single-talk state, far-end single-talk state, and far-end and near-end dual-talk state. On this basis, based on the call state, the far-end signal, the sensing signal and the near-end signal are used to perform echo cancellation to obtain the optimized signal. On the one hand, in the echo cancellation process, the vibration of the diaphragm when the speaker circuit is playing can be sensed by referring to the acceleration sensor, which is helpful to indirectly obtain the nonlinear echo component, thereby improving the echo cancellation effect. On the other hand, the acceleration sensor is provided by the call device, so there is no need to add additional components, so as not to bring new uncertainties, and will not increase additional hardware costs and power consumption. Therefore, echo cancellation can be achieved by using the original components of the call device without adding additional components. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 It is a flowchart of an embodiment of the echo cancellation method of the present application;
[0011] Figure 2 It is a schematic diagram of the framework of an embodiment of the echo cancellation device of the present application;
[0012] Figure 3 It is a schematic diagram of the framework of an embodiment of a communication device;
[0013] Figure 4 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0014] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0015] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0016] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.
[0017] See also Figure 1 , Figure 1 It is a flowchart of an embodiment of the echo cancellation method of the present application.
[0018] Specifically, the following steps may be included:
[0019] Step S11: Acquire a near-end signal, a far-end signal, and a sensing signal of an acceleration sensor.
[0020] In the embodiment of the present disclosure, the near-end signal is collected by the pickup circuit of the call device, the far-end signal is played by the speaker circuit of the call device, and the acceleration sensor, the pickup circuit, and the speaker circuit all belong to the call device. For ease of description, the near-end signal can be denoted as d(n), the far-end signal as r(n), and the sensing signal as v(n).
[0021] In one implementation scenario, it should be noted that the acceleration sensor, the pickup circuit, and the speaker circuit all belong to the call device, that is, the acceleration sensor in the embodiment of the present disclosure is provided by the call device, and the call device may include but is not limited to: mobile phones, tablet computers, laptop computers, etc., which are not limited here. In order to sense the vibration of the diaphragm, the acceleration sensor is generally set on the diaphragm of the speaker circuit when the acceleration sensor is added to the call device, and usually only senses the acceleration signal of the diaphragm of the speaker circuit (which points to the vibration direction of the diaphragm of the speaker circuit). Therefore, the acceleration sensor added to the call device is essentially not a call device like the pickup circuit and the speaker circuit, but an additional electronic component attached to the call device, which is added to the call device without sensing the vibration of the diaphragm. Different from the above-mentioned method, the acceleration sensor in the embodiment of the present disclosure can capture the acceleration information of the target position in the first direction, the second direction and the third direction respectively, so as to obtain a sensing signal based on the acceleration information in at least one of the above three directions, and the target position represents the installation position of the acceleration sensor in the call device, and the first direction, the second direction and the third direction are perpendicular to each other, that is, the acceleration information sensed by the acceleration sensor of the call device can not only sense the vibration of the diaphragm in various directions, but also sense the vibration of other components / parts in the call device caused by the vibration of the diaphragm, that is, the sensing signal of the built-in acceleration sensor is also richer, and the above-mentioned vibrations will more or less cause echoes, so using the acceleration sensor that belongs to the call device together with the pickup circuit and the speaker circuit to obtain the sensing signal is also helpful to more effectively assist echo cancellation. In the above-mentioned method, the acceleration sensor captures the acceleration information of the target position in at least one of the first direction, the second direction and the third direction respectively to obtain the sensing signal, and the target position represents the installation position of the acceleration sensor in the call device, and the first direction, the second direction and the third direction are perpendicular to each other, so as to improve the accuracy of the sensing signal as much as possible.
[0022] In a specific implementation scenario, taking the call device as a mobile terminal such as a mobile phone or a tablet computer as an example, the first direction may be a direction perpendicular to the screen of the mobile terminal, and the second direction and the third direction may be directions of two adjacent edges of the mobile terminal. Other situations may be deduced by analogy, and examples are not given one by one here. For the convenience of description, the acceleration information in the first direction may be recorded as v z (n), and the acceleration information in the second direction is recorded as v x (n), the acceleration information in the third direction is recorded as v y (n).
[0023] In a specific implementation scenario, a sensing signal can be obtained based on acceleration information in any one of the first direction, the second direction, and the third direction, or based on acceleration information in any two of the first direction, the second direction, and the third direction, or based on acceleration information in the first direction, the second direction, and the third direction. Exemplarily, the acceleration information with the largest energy in the above three directions can be selected, and the acceleration information in this direction can be used as the sensing signal; or, the acceleration information in the above three directions can be averaged, and the averaged acceleration information can be used as the sensing signal. Of course, the acceleration information in at least one direction can also be arbitrarily selected as the sensing information, which is not limited here.
[0024] Step S12: Detect the near-end signal to obtain the call status.
[0025] In the disclosed embodiment, the call state includes any of the following: near-end single talk state, far-end single talk state, and near-end and far-end dual talk state. It should be noted that the near-end single talk state means that only the near-end speaker at the call device is currently speaking, while the far-end speaker is not speaking. At this time, the pickup circuit of the call device only picks up the near-end speaker's voice and the noise of the environment in which the call device is located; the far-end single talk state means that only the far-end speaker is currently speaking, while the near-end speaker at the call device is not speaking. At this time, the pickup circuit of the call device only picks up the far-end speaker's voice broadcast by the speaker circuit and the noise of the environment in which the call device is located; and the near-end and far-end dual talk state means that not only the near-end speaker at the call device is currently speaking, but also the far-end speaker is speaking. Therefore, at this time, the pickup circuit of the call device can not only pick up the far-end speaker's voice broadcast by the speaker, but also the near-end speaker's voice and the noise of the environment in which the call device is located. In addition, generally speaking, in real scenarios, in normal call conditions, most of the time is in the near-end single talk state or the far-end single talk state, and generally only a small part of the time is in the far-end dual talk state. Of course, the embodiments of the present disclosure do not limit the length of the time periods occupied by the above three states.
[0026] In one implementation scenario, the above-mentioned call status can be detected by double-talk detection (DTD), and the double-talk detection can specifically include but is not limited to: energy comparison method, correlation comparison method, probability statistics-based method, etc., which are not limited here. Exemplarily, the energy comparison method is based on the assumption that the near-end signal during double-talk is much higher than the far-end signal during single-talk, and the correlation comparison method is based on the assumption that the correlation between the near-end signal and the far-end signal during double-talk is lower than that of single-talk. For the specific process of double-talk detection, please refer to the technical details of double-talk detection, which will not be repeated here.
[0027] Step S13: Based on the call status, the far-end signal, the sensing signal, and the near-end signal are used to perform echo cancellation to obtain an optimized signal.
[0028] Specifically, echo cancellation can be achieved through an adaptive filter. For example, based on the call status, the target filter and target signal required for echo cancellation can be determined, and the target filter includes at least one of the first filter, the second filter, and the third filter. The target signal includes at least the near-end signal, and the target filter is used to process the target signal to obtain an optimized signal. In the above method, the target filter is selected from the first filter, the second filter, and the third filter according to the call status, and the target signal is selected from the near-end signal, the far-end signal, and the sensing signal, so that the template filter is used to process the target signal to obtain an optimized signal, which is conducive to adaptively performing echo cancellation for different call states and improving the echo cancellation effect.
[0029] In an implementation scenario, during the first call, the first filter, the second filter, and the third filter can be initialized first, and the first filter is used to eliminate the nonlinear echo component in the sensing signal, the second filter is used to eliminate the linear echo component in the near-end signal, and the third filter is used to eliminate the nonlinear echo component in the near-end signal. On this basis, the above-mentioned steps of determining the target filter and target signal required for echo cancellation based on the call status and subsequent steps are performed. For the convenience of initialization, the filter coefficients of the above three filters can all be set to 0 during the initialization stage. Of course, they can also be set to other values as needed, which are not limited here.
[0030] In an implementation scenario, the first filter, the second filter, and the third filter can all be designed based on FIR (Finite Impulse Response) filters. It should be noted that the implementation forms of the above three filters may include but are not limited to: direct type, second-order cascade type, and Lattice structure, and the embodiments of the present disclosure do not limit the transfer functions of the above three filters. For the specific implementation of the filter, please refer to the relevant technical details about the FIR filter, which is not limited here.
[0031] In one implementation scenario, in response to the call state being the far-end single-talk state, it can be determined that the target filter includes the first filter, the second filter, and the third filter, and it can be determined that the target signal includes the near-end signal, the far-end signal, and the sensing signal. On this basis, the far-end signal d(n) can be further used as the first reference signal of the first filter, and the first expected signal of the first filter can be obtained based on the sensing signal. In this case, the first filter can be filtered based on the first reference signal and the first expected signal, so that the first error signal e1(n) output by the first filter can be obtained, and the first error signal e1(n) represents the nonlinear echo component in the sensing signal. After that, the far-end signal d(n) can continue to be used as the second reference signal, and the near-end signal r(n) can be used as the second expected signal of the second filter. In this case, the second filter can be filtered based on the second reference signal and the second expected signal, so that the second error signal e2(n) output by the second filter can be obtained, and the second error signal e2(n) represents the near-end signal after the linear echo component is eliminated. Finally, the first error signal e1(n) can be used as the third reference signal of the third filter, and the second error signal e2(n) can be used as the third expected signal of the third filter. In this case, the third filter can perform filtering based on the third reference signal and the third expected signal, so that the third error signal e3(n) output by the third filter can be obtained as an optimization signal, and the third error signal e3(n) represents the near-end signal after eliminating the linear echo component and the nonlinear echo component. In the above method, the far-end signal is first used as a reference to eliminate the nonlinear echo component in the sensing signal through the first filter, and the far-end signal is used as a reference signal to eliminate the linear echo component in the near-end signal through the second filter. On this basis, the above nonlinear echo component is used as a reference signal, and the nonlinear echo component in the near-end signal is further eliminated through the third filter. The linear echo component and the nonlinear echo component in the near-end signal are eliminated through the far-end signal and the sensing signal of the acceleration sensor, which helps to improve the echo cancellation effect.
[0032] In a specific implementation scenario, in order to further improve the echo cancellation effect, the sensing signal can be upsampled to a target sampling rate, and the target sampling rate is the sampling rate of the near-end signal. For example, the near-end signal contains 100 sampling data per second, and the sensing signal contains 10 sampling data per second, then the sensing signal can be upsampled, and the sensing signal after upsampling also contains 100 sampling data per second. Other situations can be deduced by analogy, and examples are not given one by one here. In addition, the sensing signal can be upsampled specifically by a difference algorithm such as trilinear difference. The specific process of upsampling can refer to the technical details of the difference algorithm such as trilinear difference, which will not be repeated here. In the above method, by upsampling the sensing signal to the target sampling rate, and the target sampling rate is the sampling rate of the near-end signal, the sampling rate of the sensing signal can be kept consistent with that of the near-end signal through upsampling, which is conducive to further improving the echo cancellation effect.
[0033] In a specific implementation scenario, in order to further improve the filtering performance of the first filter, when the call state is the far-end single-talk state, after echo elimination is performed through the first filter, the first expected signal and the first error signal can be further used to update the first filter. Exemplarily, the coefficients of the first filter can be updated by algorithms such as LMS (Least Mean Squares) and RLS (Recursive Least Squares), which are not limited here. In addition, the specific process of filter updating can refer to technical details such as LMS and RLS, which will not be repeated here. The above method, after obtaining the first error signal through the first filter, further uses the first expected signal and the first error signal to update the first filter, which is conducive to further continuously improving the filtering performance of the first filter during the echo elimination process.
[0034] In a specific implementation scenario, in order to further improve the filtering performance of the second filter, when the call state is the far-end single talk state, after the echo is eliminated through the second filter, the second expected signal and the second error signal can be further used to update the second filter. Exemplarily, the coefficients of the second filter can be updated by algorithms such as LMS and RLS, which are not limited here. In the above manner, after obtaining the second error signal through the second filter, the second expected signal and the second error signal are further used to update the second filter, which is conducive to further continuously improving the filtering performance of the second filter during the echo elimination process.
[0035] In a specific implementation scenario, in order to further improve the filtering performance of the third filter, when the call state is the far-end single talk state, after the echo is eliminated through the third filter, the third reference signal and the third error signal can be further used to update the third filter. Exemplarily, the coefficients of the third filter can be updated by algorithms such as LMS, RLS, etc., which are not limited here. In the above manner, after obtaining the third error signal through the third filter, the third reference signal and the third error signal are further used to update the third filter, which is conducive to further continuously improving the filtering performance of the third filter during the echo elimination process.
[0036] In one implementation scenario, in response to the call state being a far-end and near-end dual-talk state, it can be determined that the target filter includes a second filter, and that the target signal includes a near-end signal d(n) and a far-end signal r(n). On this basis, the far-end signal r(n) can be further used as a fourth reference signal of the second filter, and the near-end signal d(n) can be used as a fourth expected signal of the second filter. In this case, the second filter can perform filtering processing based on the fourth reference signal and the fourth expected signal, so that the fourth error signal e4(n) output by the second filter can be obtained as an optimized signal, and the fourth error signal e4(n) represents the near-end signal after eliminating the linear echo component. In the above method, when the call state is the far-end and near-end dual-talk state, it is determined that the target filter includes the second filter, and the target signal includes the near-end signal and the far-end signal. On this basis, the far-end signal is used as the fourth reference signal of the second filter, and the near-end signal is used as the fourth expected signal of the second filter, and the fourth error signal output by the second filter is obtained as the optimization signal. The fourth error signal represents the near-end signal after eliminating the linear echo component, and the third filter is no longer used for further echo elimination. This can reduce the possibility of affecting normal calls due to attempts to further eliminate nonlinear echo components, and help improve audio call effects.
[0037] In a specific implementation scenario, it should be noted that in the embodiment of the present disclosure, during the echo cancellation process, the above-mentioned first filter, second filter and third filter can be maintained, and the second filter used when the call status is the far-end single-talk state and the second filter used when the call status is the far-end and near-end dual-talk state are the same filter.
[0038] In a specific implementation scenario, different from the aforementioned situation where the call state is the far-end single talk state, when the call state is the far-end and near-end dual talk state, after the echo cancellation is performed through the second filter, there is no need to update the target filter (i.e., the second filter). In the above manner, the target filter is not updated in the far-end and near-end dual talk state. On the one hand, it can avoid the influence of the introduction of nonlinear echo factors on the performance of the target filter, so that the existing performance of the target filter can be at least maintained without reducing its original performance in the far-end and near-end dual talk state. On the other hand, it also helps to reduce the amount of calculation in the echo cancellation process, which helps to reduce the calculation load of the echo cancellation.
[0039] In one implementation scenario, in response to the call state being a near-end single-talk state, it can be determined that no filter needs to be selected as a target filter, and it can be determined that the target signal includes the near-end signal. On this basis, the near-end signal can be directly used as an optimization signal. That is, when the call state is a near-end single-talk state, since the near-end signal collected by the pickup circuit includes the voice of the near-end speaker at the call device end and the noise of the environment in which the call device is located, but does not include the voice of the far-end speaker, that is, when the call state is a near-end single-talk state, no echo will be formed. Therefore, when the call state is a near-end single-talk state, it is determined that no filter needs to be selected as a target filter, and it is determined that the target signal includes the near-end signal, and the near-end signal is directly used as an optimization signal, which can greatly reduce the calculation load when the call state is a near-end single-talk state, and help improve the efficiency of echo cancellation.
[0040] In addition, after the near-end signal d(n), far-end signal r(n) and sensing signal v(n) are collected at time step n, the above-mentioned filter can be combined to perform echo elimination to obtain an optimized signal. Furthermore, after the near-end signal d(n+1), far-end signal r(n+1) and sensing signal v(n+1) are collected at the next time step n+1, the filter updated at time step n can be combined to perform echo elimination to obtain an optimized signal. The subsequent time steps can be deduced by analogy to perform echo elimination. Therefore, it is beneficial to continuously improve the call quality through adaptive filters during audio calls.
[0041] Of course, different from the implementation method of mixing filters in different call states, different filter groups can also be designed for each call state. Exemplarily, corresponding to the far-end single-talk state, a first filter group can be set, and the first filter group can specifically include: a first filter, a second filter and a third filter, and the first filter is used to eliminate the nonlinear echo component in the sensing signal, the second filter is used to eliminate the linear echo component in the near-end signal, and the third filter is used to eliminate the nonlinear echo component in the near-end signal. For details, please refer to the relevant description in the above-mentioned public embodiment, which will not be repeated here; similarly, corresponding to the far-end and near-end double-talk state, a second filter group can be set, and the second filter group can specifically include: a fourth filter, and the fourth filter is used to eliminate the linear echo component in the near-end signal; similarly, corresponding to the near-end single-talk state, a third filter group can be set, and the third filter group can include: a fifth filter, and exemplarily, the fifth filter can be a bandpass filter, which is used to filter out components outside the preset frequency range in the near-end signal, such as components that are significantly higher than the normal voice frequency of the speaker, or components that are significantly lower than the normal voice frequency of the speaker. Of course, the fifth filter can also be an all-pass filter, which is not limited here. On this basis, the target filter group and target signal required for echo cancellation can be determined based on the call state, and the target filter group includes any one of the first filter group, the second filter group, and the third filter group, and the target signal can at least include the near-end signal, so that the target filter group can be used to process the target signal to obtain an optimized signal. In addition, as mentioned above, when the call state is a far-end single-talk state, after the echo cancellation operation is performed using the first filter, the second filter, and the third filter in the first filter group, the filter can be further updated, and when the call state is a far-end double-talk state, after the echo cancellation operation is performed using the fourth filter in the second filter group, the filter update operation may not be performed, and the details can be referred to the above-mentioned related description, which will not be repeated here. In addition, different from the above-mentioned embodiment, when the call state is a near-end single-talk state, after the echo cancellation operation is performed using the fifth filter in the third filter group, the filter update operation may not be performed.
[0042] The above scheme obtains the near-end signal, the far-end signal and the sensing signal of the acceleration sensor, and the near-end signal is collected by the pickup circuit of the call device, and the far-end signal is played by the speaker circuit of the call device. The acceleration sensor, the pickup circuit and the speaker circuit belong to the call device, and the near-end signal is detected to obtain the call state, and the call state includes any of the following: near-end single-talk state, far-end single-talk state, and far-end and near-end dual-talk state. On this basis, based on the call state, the far-end signal, the sensing signal and the near-end signal are used to perform echo cancellation to obtain the optimized signal. On the one hand, in the echo cancellation process, the vibration of the diaphragm when the speaker circuit is playing can be sensed by referring to the acceleration sensor, which is helpful to indirectly obtain the nonlinear echo component, thereby improving the echo cancellation effect. On the other hand, the acceleration sensor is provided by the call device, so there is no need to add additional components, so as not to bring new uncertainties, and will not increase additional hardware costs and power consumption. Therefore, echo cancellation can be achieved by using the original components of the call device without adding additional components.
[0043] See also Figure 2 , Figure 2 It is a schematic diagram of the framework of an embodiment of an echo cancellation device 20 of the present application. The echo cancellation device 20 includes: a signal acquisition module 21, a state detection module 22 and an echo cancellation module 23. The signal acquisition module 21 is used to acquire a near-end signal, a far-end signal and a sensing signal of an acceleration sensor; wherein the near-end signal is collected by a pickup circuit of a call device, and the far-end signal is played by a speaker circuit of the call device, and the acceleration sensor, the pickup circuit and the speaker circuit all belong to the call device; the state detection module 22 is used to detect the near-end signal to obtain a call state; wherein the call state includes any one of the following: a near-end single-talk state, a far-end single-talk state, and a far-end and near-end dual-talk state; the echo cancellation module 23 is used to perform echo cancellation based on the call state using the far-end signal, the sensing signal and the near-end signal to obtain an optimized signal.
[0044] The above scheme obtains the near-end signal, the far-end signal and the sensing signal of the acceleration sensor, and the near-end signal is collected by the pickup circuit of the call device, and the far-end signal is played by the speaker circuit of the call device. The acceleration sensor, the pickup circuit and the speaker circuit belong to the call device, and the near-end signal is detected to obtain the call state, and the call state includes any of the following: near-end single-talk state, far-end single-talk state, and far-end and near-end dual-talk state. On this basis, based on the call state, the far-end signal, the sensing signal and the near-end signal are used to perform echo cancellation to obtain the optimized signal. On the one hand, in the echo cancellation process, the vibration of the diaphragm when the speaker circuit is playing can be sensed by referring to the acceleration sensor, which is helpful to indirectly obtain the nonlinear echo component, thereby improving the echo cancellation effect. On the other hand, the acceleration sensor is provided by the call device, so there is no need to add additional components, so as not to bring new uncertainties, and will not increase additional hardware costs and power consumption. Therefore, echo cancellation can be achieved by using the original components of the call device without adding additional components.
[0045] In some disclosed embodiments, the echo cancellation module 23 includes a determination submodule for determining a target filter and a target signal required to perform echo cancellation based on a call status; wherein the target filter includes at least one of a first filter, a second filter, and a third filter, and the target signal includes at least a near-end signal; the echo cancellation module 23 includes a processing submodule for processing the target signal using the target filter to obtain an optimized signal.
[0046] Therefore, the target filter is selected from the first filter, the second filter and the third filter according to the call status, and the target signal is selected from the near-end signal, the far-end signal and the sensing signal, so that the template filter is used to process the target signal to obtain an optimized signal, which is beneficial to adaptively perform echo cancellation for different call states and improve the echo cancellation effect.
[0047] In some disclosed embodiments, the determination submodule includes a first response unit, which is used to determine that the target filter includes a first filter, a second filter and a third filter in response to the call state being a far-end single-talk state, and determine that the target signal includes a near-end signal, a far-end signal and a sensing signal; the processing submodule includes a first elimination unit, which is used to use the far-end signal as a first reference signal of the first filter, obtain a first expected signal of the first filter based on the sensing signal, and obtain a first error signal output by the first filter; wherein the first error signal represents a nonlinear echo component in the sensing signal; the processing submodule includes a second elimination unit, which is used to use the far-end signal as a second reference signal of the second filter, and use the near-end signal as a second expected signal of the second filter, and obtain a second error signal output by the second filter; wherein the second error signal represents the near-end signal after eliminating the linear echo component; the processing submodule includes a third elimination unit, which is used to use the first error signal as a third reference signal of the third filter, and use the second error signal as a third expected signal of the third filter, and obtain a third error signal output by the third filter as an optimization signal; wherein the third error signal represents the near-end signal after eliminating the linear echo component and the nonlinear echo component.
[0048] Therefore, the far-end signal is first used as a reference to eliminate the nonlinear echo component in the sensing signal through the first filter, and the far-end signal is used as a reference signal to eliminate the linear echo component in the near-end signal through the second filter. On this basis, the above-mentioned nonlinear echo component is used as a reference signal and further eliminated through the third filter. The linear echo component and the nonlinear echo component in the near-end signal are eliminated through the far-end signal and the sensing signal of the acceleration sensor, which helps to improve the echo cancellation effect.
[0049] In some disclosed embodiments, the echo cancellation module 23 includes an update submodule for performing at least one of the following: updating the first filter using the first desired signal and the first error signal; updating the second filter using the second desired signal and the second error signal; updating the third filter using the third reference signal and the third error signal.
[0050] Therefore, after obtaining the first error signal through the first filter, the first expected signal and the first error signal are further used to update the first filter, which is beneficial to further continuously improving the filtering performance of the first filter during the echo elimination process; and after obtaining the second error signal through the second filter, the second expected signal and the second error signal are further used to update the second filter, which is beneficial to further continuously improving the filtering performance of the second filter during the echo elimination process; and after obtaining the third error signal through the third filter, the third reference signal and the third error signal are further used to update the third filter, which is beneficial to further continuously improving the filtering performance of the third filter during the echo elimination process.
[0051] In some disclosed embodiments, the first elimination unit is further specifically configured to upsample the sensing signal to a target sampling rate; wherein the target sampling rate is a sampling rate of the near-end signal.
[0052] Therefore, by upsampling the sensing signal to a target sampling rate, and the target sampling rate is the sampling rate of the near-end signal, the sampling rates of the sensing signal and the near-end signal can be kept consistent through upsampling, which is beneficial to further improve the echo cancellation effect.
[0053] In some disclosed embodiments, the determination submodule includes a second response unit, which is used to determine that the target filter includes a second filter in response to the call state being a near-end and far-end double-talk state, and to determine that the target signal includes a near-end signal and a far-end signal; the processing submodule includes a fourth elimination unit, which is used to use the far-end signal as a fourth reference signal of the second filter, and use the near-end signal as a fourth expected signal of the second filter, and obtain a fourth error signal output by the second filter as an optimization signal; wherein the fourth error signal represents the near-end signal after eliminating the linear echo component.
[0054] Therefore, when the call state is the far-end and near-end double-talk state, it is determined that the target filter includes the second filter, and it is determined that the target signal includes the near-end signal and the far-end signal. On this basis, the far-end signal is used as the fourth reference signal of the second filter, and the near-end signal is used as the fourth expected signal of the second filter, and the fourth error signal output by the second filter is obtained as the optimization signal. The fourth error signal represents the near-end signal after eliminating the linear echo component, and the third filter is no longer used for further echo elimination. This can reduce the possibility of affecting normal calls due to attempts to further eliminate nonlinear echo components, and help improve audio call effects.
[0055] In some disclosed embodiments, the determination submodule includes a third response unit, which is used to determine that there is no need to select any filter as the target filter in response to the call state being a near-end single-talk state, and to determine that the target signal includes a near-end signal; the processing submodule includes a fifth elimination unit, which is used to directly use the near-end signal as the optimized signal.
[0056] Therefore, when the call state is the near-end single-talk state, it is determined that there is no need to select any filter as the target filter, and the target signal is determined to include the near-end signal, and the near-end signal is directly used as the optimization signal. This can greatly reduce the calculation load when the call state is the near-end single-talk state, and help improve the echo cancellation efficiency.
[0057] In some disclosed embodiments, the echo elimination device 20 also includes: an initialization module, used to initialize the first filter, the second filter and the third filter; wherein the first filter is used to eliminate the nonlinear echo component in the sensing signal, the second filter is used to eliminate the linear echo component in the near-end signal, and the third filter is used to eliminate the nonlinear echo component in the near-end signal.
[0058] Therefore, before filtering, the first filter, the second filter, and the third filter are set first, and the coefficients are initialized respectively. The first filter is used to eliminate the nonlinear echo component in the sensing signal, the second filter is used to eliminate the linear echo component in the near-end signal, and the third filter is used to eliminate the nonlinear echo component in the near-end signal. Therefore, filters with various properties can be fully utilized to achieve echo cancellation.
[0059] In some disclosed embodiments, an acceleration sensor is used to capture acceleration information at a target position in a first direction, a second direction, and a third direction, respectively, and a sensing signal is obtained based on the acceleration information in at least one of the first direction, the second direction, and the third direction; wherein the target position represents an installation position of the acceleration sensor in the call device, and the first direction, the second direction, and the third direction are perpendicular to each other.
[0060] Therefore, the acceleration sensor captures the acceleration information of the target position in at least one of the first direction, the second direction and the third direction to obtain a sensing signal, and the target position represents the installation position of the acceleration sensor in the call device. The first direction, the second direction and the third direction are perpendicular to each other, so the accuracy of the sensing signal is improved as much as possible.
[0061] See also Figure 3 , Figure 31 is a schematic diagram of a framework of an embodiment of a call device 30 of the present application. The call device 30 includes: a speaker circuit 31, a pickup circuit 32, a communication circuit 33, an acceleration sensor 34, a memory 35 and a processor 36. The speaker circuit 31, the pickup circuit 32, the communication circuit 33, the acceleration sensor 34 and the memory 35 are coupled to the processor 36. The memory 35 stores program instructions. The processor 36 is used to execute the program instructions to implement the steps in any of the above-mentioned echo cancellation method embodiments. Specifically, the call device 30 may include but is not limited to: a mobile phone, a tablet computer, a laptop computer, etc., which are not limited here.
[0062] Specifically, the processor 36 is used to control itself as well as the speaker circuit 31, the pickup circuit 32, the communication circuit 33, the acceleration sensor 34, and the memory 35 to implement the steps in any of the above-mentioned echo cancellation method embodiments. The processor 36 can also be called a CPU (Central Processing Unit). The processor 36 may be an integrated circuit chip with signal processing capabilities. The processor 36 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 36 can be implemented by an integrated circuit chip.
[0063] In the above scheme, on the one hand, in the echo cancellation process, the reference acceleration sensor can sense the vibration of the diaphragm when the speaker circuit is playing, which is helpful to indirectly obtain the nonlinear echo component, thereby improving the echo cancellation effect. On the other hand, the acceleration sensor is built into the call device, so there is no need to add additional components, which will not bring new uncertainties, nor will it increase additional hardware costs and power consumption. Therefore, echo cancellation can be achieved by using the original components of the call device without adding additional components.
[0064] See also Figure 4 , Figure 4 The schematic diagram is a schematic diagram of a computer-readable storage medium 40 of the present application. The computer-readable storage medium 40 stores program instructions 41 that can be executed by a processor, and the program instructions 41 are used to implement the steps in any of the above-mentioned echo cancellation method embodiments.
[0065] In the above scheme, on the one hand, in the echo cancellation process, the reference acceleration sensor can sense the vibration of the diaphragm when the speaker circuit is playing, which is helpful to indirectly obtain the nonlinear echo component, thereby improving the echo cancellation effect. On the other hand, the acceleration sensor is built into the call device, so there is no need to add additional components, which will not bring new uncertainties, nor will it increase additional hardware costs and power consumption. Therefore, echo cancellation can be achieved by using the original components of the call device without adding additional components.
[0066] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0067] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0068] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0069] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0070] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0071] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
Claims
1. An echo cancellation method, It is characterized in that include: Acquire a near-end signal, a far-end signal and a sensing signal of an acceleration sensor; wherein the near-end signal is collected by a pickup circuit of a call device, the far-end signal is played by a speaker circuit of the call device, and the acceleration sensor, the pickup circuit and the speaker circuit all belong to the call device; The near-end signal is detected to obtain a call state; wherein the call state includes any one of the following: a near-end single talk state, a far-end single talk state, and a far-end and near-end dual talk state; Based on the call status, determining a target filter and a target signal required to perform the echo cancellation; wherein the target filter includes at least one of a first filter, a second filter, and a third filter, and the target signal includes at least the near-end signal; Processing the target signal using the target filter to obtain an optimized signal; Wherein, determining the target filter and target signal required for performing the echo cancellation based on the call status includes: In response to the call state being the far-end single-talk state, determining that the target filter includes the first filter, the second filter, and the third filter, and determining that the target signal includes the near-end signal, the far-end signal, and the sensing signal; The step of using the target filter to process the target signal to obtain an optimized signal includes: Using the far-end signal as a first reference signal of the first filter, obtaining a first expected signal of the first filter based on the sensing signal, and acquiring a first error signal output by the first filter; wherein the first error signal represents a nonlinear echo component in the sensing signal; Using the far-end signal as a second reference signal of the second filter, using the near-end signal as a second expected signal of the second filter, and acquiring a second error signal output by the second filter; wherein the second error signal represents the near-end signal after eliminating the linear echo component; The first error signal is used as a third reference signal of the third filter, and the second error signal is used as a third expected signal of the third filter, and the third error signal output by the third filter is obtained as the optimization signal; wherein the third error signal represents the near-end signal after eliminating the linear echo component and the nonlinear echo component.
2. The method according to claim 1, It is characterized in that The method further comprises at least one of: updating the first filter using the first desired signal and the first error signal; updating the second filter using the second desired signal and the second error signal; The third filter is updated using the third reference signal and the third error signal.
3. The method according to claim 1, It is characterized in that The obtaining a first expected signal of the first filter based on the sensing signal comprises: The sensing signal is upsampled to a target sampling rate; wherein the target sampling rate is a sampling rate of the near-end signal.
4. The method according to claim 1, It is characterized in that The step of determining a target filter and a target signal required for performing the echo cancellation based on the call status includes: In response to the call state being the near-end and far-end double talk state, determining that the target filter includes the second filter, and determining that the target signal includes the near-end signal and the far-end signal; The step of using the target filter to process the target signal to obtain an optimized signal includes: Using the far-end signal as a fourth reference signal of the second filter, using the near-end signal as a fourth expected signal of the second filter, and acquiring a fourth error signal output by the second filter as the optimization signal; The fourth error signal represents the near-end signal after the linear echo component is eliminated.
5. The method according to claim 1, It is characterized in that The step of determining a target filter and a target signal required for performing the echo cancellation based on the call status includes: In response to the call state being the near-end single-talk state, determining that no filter needs to be selected as the target filter, and determining that the target signal includes the near-end signal; The step of using the target filter to process the target signal to obtain an optimized signal includes: The near-end signal is directly used as the optimized signal.
6. The method according to claim 1, It is characterized in that Before using the target filter to process the target signal to obtain an optimized signal, the method further includes: Initializing the first filter, the second filter, and the third filter; The first filter is used to output the nonlinear echo component in the sensing signal, the second filter is used to eliminate the linear echo component in the near-end signal, and the third filter is used to eliminate the nonlinear echo component in the near-end signal.
7. The method according to claim 1, It is characterized in that The acceleration sensor is used to capture acceleration information of a target position in a first direction, a second direction and a third direction respectively, and the sensing signal is obtained based on the acceleration information in at least one of the first direction, the second direction and the third direction; The target position indicates an installation position of the acceleration sensor in the communication device, and the first direction, the second direction and the third direction are perpendicular to each other.
8. An echo cancellation device, It is characterized in that include: A signal acquisition module, used to acquire a near-end signal, a far-end signal and a sensing signal of an acceleration sensor; wherein the near-end signal is collected by a pickup circuit of a call device, the far-end signal is played by a speaker circuit of the call device, and the acceleration sensor, the pickup circuit and the speaker circuit all belong to the call device; A state detection module is used to detect the near-end signal to obtain a call state; wherein the call state includes any one of the following: a near-end single-talk state, a far-end single-talk state, and a far-end and near-end dual-talk state; An echo cancellation module, configured to determine a target filter and a target signal required to perform the echo cancellation based on the call status; wherein the target filter includes at least one of a first filter, a second filter, and a third filter, and the target signal includes at least the near-end signal; and the target signal is processed by using the target filter to obtain an optimized signal; Wherein, determining the target filter and target signal required for performing the echo cancellation based on the call status includes: In response to the call state being the far-end single-talk state, determining that the target filter includes the first filter, the second filter, and the third filter, and determining that the target signal includes the near-end signal, the far-end signal, and the sensing signal; The step of using the target filter to process the target signal to obtain an optimized signal includes: Using the far-end signal as a first reference signal of the first filter, obtaining a first expected signal of the first filter based on the sensing signal, and acquiring a first error signal output by the first filter; wherein the first error signal represents a nonlinear echo component in the sensing signal; Using the far-end signal as a second reference signal of the second filter, using the near-end signal as a second expected signal of the second filter, and acquiring a second error signal output by the second filter; wherein the second error signal represents the near-end signal after eliminating the linear echo component; The first error signal is used as a third reference signal of the third filter, and the second error signal is used as a third expected signal of the third filter, and the third error signal output by the third filter is obtained as the optimization signal; wherein the third error signal represents the near-end signal after eliminating the linear echo component and the nonlinear echo component.
9. A communication device, It is characterized in that The invention comprises a speaker circuit, a pickup circuit, a communication circuit, an acceleration sensor, a memory and a processor, wherein the speaker circuit, the pickup circuit, the communication circuit, the acceleration sensor and the memory are coupled to the processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the echo cancellation method according to any one of claims 1 to 7.
10. A computer-readable storage medium, It is characterized in that Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the echo cancellation method according to any one of claims 1 to 7.
