Audio device, echo cancellation method and apparatus

Abstract

The application provides an example in the technical field of signal processing, and provides an audio device, an echo cancellation method and an apparatus. The audio device comprises a loudspeaker, a microphone, and a processor connected with the microphone and the loudspeaker respectively. The processor is configured to: acquire an audio signal frame received by the microphone; perform echo cancellation processing on the audio signal frame; if a nonlinear filter suppression coefficient used in the echo cancellation processing is less than a coefficient threshold value, and a current call state is detected as a double-talk state, then add 1 to a count value; if the count value is greater than or equal to a count threshold value, then reduce the volume of the loudspeaker; if the count value is less than the count threshold value, then perform the acquisition of the audio signal frame received by the microphone; if the nonlinear filter suppression coefficient is greater than or equal to the coefficient threshold value, or the current call state is detected as a non-double-talk state, then clear the count value, and perform the acquisition of the audio signal frame received by the microphone. The application can effectively eliminate echo.

Description

Technical Field

[0001] This application relates to the field of signal processing technology. More specifically, it relates to an audio device, an echo cancellation method, and an apparatus. Background Technology

[0002] In audio devices that combine speakers and microphones, the audio signal emitted by the speakers is reflected multiple times in space before reaching the microphone, creating an echo. Since echoes severely degrade communication quality and reduce speech recognition rates, echo cancellation (AEC) technology is needed to eliminate them.

[0003] Currently, when using AEC (Audio Echo Cancellation) technology to cancel echoes, a reference signal of the far-end speech is typically obtained from the speaker amplifier of the audio equipment. Based on this reference signal, an estimated value of the echo is obtained through an adaptive estimation method, and this estimated value is subtracted from the audio signal received by the microphone. However, when the speaker has nonlinear distortion, the echo cancellation effect of this method is poor. Summary of the Invention

[0004] Exemplary embodiments of this application provide an audio device, echo cancellation method, and apparatus that can effectively eliminate echoes and improve the echo cancellation effect when there is nonlinear distortion in the speaker.

[0005] In a first aspect, embodiments of this application provide an audio device, including:

[0006] speaker;

[0007] microphone;

[0008] The processor, connected to both the microphone and speaker, is configured as follows:

[0009] Acquire audio signal frames received through the microphone, the audio signal frames containing echoes;

[0010] Perform echo cancellation processing on audio signal frames;

[0011] If the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold, and the current call state is detected to be two-way talk, then the count value is incremented by 1; if the count value is greater than or equal to the count threshold, then the speaker volume is reduced; if the count value is less than the count threshold, then the audio signal frame received through the microphone is acquired.

[0012] If the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected to be non-dual talk state, the count value is cleared and the audio signal frame received through the microphone is acquired.

[0013] In some possible implementations, the echo includes linear echo and nonlinear echo, and the processor is specifically configured to: perform echo cancellation processing on linear echo in the audio signal frame through a preset linear filter; and perform echo cancellation processing on nonlinear echo in the audio signal frame through a preset nonlinear filter, wherein the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter.

[0014] In some possible implementations, the processor is specifically configured to reduce the speaker volume by a preset volume value if the count value is greater than or equal to the count threshold.

[0015] In some possible implementations, the processor is specifically configured to determine the current call state as dual-talk if it detects that the microphone module is operating in dual-talk mode.

[0016] In some possible implementations, the processor is also configured to: after reducing the volume of the speaker, clear the count value to zero and perform the acquisition of audio signal frames received through the microphone.

[0017] Secondly, embodiments of this application provide an echo cancellation method applied to an audio device, the echo cancellation method comprising:

[0018] Acquire audio signal frames received through the microphone, the audio signal frames containing echoes;

[0019] Perform echo cancellation processing on audio signal frames;

[0020] If the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold, and the current call state is detected to be two-way talk, then the count value is incremented by 1; if the count value is greater than or equal to the count threshold, then the speaker volume is reduced; if the count value is less than the count threshold, then the audio signal frame received through the microphone is acquired.

[0021] If the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected to be non-dual talk state, the count value is cleared and the audio signal frame received through the microphone is acquired.

[0022] In some possible implementations, the echo includes linear echo and nonlinear echo. Echo cancellation processing of the audio signal frame includes: echo cancellation processing of linear echo in the audio signal frame through a preset linear filter; echo cancellation processing of nonlinear echo in the audio signal frame through a preset nonlinear filter, wherein the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter.

[0023] In some possible implementations, if the count value is greater than or equal to a count threshold, the speaker volume is reduced, including: if the count value is greater than or equal to the count threshold, the speaker volume is reduced by a preset volume value.

[0024] In some possible implementations, detecting that the current call state is a two-way talk state includes: if the microphone module is detected to be working in a two-way talk state, then determining that the current call state is a two-way talk state.

[0025] In some possible implementations, after reducing the speaker volume, the echo cancellation method further includes: resetting the count value to zero and acquiring the audio signal frame received through the microphone.

[0026] Thirdly, embodiments of this application provide an echo cancellation device applied to an audio device, the echo cancellation device comprising:

[0027] The acquisition module is used to acquire audio signal frames received through the microphone, and the audio signal frames contain echoes;

[0028] The echo cancellation module is used to perform echo cancellation processing on audio signal frames;

[0029] The processing module is configured to increment the count value by 1 if the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold and the current call state is detected as a two-way talk state; decrease the speaker volume if the count value is greater than or equal to the count threshold; acquire the audio signal frame received through the microphone if the count value is less than the count threshold; and clear the count value and acquire the audio signal frame received through the microphone if the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected as a non-two-way talk state.

[0030] In some possible implementations, the echo includes linear echo and nonlinear echo. The cancellation module is specifically used to: cancel the linear echo in the audio signal frame using a preset linear filter; and cancel the nonlinear echo in the audio signal frame using a preset nonlinear filter, wherein the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter.

[0031] In some possible implementations, the processing module is specifically used to: reduce the speaker volume by a preset volume value if the count value is greater than or equal to the count threshold.

[0032] In some possible implementations, the processing module is specifically used to: if the microphone module is detected to be working in dual-talk mode, then determine that the current call state is dual-talk mode.

[0033] In some possible implementations, after reducing the volume of the speaker, the processing module is also used to: clear the count value and execute the acquisition module to acquire the audio signal frame received through the microphone.

[0034] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed, implement the echo cancellation method as described in the second aspect of this application.

[0035] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the echo cancellation method as described in the second aspect of this application.

[0036] The audio device, echo cancellation method, and apparatus provided in this application acquire audio signal frames received through a microphone, the audio signal frames containing echoes; perform echo cancellation processing on the audio signal frames; if the nonlinear filtering suppression coefficient used in the echo cancellation processing is less than a coefficient threshold, and the current call state is detected as a two-way talk state, then increment the count value by 1; if the count value is greater than or equal to the count threshold, then reduce the speaker volume; if the count value is less than the count threshold, then acquire the audio signal frames received through the microphone; if the nonlinear filtering suppression coefficient used in the echo cancellation processing is greater than or equal to the coefficient threshold, or if the current call state is detected as a non-two-way talk state, then clear the count value and acquire the audio signal frames received through the microphone. Because this application reduces the speaker volume when the nonlinear filtering suppression coefficient in consecutive two-way talk state audio signal frames greater than or equal to the count threshold is less than the coefficient threshold, it can automatically adjust the speaker volume to a suitable level when nonlinear distortion exists in the speaker, effectively eliminating echoes and improving the echo cancellation effect.

[0037] These and other aspects of this application will become more apparent in the description of the following embodiments(s). Attached Figure Description

[0038] To more clearly illustrate the implementation methods in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0039] Figure 1 This is a schematic diagram illustrating an operational scenario between an audio device and a user, provided in an embodiment of this application.

[0040] Figure 2 A hardware configuration block diagram of an audio device provided in an embodiment of this application;

[0041] Figure 3 A flowchart of an echo cancellation method provided in an embodiment of this application;

[0042] Figure 4 A flowchart of an echo cancellation method provided in another embodiment of this application;

[0043] Figure 5 This is a schematic diagram of the structure of an echo cancellation device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0045] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation on its own.

[0046] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0047] In this application, the terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, in situations where implementation is possible in a sequence other than those given in the embodiments illustrated or described in this application.

[0048] Furthermore, the terms “including” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0049] As used in this application, the term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0050] As used in this application, the term "remote control" refers to a component of an electronic device (such as the audio device disclosed in this application) that typically allows for wireless control of the electronic device over a short distance. It generally uses infrared and / or radio frequency (RF) signals and / or Bluetooth to connect to the electronic device, and may also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors. For example, a handheld touch remote control replaces most of the physical built-in hard buttons in a typical remote control device with a user interface on a touchscreen.

[0051] As used in this application, the term "gesture" refers to user behavior that uses a change in hand shape or hand movement to express an expected idea, action, purpose, and / or result.

[0052] Figure 1 This is a schematic diagram illustrating an operational scenario between an audio device and a user, provided in an embodiment of this application. Figure 1 As shown, the audio device 100 includes a speaker 101 and a microphone 102. The user receives voice signals from a remote user through the speaker 101 and sends voice signals to the remote user through the microphone 102.

[0053] Figure 2 This is a hardware configuration block diagram of an audio device provided in one embodiment of this application. For example... Figure 2 As shown, the audio device 100 includes a controller 110, an audio processor 120, a communication interface 130, a user input interface 140, an audio output interface 150, and a memory 160.

[0054] In some embodiments, the controller 110 includes at least one of a processor 111, a random access memory (RAM) 112, a read-only memory (ROM) 113, an audio processor 120, a communication interface 130, and a communication bus. The communication bus connects the various components. The controller 110 is used to control the operation and function of the audio device 100, as well as communication and cooperation between internal components and external and internal data processing functions.

[0055] In some embodiments, processor 111 is a central processing unit (CPU) for executing operating system and application instructions, as well as various interactive instructions received from external input, to execute various applications, data, and content. In some exemplary embodiments, processor 111 may include multiple processors. The multiple processors may include a main processor and one or more sub-processors.

[0056] In some embodiments, RAM 112 is used to store temporary data of the operating system or other running programs.

[0057] In some embodiments, ROM 113 is used to store various system startup instructions.

[0058] In some embodiments, the audio processor 120 is used to receive external audio signals, and perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing according to the standard codec protocol of the input signal, to obtain a sound signal that can be played in the speaker 101.

[0059] In some embodiments, the audio output interface 150 receives sound signals output by the audio processor 120, such as those from the speaker 101, under the control of the controller 110.

[0060] In some embodiments, the communication interface 130 is a component used to communicate with external devices or external servers according to various communication protocol types. For example, the communication interface 130 may include network communication protocol chips or near-field communication protocol chips such as a WiFi chip (corresponding to WiFi module 131), a Bluetooth communication protocol chip (corresponding to Bluetooth module 132), and a wired Ethernet communication protocol chip (corresponding to wired Ethernet module 133).

[0061] In some embodiments, the user input interface 140 is used to convert the received analog signal into a digital signal for transmission to an external device. The user input interface 140 may include a microphone 102, which can be used to collect voice data.

[0062] In some embodiments, the memory 160 is used to store various operating programs, data and applications that drive and control the audio device 100 under the control of the controller 110.

[0063] In some embodiments, the audio device 100 may also have a display function, such as including a display ( Figure 2 (Not shown in the image). Specifically, the audio device 100 may be, for example, a large conference screen with display capabilities.

[0064] In audio devices that have both speakers and microphones, the audio signal emitted by the speaker is reflected multiple times in space before reaching the microphone, creating an echo. For example, in a remote conferencing scenario, the audio signal from the other end of the conference is played through the speaker of the local audio device and then picked up by the microphone, resulting in an echo. Therefore, echo cancellation technology is needed to eliminate the echo; otherwise, the other end of the conference will hear their own voice, significantly impacting the quality of the conference call.

[0065] Currently, when using AEC (Audio Echo Cancellation) technology to cancel echoes, a reference signal of the far-end speech is typically obtained from the speaker amplifier of the audio equipment. Based on this reference signal, an estimated value of the echo is obtained through an adaptive estimation method (such as a segmented block frequency domain adaptive filter algorithm), and this estimated value is subtracted from the audio signal received by the microphone. The processing effect of AEC technology is closely related to the linear relationship between the reference signal and the echo signal. However, the distortion of the speaker itself and the reflected reverberation generated during speech playback can affect the linear relationship between the reference signal and the echo signal. Especially at high volumes, the distortion and reflected reverberation generated by the high-power operation of the speaker will significantly increase, severely disrupting the linear relationship between the reference signal and the echo signal, causing a significant decrease in the AEC processing effect, leading to nonlinear filter mismatch, and ultimately resulting in echo leakage or local speech cancellation.

[0066] To address the aforementioned issues, this application provides an audio device, an echo cancellation method, and an apparatus that can automatically adjust the speaker volume to an appropriate level when a mismatch or over-convergence is detected in the nonlinear filtering during adaptive estimation, thereby improving the echo cancellation effect.

[0067] The following detailed embodiments illustrate how this application performs echo cancellation.

[0068] Figure 3 The flowchart illustrates an echo cancellation method provided in one embodiment of this application, applied to an audio device including a speaker, a microphone, and a processor connected to both the microphone and the speaker. Figure 3 As shown, the processor in the audio device is configured to perform the following steps:

[0069] In S301, an audio signal frame received through the microphone is acquired, and the audio signal frame contains an echo.

[0070] In this embodiment of the application, the audio processing is exemplary, which is streaming processing. 10ms of data can be treated as a frame to obtain an audio signal frame. The audio signal frame received by the microphone contains echo and near-end speech signal (i.e., the speech signal corresponding to near-end speech). Therefore, the audio signal frame received by the microphone, which contains echo, can be obtained.

[0071] In S302, echo cancellation processing is performed on the audio signal frame.

[0072] In this step, the echo cancellation algorithm used for echo cancellation processing of the audio signal frame is, for example, a segmented block frequency domain adaptive filter algorithm. For details on how to perform echo cancellation processing on the audio signal frame, please refer to subsequent embodiments; it will not be repeated here.

[0073] In S303, if the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold and the current call state is detected to be a two-way talk state, then the count value is incremented by 1; if the count value is greater than or equal to the count threshold, then the speaker volume is reduced; if the count value is less than the count threshold, then the audio signal frame received through the microphone is acquired.

[0074] In this step, the nonlinear filtering suppression coefficient is used to characterize the energy ratio of the expected near-end signal and the residual nonlinear echo signal in different frequency bands of the estimated echo signal. The nonlinear filtering suppression coefficient is represented by hNI, and the value of hNI ranges from [0, 1.0]. The smaller the value of hNI, the more nonlinear cancellation the echo will undergo. The coefficient threshold is, for example, 0.5. For example, when performing echo cancellation processing on an audio signal frame, the echo cancellation algorithm used also includes a dual-end detection function, which can detect whether the current call state is a two-way talk state. Among them, dual-end detection means that the call state is divided into four types, including near-end speaking, far-end speaking, simultaneous speaking at both near and far ends (i.e., two-way talk state), and neither the far end nor the near end speaking at the same time. If the hNI used in the echo cancellation processing is less than 0.5, and the current call state is detected to be a two-way talk state, it indicates that the current audio signal frame is a two-way talk signal, and there is also a high degree of nonlinear cancellation. Therefore, the count value is incremented by 1.

[0075] The counting threshold is, for example, 20. For instance, if the count value is greater than or equal to the counting threshold, it means that the hNI of 20 consecutive dual-talk audio signal frames is less than 0.5. If the duration of one audio signal frame is, for example, 10ms, then the total duration of the 20 consecutive dual-talk audio signal frames is 200ms. This indicates significant near-end speech cancellation, suggesting the loss of whole words in the near-end speech, which affects semantic understanding. Therefore, the speaker volume is reduced. If the count value is less than the counting threshold, the speaker volume remains unchanged, and step S301 is executed, i.e., the next audio signal frame received through the microphone is acquired, and then the steps after S301 are executed.

[0076] In S304, if the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected to be a non-dual talk state, the count value is cleared and the audio signal frame received through the microphone is acquired.

[0077] In this step, the coefficient threshold is, for example, 0.5. For example, if the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to 0.5, or if the current call state is detected to be a non-dual talk state (e.g., near-end speaking, far-end speaking, or both near and far ends not speaking at the same time), then the count value is cleared, the speaker volume remains unchanged, and step S301 is executed, that is, the next audio signal frame received through the microphone is acquired, and then the steps after S301 are executed.

[0078] It is understandable that steps S301 to S304 constitute a cyclical process. If the count value is determined to be greater than or equal to the count threshold, the volume of the speaker is reduced.

[0079] The echo cancellation method provided in this application embodiment acquires an audio signal frame received through a microphone, the audio signal frame containing an echo; performs echo cancellation processing on the audio signal frame; if the nonlinear filtering suppression coefficient used in the echo cancellation processing is less than a coefficient threshold, and the current call state is detected as a two-way talk state, then the count value is incremented by 1; if the count value is greater than or equal to the count threshold, then the speaker volume is reduced; if the count value is less than the count threshold, then the acquisition of the audio signal frame received through the microphone is performed; if the nonlinear filtering suppression coefficient used in the echo cancellation processing is greater than or equal to the coefficient threshold, or if the current call state is detected as a non-two-way talk state, then the count value is cleared, and the acquisition of the audio signal frame received through the microphone is performed. Since this application embodiment reduces the speaker volume when the nonlinear filtering suppression coefficient in consecutive two-way talk state audio signal frames greater than or equal to the coefficient threshold is less than the coefficient threshold, it can automatically adjust the speaker volume to a suitable level when nonlinear distortion exists in the speaker, effectively eliminating echo and improving the echo cancellation effect.

[0080] The echo cancellation method provided in this application embodiment will be described in detail below with reference to specific steps.

[0081] Figure 4 A flowchart of an echo cancellation method provided in another embodiment of this application. For example... Figure 4 As shown, the processor in the audio device is configured to perform the following steps:

[0082] In S401, an audio signal frame received through the microphone is acquired, and the audio signal frame contains an echo.

[0083] The specific implementation process of this step can be found in the relevant description of S301, and will not be repeated here.

[0084] In this embodiment, the echo includes linear echo and nonlinear echo. An echo can also be understood as the superposition of linear and nonlinear echoes. Linear echo is the signal captured by a microphone after a distant speech signal is played through a speaker and reflected through space (e.g., a room). Nonlinear echo is the signal captured by a microphone after a sudden change in the echo path, resulting from nonlinear distortion in the speaker's playback. Figure 3 Step S302 can further include the following two steps, S402 and S403:

[0085] In S402, echo cancellation processing is performed on the linear echo in the audio signal frame by using a preset linear filter.

[0086] In this step, a preset linear filter, such as a fixed-step normalized least mean square (NLMS) filter, is used to eliminate linear echoes. Therefore, echo cancellation can be performed on linear echoes in audio signal frames using a preset linear filter. Preset linear filters can be found in current related technologies, and will not be elaborated upon here.

[0087] In S403, echo cancellation processing is performed on the nonlinear echo in the audio signal frame by using a preset nonlinear filter, wherein the nonlinear filter suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter.

[0088] In this step, a preset nonlinear filter is used to determine the state of far-end and near-end audio signal frames by calculating signal coherence. Then, the nonlinear filter suppression coefficient is adjusted to calculate the filtering parameters for nonlinear echo cancellation. Therefore, the preset nonlinear filter performs echo cancellation processing on the nonlinear echoes in the audio signal frames. The nonlinear filter suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter, which can control the convergence speed of the nonlinear filter and can adaptively adjust its value. Referring to step S303, the nonlinear filter suppression coefficient is represented by hNI, and the value range of hNI is [0, 1.0]. The smaller the value of hNI, the more nonlinear cancellation the echo signal will undergo. The preset nonlinear filter can be found in current related technologies, which will not be elaborated here.

[0089] In S404, it is determined whether the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold.

[0090] For example, the coefficient threshold is 0.5. After performing echo cancellation processing on the nonlinear echo in the audio signal frame using a preset nonlinear filter, the nonlinear filtering suppression coefficient used by the preset nonlinear filter can be obtained. Therefore, it can be determined whether the nonlinear filtering suppression coefficient used in the echo cancellation process is less than 0.5. If it is determined that the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold, it indicates that the current audio signal frame has high nonlinearity cancellation, and step S405 is executed; if it is determined that the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, step S410 is executed.

[0091] In S405, it is determined whether the current call status is dual-talk mode.

[0092] For example, when performing echo cancellation processing on audio signal frames, the echo cancellation algorithm used also includes a dual-end detection function, so it can determine whether the current call state is a dual-talk state.

[0093] Furthermore, optionally, the processor is also configured to determine that the current call state is a dual-talk state if the microphone module is detected to be operating in dual-talk mode.

[0094] For example, when performing echo cancellation processing on audio signal frames, the echo cancellation algorithm used also includes a dual-end detection function. If the microphone pickup module is detected to be working in dual-talk mode, it indicates that the current audio signal frame is a dual-talk signal, and the current call state is determined to be dual-talk mode.

[0095] If the current call status is determined to be dual-talk mode, then execute step S406; if the current call status is determined to be non-dual-talk mode, then execute step S410.

[0096] In S406, the count value is incremented by 1.

[0097] In S407, it is determined whether the count value is greater than or equal to the count threshold.

[0098] For example, the counting threshold is 20. If the count value is determined to be greater than or equal to the counting threshold, it means that the hNI of 20 consecutive dual-talk audio signal frames is less than 0.5. If the duration of one audio signal frame is, for example, 10ms, then the total duration of 20 consecutive dual-talk audio signal frames is 200ms. This indicates that there is obvious near-end speech cancellation, which can be judged as the loss of whole words in the near-end speech, which has affected the understanding of semantics. Therefore, the speaker volume is reduced, and step S408 is executed. If the count value is determined to be less than the counting threshold, step S401 is executed, that is, the next audio signal frame received through the microphone is acquired, and then the steps after S401 are executed.

[0099] In S408, the speaker volume is reduced to a preset volume value.

[0100] For example, if the preset volume value is 3dB, the speaker volume will be reduced by 3dB.

[0101] In S409, after reducing the volume of the speaker, the count value is cleared and step S401 is executed.

[0102] In this step, after reducing the speaker volume, the count value is cleared to determine whether there is still significant near-end speech cancellation in subsequent audio signal frames. If it is determined that there is still significant near-end speech cancellation, the speaker volume is further reduced by performing the above steps.

[0103] In S410, the count value is cleared and step S401 is executed.

[0104] This step involves resetting the count value to zero and then re-counting whether there are audio signal frames whose nonlinear filtering suppression coefficient is less than the coefficient threshold and whose current call status is dual-talk mode.

[0105] The echo cancellation method provided in this application embodiment acquires an audio signal frame received through a microphone, the audio signal frame containing an echo; performs echo cancellation processing on the linear echo in the audio signal frame using a preset linear filter; performs echo cancellation processing on the nonlinear echo in the audio signal frame using a preset nonlinear filter, wherein the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter; determines whether the nonlinear filtering suppression coefficient used in the echo cancellation processing is less than a coefficient threshold; if it is determined that the nonlinear filtering suppression coefficient used in the echo cancellation processing is less than the coefficient threshold, then determines whether the current call state is a two-way talk state; if it is determined that the current call state is a two-way talk state, then increments the count value by 1; determines whether the count value is greater than or equal to a counting threshold; if it is determined that the count value is greater than or equal to the counting threshold, then reduces the speaker volume by a preset volume value; if the count value is less than the counting threshold, then acquires the audio signal frame received through the microphone; if it is determined that the nonlinear filtering suppression coefficient used in the echo cancellation processing is greater than or equal to the coefficient threshold, or if it is detected that the current call state is not a two-way talk state, then clears the count value and acquires the audio signal frame received through the microphone. Since the embodiments of this application reduce the volume of the speaker by a preset volume value when the nonlinear filtering suppression coefficient in a number of consecutive dual-talk audio signal frames that are greater than or equal to the counting threshold is less than the coefficient threshold, the volume of the speaker can be automatically adjusted to a suitable level when there is nonlinear distortion in the speaker, effectively eliminating echo and improving the echo cancellation effect.

[0106] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0107] Figure 5 This is a schematic diagram of an echo cancellation device provided in one embodiment of this application. The echo cancellation device is applied to an audio device, which includes a speaker, a microphone, and a processor connected to both the microphone and the speaker. Figure 5 As shown, the echo cancellation device 500 provided in this embodiment includes: an acquisition module 501, an echo cancellation module 502, and a processing module 503. Wherein:

[0108] The acquisition module 501 is used to acquire audio signal frames received through the microphone, the audio signal frames containing echoes.

[0109] The echo cancellation module 502 is used to perform echo cancellation processing on audio signal frames.

[0110] The processing module 503 is configured to increment the count value by 1 if the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold and the current call state is detected as a two-way talk state; decrease the speaker volume if the count value is greater than or equal to the count threshold; execute the acquisition module 501 to acquire the audio signal frame received through the microphone if the count value is less than the count threshold; and clear the count value and execute the acquisition module 501 to acquire the audio signal frame received through the microphone if the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected as a non-two-way talk state.

[0111] In some embodiments, the echo includes linear echo and nonlinear echo, and the cancellation module 502 can be specifically used to: perform echo cancellation processing on linear echo in the audio signal frame through a preset linear filter; and perform echo cancellation processing on nonlinear echo in the audio signal frame through a preset nonlinear filter, wherein the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter.

[0112] In some embodiments, the processing module 503 may be specifically used to: reduce the volume of the speaker by a preset volume value if the count value is greater than or equal to the count threshold.

[0113] In some embodiments, the processing module 503 may be specifically used to: if the microphone module is detected to be working in dual-talk mode, determine that the current call state is dual-talk mode.

[0114] In some embodiments, after reducing the volume of the speaker, the processing module 503 may also be used to: clear the count value and execute the acquisition module 501 to acquire the audio signal frame received through the microphone.

[0115] It should be noted that the device provided in this embodiment can be used to perform the echo cancellation method described above, and its implementation and technical effects are similar, so it will not be described again here.

[0116] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a processing module can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as program code in the device's memory, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0117] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more ASICs (Application Specific Integrated Circuits), one or more DSPs (Digital Signal Processors), or one or more FPGAs (Field Programmable Gate Arrays). As another example, when a module is implemented through processing element scheduler code, the processing element can be a general-purpose processor, such as a CPU or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SoC).

[0118] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer programs. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, a computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0119] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the echo cancellation method as described in any of the above method embodiments.

[0120] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the echo cancellation method as described in any of the above method embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0122] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the embodiments and various different variations of embodiments suitable for specific application considerations.

Claims

1. An audio device, characterized in that, include: speaker; microphone; The processor, connected to the microphone and the speaker respectively, is configured to: Acquire audio signal frames received through the microphone, the audio signal frames containing echoes; the echoes include linear echoes and nonlinear echoes; Echo cancellation is performed on the linear echoes in the audio signal frame by using a preset linear filter; Echo cancellation is performed on the nonlinear echoes in the audio signal frame by using a preset nonlinear filter. If the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold, and the current call state is detected to be a two-way talk state, then the count value is incremented by 1; if the count value is greater than or equal to the count threshold, then the volume of the speaker is reduced. If the count value is less than the count threshold, then the acquisition of the audio signal frame received through the microphone is performed; wherein, the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter, and the nonlinear filtering suppression coefficient is used to characterize the energy ratio of the expected near-end signal and the residual nonlinear echo signal in different frequency bands in the estimated echo signal; If the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected to be a non-dual talk state, then the count value is cleared and the process of acquiring the audio signal frame received through the microphone is executed.

2. The audio device according to claim 1, characterized in that, The processor is specifically configured as follows: If the count value is greater than or equal to the count threshold, the volume of the speaker will be reduced by a preset volume value.

3. The audio device according to claim 1, characterized in that, The processor is specifically configured as follows: If the microphone module is detected to be in dual-talk mode, then the current call status is determined to be dual-talk mode.

4. The audio device according to any one of claims 1 to 3, characterized in that, The processor is also configured to: After reducing the volume of the speaker, the count value is cleared to zero, and the process of acquiring the audio signal frame received through the microphone is performed.

5. An echo cancellation method, characterized in that, The echo cancellation method, applied to audio devices, includes: Acquire audio signal frames received via a microphone, the audio signal frames containing echoes; the echoes include linear echoes and nonlinear echoes; Echo cancellation is performed on linear echoes in the audio signal frame using a preset linear filter; echo cancellation is performed on nonlinear echoes in the audio signal frame using a preset nonlinear filter. If the nonlinear filtering suppression coefficient used in the echo cancellation process is less than the coefficient threshold, and the current call state is detected to be a two-way talk state, then the count value is incremented by 1; if the count value is greater than or equal to the count threshold, then the speaker volume is reduced; if the count value is less than the count threshold, then the acquisition of the audio signal frame received through the microphone is performed; wherein, the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter; the nonlinear filtering suppression coefficient is used to characterize the energy ratio of the expected near-end signal and the residual nonlinear echo signal in different frequency bands in the estimated echo signal; If the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or if the current call state is detected to be a non-dual talk state, then the count value is cleared and the process of acquiring the audio signal frame received through the microphone is executed.

6. An echo cancellation device, characterized in that, The echo cancellation device, applied to audio equipment, includes: An acquisition module is used to acquire audio signal frames received through a microphone, wherein the audio signal frames contain echoes; the echoes include linear echoes and nonlinear echoes. The echo cancellation module is used to perform echo cancellation processing on linear echoes in the audio signal frame using a preset linear filter; and to perform echo cancellation processing on nonlinear echoes in the audio signal frame using a preset nonlinear filter. The processing module is configured to: increment a count value by 1 if the nonlinear filtering suppression coefficient used in the echo cancellation process is less than a coefficient threshold and the current call state is detected as a two-way talk state; decrease the speaker volume if the count value is greater than or equal to the count threshold; acquire the audio signal frame received through the microphone if the count value is less than the count threshold; and clear the count value and acquire the audio signal frame received through the microphone if the nonlinear filtering suppression coefficient used in the echo cancellation process is greater than or equal to the coefficient threshold, or the current call state is detected as a non-two-way talk state; wherein the nonlinear filtering suppression coefficient is the filtering suppression coefficient used by the preset nonlinear filter, and the nonlinear filtering suppression coefficient is used to characterize the energy ratio of the expected near-end signal and the residual nonlinear echo signal in different frequency bands in the estimated echo signal.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed, implement the echo cancellation method as described in claim 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the echo cancellation method as described in claim 5.

Images