An echo cancellation method and apparatus

Abstract

The application discloses an echo cancellation method and device. Remote data and near-end data are buffered by appropriate buffers, the delay of the near-end data buffer is set according to the delay of the remote data buffer, and the delays of the remote data buffer and the near-end data buffer are ensured to be consistent. Then, the remote data interface is called first, and the near-end data interface is called later, so that the remote signal in the near-end data is received by the echo cancellation module before being processed, the echo canceller can be well operated, and the echo cancellation effect is ensured.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an echo cancellation method and apparatus. Background Technology

[0002] In the field of real-time audio communication, acoustic echoes can cause significant interference to calls, necessitating echo cancellation. Echoes occur when sound recorded by a remote device is played back by a nearby playback device, then re-recorded by the nearby recording device, and finally transmitted over the network to the remote device for playback. Typically, echo cancellation modules in VoIP (Voice Over Internet Protocol) clients have two interfaces: one for receiving reference audio data from the remote device, and the other for receiving and processing the nearby audio data. The echo canceller converts the remote audio data into a frequency domain signal using a Fourier transform, then locates and cancels the corresponding audio data from the nearby device. This echo cancellation requires two conditions: first, the amounts of remote and nearby data must match; second, the remote signal must be received by the echo cancellation module first, and then appear in the nearby data sent to the echo cancellation module. These two conditions are crucial for ensuring the proper functioning of the echo cancellation module.

[0003] In existing VoIP programs, the two interfaces of the echo cancellation module are usually called separately in the playback module and the recording module. If there is clock drift in the recording module and the playback module, it will cause a mismatch between the near-end data volume and the far-end data volume after running for a period of time, which will affect the operation of the echo cancellation module. Summary of the Invention

[0004] This application provides an echo cancellation method and apparatus to solve the problems of mismatched echo cancellation data and poor echo cancellation effect in existing echo cancellation methods.

[0005] Firstly, to achieve the above objectives, this application provides an echo cancellation device, the device comprising:

[0006] A remote data buffer is used to cache the audio data that needs to be played and automatically adjusts the amount of cached data.

[0007] A near-end data buffer is used to buffer audio data after recording. The near-end data buffer can be set with a fixed delay, and the delay of the near-end data buffer can be configured according to the delay of the far-end data buffer.

[0008] The echo cancellation module is used to obtain the latency of the remote data buffer and send the latency of the remote data buffer to the near-end data buffer; obtain the first audio frame and the data volume of the first audio frame from the near-end data buffer; determine whether an equal amount of second audio frames can be obtained from the remote data buffer based on the data volume of the first audio frame; if it can be obtained, send the second audio frames to the echo canceller, and then send the first audio frames to the echo canceller.

[0009] An echo canceller is used to receive the second audio data first, then the first audio data, and perform echo cancellation processing.

[0010] Furthermore, the echo cancellation device further includes:

[0011] The playback module is used to send the audio data to be played to the remote data buffer;

[0012] The recording module is used to send the recorded audio data to the near-end data buffer.

[0013] Furthermore, the latency of the near-end data buffer is greater than or equal to the latency of the far-end data buffer.

[0014] Furthermore, the echo cancellation module is also used for:

[0015] Call the remote interface of the echo canceller and send the second audio frame to the echo canceller;

[0016] The first audio frame is sent to the echo canceller by calling the near-end interface of the echo canceller.

[0017] Secondly, to achieve the above objectives, this application provides an echo cancellation method, the method comprising:

[0018] Determine whether the first audio frame and the amount of data in the first audio frame can be obtained from the near-end data buffer;

[0019] If available, retrieve an equal amount of second audio frames from the remote data buffer, based on the amount of data in the first audio frame;

[0020] The second audio frame is sent to the echo canceller, and then the first audio frame is sent to the echo canceller for echo cancellation processing to obtain the target audio data.

[0021] Furthermore, after determining whether the first audio frame and the amount of data in the first audio frame can be obtained from the near-end data buffer, the method further includes:

[0022] If it is not available, check again after a preset interval to see if it is available.

[0023] Further, the second audio frame is sent to the echo canceller, and then the first audio frame is sent to the echo canceller for echo cancellation processing to obtain the target audio data, including:

[0024] Call the remote interface of the echo canceller and send the second audio frame to the echo canceller;

[0025] The first audio frame is sent to the echo canceller by calling the near-end interface of the echo canceller.

[0026] Furthermore, the latency of the near-end data buffer is greater than or equal to the latency of the far-end data buffer.

[0027] Thirdly, to achieve the above objectives, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the echo cancellation methods of the second aspect.

[0028] Fourthly, to achieve the above objectives, this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the echo cancellation methods of the second aspect.

[0029] As can be seen from the above technical solutions, the echo cancellation method and apparatus provided in this application caches remote and near-end data using appropriate buffers. The delay of the near-end data buffer is set to the delay of the remote data buffer, ensuring consistency between their delays. Each call to the remote data interface corresponds to a call to the near-end data interface, with equal amounts of data in both, guaranteeing matching of remote and near-end data volumes. By calling the remote data interface first, followed by the near-end data interface, it is ensured that the remote signal in the near-end data is received by the echo cancellation module before being processed, guaranteeing proper operation of the echo canceller and ensuring the effectiveness of echo cancellation.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the echo cancellation device provided in the embodiments of this application;

[0033] Figure 2This is a schematic flowchart of the echo cancellation method provided in the embodiments of this application. Detailed Implementation

[0034] The features and advantages of this application will become clearer and more explicit through the following detailed description.

[0035] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0036] In the description of this application, it should be noted that the terms "upper," "lower," "inner," "outer," "front," "rear," "left," and "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship in the working state of this application. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. In addition, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0037] To address the aforementioned problems, this application provides an echo cancellation method and apparatus. Specific embodiments of this application are described in detail below with reference to the accompanying drawings.

[0038] Figure 1 This is a schematic diagram of the structure of an echo cancellation device provided in an embodiment of this application, as shown below. Figure 1 As shown, the device includes: a playback module 100, a recording module 200, a remote data buffer 300, a near-end data buffer 400, an echo cancellation module 500, and an echo canceller 600.

[0039] The playback module 100 is used to send the audio data to be played to the remote data buffer;

[0040] The recording module 200 is used to send the recorded audio data to the near-end data buffer.

[0041] In practice, before playing the sound, the playback module 100 sends the audio data to be played to the remote data buffer 300, while the recording module 200 sends the recorded audio data to the near-end data buffer 400 for buffering after recording the sound.

[0042] Echo cancellation requires two conditions to be met: first, the amount of data at the far end and the amount of data at the near end must match; second, the far-end audio data must be received by the echo canceller first, and then appear in the near-end data sent to the echo canceller.

[0043] Based on the above requirements, a remote data buffer 300 and a near-end data buffer 400 are set up. The remote data buffer 300 is used to buffer the audio data that needs to be played, and automatically adjusts the amount of buffered data. The latency caused by buffering should be as short as possible.

[0044] In practice, clock drift and other phenomena may occur, causing the echo cancellation module to consume remote audio data faster than the playback module 100 generates it. In this case, a segment of audio data with continuity can be "extended" from the actual remote audio data output in the last period to ensure smooth operation of the echo cancellation module. When there is redundancy in the amount of remote audio data—that is, when the echo cancellation module consumes remote audio data slower than the playback module 100 generates it—the remote audio data needs to be "compressed" to ensure the echo cancellation module receives the main remote audio data. Therefore, the remote data buffer is an automatically scalable buffer that can automatically adjust the amount of buffered data.

[0045] Furthermore, in practice, both the far-end and near-end data buffers introduce data delays. For example, if the delay caused by the far-end data buffer is 100ms, and the near-end data buffer is set to a delay of 40ms, while the echo delay itself (the time difference between the sound being played from the playback device and recorded by the recording device) is 20ms, then the final delay of the far-end data will be 100 - (40 + 20) = 40ms longer than that of the near-end data. This results in a negative delay for the far-end data, which violates the rule of first "notifying" the echo canceller of the far-end signal and then sending the near-end audio containing that far-end signal to the echo canceller. In this case, the echo cannot be canceled.

[0046] For the reasons mentioned above, the latency of the near-end data buffer 400 should be set based on the latency of the far-end data buffer 300. If the latency of the far-end data buffer is too long, it will lead to a larger near-end data latency, which will also increase the latency of audio encoding and transmission, affecting the immediacy of communication. Therefore, the latency of the far-end data buffer 300 needs to be as short as possible, that is, the latency of the near-end data buffer ≥ the latency of the far-end data buffer.

[0047] The near-end data buffer 400 is used to buffer the audio data after recording. It is a buffer with a fixed delay that can be set at will, so that when the echo cancellation module pulls near-end audio data from the near-end data buffer, the recording data sent from the recording module can be delayed for a specified time before it can be read.

[0048] In practice, before acquiring near-end data each time, the echo cancellation module first acquires the latency of the far-end data buffer and configures the latency of the near-end data buffer according to the latency of the far-end data buffer to ensure that the latency of the near-end data buffer is not lower than the latency of the far-end data buffer, thereby ensuring that the far-end signal in the near-end data is received by the echo cancellation module before being processed.

[0049] The echo cancellation module 500 is used to determine whether the first audio frame and the amount of data of the first audio frame can be obtained from the near-end data buffer; if it can be obtained, the second audio frame of equal amount is obtained from the far-end data buffer according to the amount of data of the first audio frame; the second audio frame is sent to the echo canceller, and then the first audio frame is sent to the echo canceller for echo cancellation processing to obtain the target audio data.

[0050] In practice, the echo cancellation module 500 executes an echo cancellation thread. After the echo cancellation thread starts, it continuously attempts to obtain near-end audio data (the first audio frame) from the near-end data buffer 400, and then attempts to retrieve an equal amount of far-end audio data (the second audio frame) from the far-end data buffer 300. It then calls the far-end data interface to send the far-end audio data to the echo canceller, and then calls the near-end data interface again to send the near-end audio data to the echo canceller. This ensures that the far-end audio data is received by the echo canceller first, ensuring that the echo canceller can cancel the echo as expected.

[0051] The echo canceller 600 is used to receive the second audio data first, followed by the first audio data, and then perform echo cancellation processing. In specific implementation, the echo canceller first receives the far-end audio data, converts it into a far-end pure tone set through Fourier transform, and when it receives the near-end audio and parses it into a near-end pure tone set, it can identify which elements are contained in the far-end audio and thus cancel them.

[0052] This application uses an appropriate buffer to buffer both remote and near-end data, and sets the delay of the near-end data buffer to the delay of the remote data, ensuring that the remote signal in the near-end data has been received by the echo cancellation module before it is processed.

[0053] Corresponding to the embodiment of an echo cancellation device provided in this application, this application also provides an embodiment of an echo cancellation method. (See reference) Figure 2 This is a schematic flowchart of an echo cancellation device provided in an embodiment of this application, wherein the method is applied to an echo cancellation module. It includes:

[0054] S101: Obtain the latency of the remote data buffer and send the latency of the remote data buffer to the near data buffer;

[0055] S102: Determine whether the first audio frame and the amount of data in the first audio frame can be obtained from the near-end data buffer;

[0056] S103: If available, obtain an equal amount of second audio frames from the remote data buffer based on the data volume of the first audio frame;

[0057] S104: Send the second audio frame to the echo canceller, and then send the first audio frame to the echo canceller for echo cancellation processing to obtain the target audio data.

[0058] The second audio frame is sent to the echo canceller, and then the first audio frame is sent to the echo canceller for echo cancellation processing to obtain the target audio data, including:

[0059] S401: Call the remote interface of the echo canceller and send the second audio frame to the echo canceller;

[0060] S402: Call the near-end interface of the echo canceller to send the first audio frame to the echo canceller.

[0061] Furthermore, after determining whether the first audio frame and the amount of data in the first audio frame can be obtained from the near-end data buffer, the method further includes:

[0062] S105: If it is not available, check again after a preset interval to see if it is available.

[0063] In practice, before playing sound, the playback module 100 sends the audio data to be played to the remote data buffer 300, while the recording module 200, after recording sound, sends the recorded audio data to the near-end data buffer 400 for buffering. Before each acquisition of near-end data, the echo cancellation module first obtains the latency of the remote data buffer and configures the latency of the near-end data buffer according to the latency of the remote data buffer to ensure that the latency of the near-end data buffer is not lower than the latency of the remote data buffer, thereby ensuring that the near-end audio data is received by the echo canceller first.

[0064] The echo cancellation module 500 is used to determine whether the first audio frame and the amount of data of the first audio frame can be obtained from the near-end data buffer; if it can be obtained, the second audio frame of equal amount is obtained from the far-end data buffer according to the amount of data of the first audio frame; the second audio frame is sent to the echo canceller, and then the first audio frame is sent to the echo canceller for echo cancellation processing to obtain the target audio data.

[0065] The echo canceller 600 is used to receive the second audio data first, followed by the first audio data, and then perform echo cancellation processing. In specific implementation, the echo canceller first receives the far-end audio data, converts it into a far-end pure tone set through Fourier transform, and when it receives the near-end audio and parses it into a near-end pure tone set, it can identify which elements are contained in the far-end audio and thus cancel them.

[0066] This application uses appropriate buffers to cache both remote and local data, setting the local data buffer's latency to match the remote data buffer's latency to ensure consistency between them. By calling the remote data interface first and then the local data interface, it ensures that the remote signal in the local data is received by the echo cancellation module before being processed, guaranteeing the echo canceller's proper operation and ensuring effective echo cancellation.

[0067] According to the above echo cancellation method, embodiments of this application also provide a readable storage medium and a computer device. The readable storage medium stores an executable program, which, when executed by a processor, implements the steps of the above echo cancellation method. The computer device includes a memory, a processor, and an executable program stored in the memory and executable on the processor; when the processor executes the program, it implements the steps of the above echo cancellation method.

[0068] The present application has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present application. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and implementation methods of the present application without departing from the spirit and scope of the present application, and all such modifications and improvements fall within the scope of the present application. The scope of protection of the present application is determined by the appended claims.

Claims

1. An echo cancellation device, characterized in that, The device includes: A remote data buffer is used to cache the audio data that needs to be played and automatically adjusts the amount of cached data. A near-end data buffer is used to buffer audio data after recording, and the near-end data buffer can be set with a fixed delay; An echo cancellation module is used to obtain the latency of a remote data buffer and send the latency of the remote data buffer to a near-end data buffer; configure the latency of the near-end data buffer according to the latency of the remote data buffer, and the latency of the near-end data buffer is greater than or equal to the latency of the remote data buffer; determine whether a first audio frame and the data volume of the first audio frame can be obtained from the near-end data buffer; if it can be obtained, obtain an equal amount of a second audio frame from the remote data buffer according to the data volume of the first audio frame; send the second audio frame to the echo canceller, and then send the first audio frame to the echo canceller. An echo canceller is used to receive the second audio data first, then the first audio data, and perform echo cancellation processing.

2. The echo cancellation device according to claim 1, characterized in that, Also includes: The playback module is used to send the audio data to be played to the remote data buffer; The recording module is used to send the recorded audio data to the near-end data buffer.

3. The echo cancellation device according to claim 1, characterized in that, The echo cancellation module is also used to: call the remote interface of the echo canceller to send the second audio frame to the echo canceller; and call the near-end interface of the echo canceller to send the first audio frame to the echo canceller.

4. An echo cancellation method, characterized in that, The method includes: Obtain the latency of the remote data buffer and send the latency of the remote data buffer to the near data buffer; configure the latency of the near data buffer according to the latency of the remote data buffer, and the latency of the near data buffer is greater than or equal to the latency of the remote data buffer; Determine whether the first audio frame and its data size can be obtained from the near-end data buffer; if so, obtain an equal amount of the second audio frame from the far-end data buffer based on the data size of the first audio frame. The second audio frame is sent to the echo canceller, and then the first audio frame is sent to the echo canceller for echo cancellation processing to obtain the target audio data.

5. The echo cancellation method according to claim 4, characterized in that, Sending the second audio frame to the echo canceller and then sending the first audio frame to the echo canceller for echo cancellation processing to obtain target audio data includes: calling the remote interface of the echo canceller to send the second audio frame to the echo canceller; and calling the near-end interface of the echo canceller to send the first audio frame to the echo canceller.

Images