Call method, computer device and chip

Abstract

The application relates to a call method, a computer device and a chip. The call method is applied to a first device, the first device is configured with a loudspeaker and at least two microphones; the method comprises the following steps: in the process that the first device and a second device call, after detecting that the call mode of the current call is an external call mode, determining the echo path state of each microphone according to a reference signal and microphone signals collected by each microphone; wherein the reference signal is a signal emitted by the loudspeaker; selecting a target microphone from each microphone according to the echo path state of each microphone; performing echo cancellation on the microphone signal collected by the target microphone according to the reference signal to obtain effective audio; and sending the effective audio to the second device. The application can improve the call quality.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a call method, computer device, and chip. Background Technology

[0002] During a call between a local device and a remote device in speakerphone mode, factors such as the placement or hand grip of the local device may cause its speaker to be obstructed. This alters the path of the sound emitted from the speaker to the microphone—the echo path—compared to the original path when the speaker was unobstructed. Consequently, residual echoes remain in the audio received by the local device after echo cancellation, affecting the audio quality heard by the remote user and thus impacting call quality. See also Figure 1 The first half of the audio signal is the signal collected by the microphone when the microphone is not blocked, and the second half is the signal collected by the microphone when the microphone is blocked. After comparison, the echo amplitude of the second half increased by 10dB.

[0003] For example, a user on one device might be playing a game while simultaneously making a call to another device using speakerphone. In this scenario, because the user's hands are gripping the top and bottom of their device and moving them frequently, their hands obstruct the sound from the sides. This obstruction alters the echo path of the sound from the speaker to the microphone. As a result, even after echo cancellation, the audio received by the user still contains some of the other user's voice and some of the game's background music. When this audio with residual echoes is sent to the other device, the user on the other device will hear their own voice and the game's background music in the audio, affecting call quality. Summary of the Invention

[0004] Therefore, it is necessary to provide a call method, computer device, and chip that can improve call quality in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a call method applied to a first device, the first device being equipped with a speaker and at least two microphones; the method includes: during a call between the first device and a second device, after detecting that the current call mode is speakerphone mode, determining the echo path state of each microphone based on a reference signal and microphone signals collected by each microphone; wherein the reference signal is a signal emitted by the speaker; selecting a target microphone from the microphones based on the echo path state of each microphone; performing echo cancellation on the microphone signal collected by the target microphone based on the reference signal to obtain valid audio; and sending the valid audio to the second device.

[0006] In one embodiment, determining the echo path state of each microphone based on a reference signal and microphone signals collected by each microphone includes: inputting the reference signal into a first filter and a second filter respectively to obtain a first estimated echo signal output by the first filter and a second estimated echo signal output by the second filter; for each microphone, determining a first residual signal based on the difference between the microphone signal collected by the microphone and the first estimated echo signal; and determining a second residual signal based on the difference between the microphone signal collected by the microphone and the second estimated echo signal; and determining the echo path state of the microphone based on the difference between the first residual signal and the second residual signal; wherein the convergence speed of the first filter is less than the convergence speed of the second filter.

[0007] In one embodiment, determining the echo path state of the microphone based on the difference between the first residual signal and the second residual signal includes: determining the degree of correlation between the first residual signal and the second residual signal and the reference signal, respectively; and determining the echo path state of the microphone based on the difference between the first residual signal and the second residual signal, and the determined degree of correlation.

[0008] In one embodiment, echo cancellation is performed on the microphone signal collected by the target microphone based on a reference signal to obtain valid audio, including: performing a nonlinear transformation on the reference signal to obtain a nonlinear signal; and performing echo cancellation on the microphone signal collected by the target microphone based on the nonlinear signal to obtain valid audio.

[0009] In one embodiment, echo cancellation is performed on the microphone signal collected by the target microphone to obtain valid audio based on a nonlinear signal, including: inputting the nonlinear signal into a third filter to obtain a third estimated echo signal output by the third filter; and determining the valid audio based on the difference between the microphone signal collected by the target microphone and the third estimated echo signal.

[0010] In one embodiment, the nonlinear signal includes multiple audio frames; inputting the nonlinear signal into a third filter to obtain a third estimated echo signal output by the third filter includes: processing the first audio frame based on the third filter according to preset filter coefficients corresponding to the first audio frame in the nonlinear signal to obtain an estimated echo signal corresponding to the first audio frame; for each non-first audio frame in the nonlinear signal, determining filter coefficients corresponding to the non-first audio frame according to filter coefficients corresponding to the previous audio frame of the non-first audio frame; and processing the non-first audio frame according to the filter coefficients corresponding to the non-first audio frame to obtain an estimated echo signal corresponding to the non-first audio frame.

[0011] In one embodiment, selecting a target microphone from the microphones based on the echo path state of each microphone includes: in response to the echo path state of each microphone being in an abnormal state, obtaining the signal-to-echo ratio of each microphone and selecting the microphone with the highest signal-to-echo ratio as the target microphone; in response to the existence of at least one microphone in a normal echo path state, selecting at least one of the microphones in a normal echo path state as the target microphone.

[0012] In one embodiment, before performing echo cancellation on the microphone signal collected by the target microphone based on the reference signal, the method further includes: reducing the background noise volume of the first device.

[0013] Secondly, this application also provides a communication device applied to a first device, the first device being equipped with a speaker and at least two microphones; the device includes: a first determining module, configured to, during a communication between the first device and the second device, after detecting that the current communication mode is speakerphone mode, determine the echo path state of each microphone based on a reference signal and microphone signals collected by each microphone; wherein the reference signal is a signal emitted by the speaker; a target selection module, configured to select a target microphone from the microphones based on the echo path state of each microphone; an echo cancellation module, configured to perform echo cancellation on the microphone signals collected by the target microphone based on the reference signal to obtain valid audio; and an audio transmission module, configured to transmit valid audio to the second device.

[0014] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the call method provided in the first aspect.

[0015] Fourthly, this application also provides a chip, including a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the call method provided in the first aspect.

[0016] Fifthly, this application also provides a chip module, including a communication module, a power module, a storage module, and a chip, wherein: the power module is used to provide power to the chip module; the storage module is used to store data and instructions; the communication module is used to perform internal communication within the chip module, or to enable communication between the chip module and external devices; and the chip is used to execute the steps of the call method provided in the first aspect above.

[0017] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the call method provided in the first aspect.

[0018] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the call method provided in the first aspect.

[0019] In the aforementioned communication method, computer equipment, and chip, during the communication between the first device in speaker mode and the second device, the first device determines the echo path status of each microphone based on a reference signal and the microphone signals collected by each microphone, thereby determining the degree of obstruction of each microphone. Then, based on the echo path status of each microphone, it selects a target microphone that is not obstructed or only slightly obstructed. Because the target microphone is not obstructed or only slightly obstructed, the echo path remains unchanged or changes to a minimal degree. The difference between the subsequently estimated echo and the actual echo is small. Therefore, after echo cancellation of the microphone signal collected by the target microphone based on the estimated echo, there is less or even no residual echo in the effective audio, thus improving the purity of the effective audio. After the effective audio is sent to the second device, the voice signal heard by the user at the other end has very little residual echo, thus improving call quality. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a comparison diagram of microphone signals in one embodiment, showing the microphone not being blocked and the microphone being blocked.

[0022] Figure 2 This is a flowchart illustrating a call method in one embodiment;

[0023] Figure 3 This is a flowchart illustrating the echo path state determination steps in one embodiment;

[0024] Figure 4 This is a flowchart illustrating the echo path state determination steps in one embodiment;

[0025] Figure 5 This is a flowchart illustrating the echo cancellation steps in one embodiment;

[0026] Figure 6 This is a flowchart illustrating the echo cancellation steps in one embodiment;

[0027] Figure 7 This is a flowchart illustrating the signal input step in one embodiment;

[0028] Figure 8 This is a flowchart illustrating the background sound volume reduction step in one embodiment;

[0029] Figure 9 This is a structural block diagram of a communication device in one embodiment;

[0030] Figure 10 This is an internal structural diagram of a computer device in one embodiment;

[0031] Figure 11 This is an internal structure diagram of a chip module in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0034] In one exemplary embodiment, a call method is provided, applied to a first device configured with a speaker and at least two microphones. See also Figure 2 The method includes:

[0035] S210, during the communication between the first device and the second device, after detecting that the current communication mode is speaker mode, the echo path status of each microphone is determined based on the reference signal and the microphone signals collected by each microphone.

[0036] In a real-world scenario, the first device is the local device, and the second device is the peer device. When the two devices are having a call, the first device's call mode is speakerphone mode, which can also be called hands-free mode.

[0037] The reference signal is the signal emitted by the speaker, and therefore can also be called the speaker signal. In addition to the voice signal from the second device (i.e., the voice signal of the user using the second device), the reference signal may also include the local background audio of the first device, such as the background music of a game played on the first device.

[0038] Understandably, the reference signal is played through the speaker and then picked up by the microphone; the picked-up signal is called the echo. If the speaker is blocked, in addition to signal attenuation, signal distortion may also occur during signal propagation. That is, the echo may experience signal attenuation and distortion relative to the reference signal.

[0039] The first device has microphones installed at its top and bottom. The number of microphones is not limited to two; it can be more. This embodiment does not impose a limitation on this. Each microphone collects voice signals, which are referred to as microphone signals.

[0040] In addition to the voice signal emitted by the user of the first device (the local user's voice signal), the microphone signal also includes the picked-up echo.

[0041] The echo path refers to the path along which the sound emitted by the speaker is picked up by the microphone after it has traveled through the sound.

[0042] The echo path status includes a normal state and an abnormal state. The normal state indicates that the corresponding microphone is not blocked, while the abnormal state indicates that the corresponding microphone is blocked.

[0043] S220 selects the target microphone from among the microphones based on the echo path status of each microphone.

[0044] Specifically, based on the path echo status of each microphone, microphones that are not obstructed or whose obstruction is not severe can be selected as target microphones, which helps improve the quality of subsequent calls.

[0045] In an optional implementation, the target microphone selection step in S220 includes:

[0046] (1) In response to the abnormal state of the echo path of each microphone, obtain the signal-to-echo ratio of each microphone and select the microphone with the largest signal-to-echo ratio as the target microphone.

[0047] The signal-to-return ratio (SRR) is the ratio of the amplitude of the microphone signal to the echo. A higher SRR indicates better microphone performance.

[0048] Understandably, the smaller the echo amplitude of a microphone, the higher the signal-to-echo ratio. Therefore, when the echo path status of each microphone is abnormal, the microphone with the smallest echo amplitude can be directly selected as the target microphone.

[0049] (2) In response to the presence of at least one microphone with a normal echo path state, at least one of the microphones with a normal echo path state is selected as the target microphone.

[0050] For example, if the top microphone does not exhibit a path abrupt change, then the top microphone is used as the target microphone. If neither the bottom nor the top microphone exhibits a path abrupt change, then both the top and bottom microphones can be used as target microphones.

[0051] The above implementation method can accurately select microphones that are not obstructed or whose obstruction is not severe as target microphones.

[0052] S230: Based on the reference signal, echo cancellation is performed on the microphone signal collected by the target microphone to obtain effective audio.

[0053] Echo cancellation refers to eliminating the echo in the microphone signal that is not part of the user's voice signal. The resulting audio is the valid audio.

[0054] S240, sends valid audio to the second device.

[0055] That is, the first device sends valid audio to the second device, so that the user on the other end can hear the voice signal of the user on this end.

[0056] In this embodiment, during a call between the first device in speaker mode and the second device, the first device determines the echo path status of each microphone based on a reference signal and the microphone signals collected by each microphone, thereby determining the degree of obstruction of each microphone. Then, based on the echo path status of each microphone, a target microphone that is not obstructed or only slightly obstructed is selected from among the microphones. Since the target microphone is not obstructed or only slightly obstructed, the echo path remains unchanged or changes to a minimal degree. The difference between the subsequently estimated echo and the actual echo is small. Therefore, after echo cancellation of the microphone signal collected by the target microphone based on the estimated echo, there is less residual echo in the effective audio, or even none at all, thus improving the purity of the effective audio. After the effective audio is sent to the second device, the voice signal heard by the user at the other end has very little residual echo, thus improving call quality.

[0057] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the echo path state determination step in S210 is refined.

[0058] See Figure 3 The detailed echo path state determination steps include:

[0059] S310, the reference signal is input into the first filter and the second filter respectively to obtain the first estimated echo signal output by the first filter and the second estimated echo signal output by the second filter.

[0060] The convergence speed of the first filter is slower than that of the second filter. Therefore, the first filter is a slow filter, and the second filter is a fast filter.

[0061] In the process of echo estimation frame by frame, the estimated echo signal gets closer and closer to the actual echo. The speed at which the estimated echo signal approaches the actual echo can be understood as the convergence speed.

[0062] Understandably, both the first and second filters perform echo estimation based on a reference signal. Because the second filter uses a larger step size, it can quickly track echo changes and converge faster when abrupt changes occur in the echo path. Therefore, the estimated echo signal output at the moment of an echo path change is more accurate. Conversely, the first filter uses a smaller step size and cannot quickly track echo changes when an echo path change occurs, resulting in slower convergence and lower accuracy in the estimated echo signal output at the moment of an echo path change. Therefore, when an echo path change occurs, there is a significant difference between the estimated echo signals output by the two filters. However, when there is no change in the echo path, there is a smaller difference between the estimated echo signals output by the two filters.

[0063] S320, for each microphone, determines a first residual signal based on the difference between the microphone signal collected by the microphone and the first estimated echo signal.

[0064] For example, for each microphone, the difference between the microphone signal collected by that microphone and the first estimated echo signal is directly used as the first residual signal.

[0065] S330, determine the second residual signal based on the difference between the microphone signal collected by the microphone and the second estimated echo signal.

[0066] For example, for each microphone, the difference between the microphone signal collected by that microphone and the second estimated echo signal is directly used as the second residual signal.

[0067] S340, determine the echo path state of the microphone based on the difference between the first residual signal and the second residual signal.

[0068] Understandably, when a sudden change in the echo path occurs, the first residual signal retains more echo, while the second residual signal retains less. Therefore, the amplitude of the first residual signal is larger, and the amplitude of the second residual signal is smaller. However, when the echo path is normal, the difference in the residual echoes in the two residual signals is not significant. Therefore, the echo path state of the microphone can be determined by the amplitude difference between the first and second residual signals.

[0069] The echo path state determination step in S340 may include: determining the amplitude ratio between the first residual signal and the second residual signal; if the amplitude ratio is greater than a preset ratio, determining that the echo path state of the microphone is abnormal; if the amplitude ratio is not greater than the preset ratio, determining that the echo path state of the microphone is normal.

[0070] The preset ratio can be set as needed and is not limited here.

[0071] Of course, other methods can be used to determine the echo path status based on the above differences, which are not limited here.

[0072] In this embodiment, echo estimation is performed using two filters with different convergence speeds. The estimated echo signal output from each filter is then subtracted from the microphone signal to obtain two residual signals. When the echo path state is abnormal, the difference between the two residual signals increases. Therefore, based on the difference between the two residual signals, it is possible to accurately determine whether the echo path has abruptly changed, i.e., the echo path state.

[0073] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the echo path state determination step in S340 is refined.

[0074] See Figure 4 The detailed echo path state determination steps include:

[0075] S410, determine the degree of correlation between the first residual signal and the second residual signal and the reference signal, respectively.

[0076] The degree of relevance can also be understood as the degree of matching.

[0077] Understandably, the more residual echoes in the residual signal, the higher the degree of matching between the residual signal and the reference signal, i.e., the higher the correlation.

[0078] S420, based on the difference between the first residual signal and the second residual signal, and the determined correlation levels, determines the echo path state of the microphone.

[0079] For example, for each microphone, if the difference between the first residual signal and the second residual signal is large (e.g., the amplitude ratio between the first residual signal and the second residual signal is greater than a preset ratio), and the correlation between the first residual signal and the reference signal is greater than a preset correlation, or the correlation between the first residual signal and the reference signal and the correlation between the second residual signal and the reference signal are both greater than the preset correlation, then the echo path state of the microphone is considered to be abnormal.

[0080] In this embodiment, in determining the echo path state, in addition to considering the difference between the first residual signal and the second residual signal, the degree of correlation between the first residual signal and the second residual signal and the reference signal is also considered, which can further improve the accuracy of the echo path state determination.

[0081] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the echo cancellation step in S230 is refined.

[0082] See Figure 5 The detailed echo cancellation steps include:

[0083] S510 performs a nonlinear transformation on the reference signal to obtain a nonlinear signal.

[0084] The reference signal can be nonlinearly transformed by means of throttling, or the reference signal can be input into the sigmoid function to achieve nonlinear transformation. Of course, other methods can also be used to perform nonlinear transformation, which are not limited here.

[0085] Understandably, in the process of echo estimation based on the reference signal by the filter, the estimated echo signal is usually determined by linear estimation. However, in addition to linear components, the echo also includes nonlinear components. Therefore, this embodiment performs a nonlinear transformation on the reference signal to create nonlinear distortion, so that the nonlinear signal has nonlinear components.

[0086] S520 performs echo cancellation on the microphone signal collected by the target microphone based on the nonlinear signal to obtain effective audio.

[0087] Specifically, echo estimation is performed based on the nonlinear signal to obtain an estimated echo signal (e.g., the third estimated echo signal below). Then, the difference between the microphone signal collected by the target microphone and the estimated echo signal is calculated to obtain the effective audio.

[0088] Understandably, since nonlinear signals contain nonlinear components, echo estimation based on nonlinear signals will result in an estimated echo signal with nonlinear components, which can then be eliminated during echo cancellation. Of course, echo estimation itself is linear echo estimation, so linear components in the echo can also be eliminated during echo cancellation.

[0089] In this embodiment, a nonlinear transformation is performed on the reference signal to obtain a nonlinear signal. Based on the nonlinear signal, echo cancellation is performed on the microphone signal collected by the target microphone. This can eliminate the nonlinear components in the echo while eliminating the linear components, thereby further improving the echo cancellation effect, further improving the purity of the effective audio, and further improving the call quality.

[0090] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the echo cancellation step in S520 is refined.

[0091] See Figure 6 The detailed echo cancellation steps include:

[0092] S610 inputs the nonlinear signal into the third filter to obtain the third estimated echo signal output by the third filter.

[0093] The principle of echo estimation for the third filter is the same as that for the first and second filters mentioned above; both are linear echo estimation methods.

[0094] Understandably, the third estimated echo signal includes both nonlinear and linear components.

[0095] S620 determines the valid audio based on the difference between the microphone signal collected by the target microphone and the third estimated echo signal.

[0096] For example, the difference between the microphone signal collected by the target microphone and the third estimated echo signal is directly used as the effective audio.

[0097] In this embodiment, the nonlinear signal is input into the third filter, thereby using the third filter to perform echo estimation based on the nonlinear signal, which can improve the efficiency of echo estimation.

[0098] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided in which the nonlinear signal includes multiple frames of audio, and the signal input step in S610 is refined.

[0099] See Figure 7 The detailed signal input steps include:

[0100] S710, based on the third filter, processes the first frame audio according to the preset filter coefficients corresponding to the first frame audio in the nonlinear signal to obtain the estimated echo signal corresponding to the first frame audio.

[0101] Understandably, the reference signal, nonlinear signal, and microphone signal are all time series, and these three signals are time-aligned, meaning that each audio value in the three signals corresponds one-to-one.

[0102] The preset filter signal is pre-set.

[0103] Specifically, the third filter filters the first frame audio based on the preset filter coefficients corresponding to the first frame audio in the nonlinear signal to obtain the estimated echo signal corresponding to the first frame audio.

[0104] S720, for each non-first frame audio in the nonlinear signal, determines the filter coefficients corresponding to the non-first frame audio based on the filter coefficients corresponding to the previous frame audio.

[0105] For example, the filter coefficients corresponding to the nth frame of audio can be determined using the following formula:

[0106]

[0107] In the formula, The filter coefficients for the nth frame of audio at the kth frequency point are... These are the filter coefficients of the (n-1)th frame of audio at the kth frequency point. This represents the conjugate data of the nonlinear signal at the k-th frequency point. The step size of the third filter. To prevent the constant from diverging; The power spectrum of the reference signal, This is the threshold value for the residual signal of the nth frame of audio at the kth frequency point. It is evident that when determining the filter coefficients, all the above parameters need to be converted to different frequency points.

[0108] Of course, other methods can be used to determine the filter coefficients corresponding to the nth frame of audio, which are not limited here.

[0109] S730 processes the non-first frame audio based on the filter coefficients corresponding to the non-first frame audio to obtain the estimated echo signal corresponding to the non-first frame audio.

[0110] Specifically, the third filter processes the non-first frame audio based on the filter coefficients corresponding to the non-first frame audio in the nonlinear signal to obtain the estimated echo signal corresponding to the non-first frame audio.

[0111] Thus, the estimated echo signals corresponding to each frame of audio in the nonlinear signal can be obtained, thereby forming the third estimated echo signal corresponding to the nonlinear signal.

[0112] In this embodiment, for each non-first frame audio in the nonlinear signal, the filter coefficients corresponding to the non-first frame audio are determined according to the filter coefficients corresponding to the previous frame audio, thereby realizing adaptive adjustment of the filter coefficients corresponding to the non-first frame audio and improving the accuracy of estimating the echo signal frame by frame.

[0113] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the call method is further refined to include a background noise volume reduction step performed before echo cancellation of the microphone signal collected by the target microphone according to the reference signal.

[0114] See Figure 8 The steps to reduce background noise volume include:

[0115] S810, reduce the background sound volume of the first device.

[0116] In this embodiment, reducing the background sound volume of the first device can reduce the volume of local background music in the subsequent echo, thereby reducing the echo, improving the signal-to-return ratio of the microphone, and thus improving the call quality.

[0117] In practical scenarios, for a local device with only one microphone, when an abnormal echo path occurs, possible measures include reducing the background noise volume of the first device. Other measures include performing a nonlinear transformation on the reference signal, estimating the echo based on the obtained nonlinear signal, and then subtracting the microphone signal from the estimated echo signal to obtain the valid audio. Of course, other measures can also be used, which are not limited here.

[0118] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0119] Based on the same inventive concept, this application also provides a communication device for implementing the aforementioned communication method. This device can be applied to or integrated into a chip or chip module, for example. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more communication device embodiments provided below can be found in the limitations of the communication method described above, and will not be repeated here.

[0120] In one exemplary embodiment, such as Figure 9 As shown, a communication device is provided, applied to a first device, the first device being configured with a speaker and at least two microphones; the device includes: a first determination module 910, a target selection module 920, and an echo cancellation module 930, wherein:

[0121] The first determining module 910 is used to determine the echo path state of each microphone based on a reference signal and the microphone signals collected by each microphone after detecting that the current call mode is speaker mode during a call between the first device and the second device; wherein the reference signal is the signal emitted by the speaker.

[0122] The target selection module 920 is used to select a target microphone from each microphone based on the echo path status of each microphone.

[0123] The echo cancellation module 930 is used to cancel the echo of the microphone signal collected by the target microphone based on the reference signal to obtain effective audio;

[0124] The audio transmission module is used to send valid audio to the second device.

[0125] In one embodiment, the first determining module includes: a first input unit, configured to input a reference signal into a first filter and a second filter respectively to obtain a first estimated echo signal output by the first filter and a second estimated echo signal output by the second filter; a first determining unit, configured to determine a first residual signal for each microphone based on the difference between the microphone signal collected by the microphone and the first estimated echo signal; and a second determining unit, configured to determine a second residual signal based on the difference between the microphone signal collected by the microphone and the second estimated echo signal; and a third determining unit, configured to determine the echo path state of the microphone based on the difference between the first residual signal and the second residual signal; wherein the convergence speed of the first filter is less than the convergence speed of the second filter.

[0126] In one embodiment, the third determining unit is specifically used to: determine the degree of correlation between the first residual signal and the second residual signal and the reference signal, respectively; and determine the echo path state of the microphone based on the difference between the first residual signal and the second residual signal, and the determined degree of correlation.

[0127] In one embodiment, the echo cancellation module includes: a signal transformation unit for performing a nonlinear transformation on a reference signal to obtain a nonlinear signal; and an echo cancellation unit for performing echo cancellation on the microphone signal collected by the target microphone based on the nonlinear signal to obtain effective audio.

[0128] In one embodiment, the echo cancellation unit includes: a first input subunit for inputting a nonlinear signal into a third filter to obtain a third estimated echo signal output by the third filter; and a first determination subunit for determining the effective audio based on the difference between the microphone signal collected by the target microphone and the third estimated echo signal.

[0129] In one embodiment, the nonlinear signal includes multiple audio frames; the first input subunit is specifically configured to: process the first audio frame based on a third filter according to the preset filter coefficients corresponding to the first audio frame in the nonlinear signal to obtain an estimated echo signal corresponding to the first audio frame; for each non-first audio frame in the nonlinear signal, determine the filter coefficients corresponding to the non-first audio frame according to the filter coefficients corresponding to the previous audio frame of the non-first audio frame; and process the non-first audio frame according to the filter coefficients corresponding to the non-first audio frame to obtain an estimated echo signal corresponding to the non-first audio frame.

[0130] In one embodiment, the target selection module is configured to: in response to the fact that the echo path states of each microphone are all in an abnormal state, obtain the signal-to-echo ratio of each microphone and select the microphone with the highest signal-to-echo ratio as the target microphone; in response to the existence of at least one microphone in a normal echo path state, select at least one of the microphones in a normal echo path state as the target microphone.

[0131] In one embodiment, the echo cancellation module is also used to reduce the background noise volume of the first device before performing echo cancellation on the microphone signal collected by the target microphone according to the reference signal.

[0132] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0133] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a communication method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0134] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the call method described above.

[0136] Based on the same inventive concept, this application also provides a chip, including a processor and a communication interface; the communication interface is used to receive or send data; the processor is configured to cause the chip to perform the steps of the call method described above.

[0137] It is understood that the chip involved in the embodiments of this application may be a field-programmable gate array (FPGA), may be an application-specific integrated circuit (ASIC), may be a system on chip (SoC), may be a central processor unit (CPU), may be a network processor (NP), may be a digital signal processor (DSP), may be a microcontroller unit (MCU), may be a programmable logic device (PLD), or other integrated chips, etc.

[0138] Based on the same inventive concept, this application also provides a chip module, such as... Figure 11 As shown, the chip module includes a communication module, a power module, a storage module, and a chip. Specifically: the power module provides power to the chip module; the storage module stores data and instructions; the communication module enables internal communication within the chip module or communication between the chip module and external devices; and the chip corresponds to the chip in the aforementioned chip embodiment. The implementation of this chip module can be found in the relevant content of the aforementioned chip embodiment, and will not be repeated here.

[0139] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the call method described above.

[0140] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the call method described above.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0144] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for making a call, characterized in that, Applied to a first device, the first device being configured with a speaker and at least two microphones; the method includes: During a call between the first device and the second device, after detecting that the current call mode is speakerphone mode, the echo path state of each microphone is determined based on a reference signal and the microphone signals collected by each microphone; wherein, the reference signal is the signal emitted by the speaker. Select a target microphone from among the microphones based on the echo path status of each microphone; Based on the reference signal, echo cancellation is performed on the microphone signal collected by the target microphone to obtain effective audio; The valid audio is sent to the second device.

2. The method according to claim 1, characterized in that, The step of determining the echo path state of each microphone based on the reference signal and the microphone signals collected by each microphone includes: The reference signal is input into the first filter and the second filter respectively to obtain the first estimated echo signal output by the first filter and the second estimated echo signal output by the second filter. For each microphone, a first residual signal is determined based on the difference between the microphone signal collected by the microphone and the first estimated echo signal; and, The second residual signal is determined based on the difference between the microphone signal collected by the microphone and the second estimated echo signal; The echo path state of the microphone is determined based on the difference between the first residual signal and the second residual signal; The convergence speed of the first filter is less than that of the second filter.

3. The method according to claim 2, characterized in that, Determining the echo path state of the microphone based on the difference between the first residual signal and the second residual signal includes: Determine the degree of correlation between the first residual signal and the second residual signal and the reference signal, respectively; The echo path state of the microphone is determined based on the difference between the first residual signal and the second residual signal, and the determined correlation levels.

4. The method according to any one of claims 1 to 3, characterized in that, The step of performing echo cancellation on the microphone signal collected by the target microphone based on the reference signal to obtain effective audio includes: The reference signal is subjected to a nonlinear transformation to obtain a nonlinear signal; Based on the nonlinear signal, echo cancellation is performed on the microphone signal collected by the target microphone to obtain effective audio.

5. The method according to claim 4, characterized in that, The step of performing echo cancellation on the microphone signal collected by the target microphone based on the nonlinear signal to obtain effective audio includes: The nonlinear signal is input into the third filter to obtain the third estimated echo signal output by the third filter; The valid audio is determined based on the difference between the microphone signal collected by the target microphone and the third estimated echo signal.

6. The method according to claim 5, characterized in that, The third estimated echo signal includes the estimated echo signal corresponding to each frame of audio in the microphone signal collected by the target microphone; The step of inputting the nonlinear signal into the third filter to obtain the third estimated echo signal output by the third filter includes: Based on the third filter, the first frame audio is processed according to the preset filter coefficients corresponding to the first frame audio in the nonlinear signal to obtain the estimated echo signal corresponding to the first frame audio. For each non-first frame audio in the nonlinear signal, the filter coefficients corresponding to the non-first frame audio are determined based on the filter coefficients corresponding to the previous frame audio; and, Based on the filter coefficients corresponding to the non-first frame audio, the non-first frame audio is processed to obtain the estimated echo signal corresponding to the non-first frame audio.

7. The method according to any one of claims 1 to 3, characterized in that, The step of selecting a target microphone from the microphones based on the echo path state of each microphone includes: In response to the abnormal state of the echo path of each microphone, the signal-to-return ratio of each microphone is obtained, and the microphone with the highest signal-to-return ratio is selected as the target microphone; In response to the presence of at least one microphone in the normal echo path state, at least one of the microphones in the normal echo path state is selected as the target microphone.

8. The method according to any one of claims 1 to 3, characterized in that, Before performing echo cancellation on the microphone signal collected by the target microphone based on the reference signal, the method further includes: Reduce the background sound volume of the first device.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A chip, characterized in that, The device includes a processor and a communication interface, the processor being configured to cause the chip to perform the steps of the method described in any one of claims 1 to 8.

Images