Communication state detection method and apparatus, computer device, and storage medium

Abstract

The application relates to a communication state detection method and device, computer equipment and a storage medium. The method comprises the following steps: determining whether a far-end voice signal exists according to a far-end reference signal; if yes, determining a first voice signal existence probability of a far-end muting frequency point and a second voice signal existence probability of a neighboring frequency point of the far-end muting frequency point according to a microphone acquisition signal; determining a first signal energy of the far-end muting frequency point and a second signal energy of the neighboring frequency point; determining a frequency point energy of the far-end muting frequency point according to the first voice signal existence probability and the first signal energy, and determining a frequency point energy of the neighboring frequency point according to the second voice signal existence probability and the second signal energy; and determining a communication state according to the frequency point energy of the far-end muting frequency point and the frequency point energy of the neighboring frequency point. The above method improves the detection accuracy of the communication state, thereby realizing accurate elimination of echo and improving communication quality.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a communication status detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] In multi-person voice communication scenarios, the communication signal often contains echo signals, which affects the call experience of communication users. Therefore, an echo cancellation system is needed to eliminate echoes during multi-person voice communication. When the echo cancellation system detects that the communication status is two-way, it will activate the echo cancellation mode to eliminate the communication echo. Therefore, to improve the accuracy of echo cancellation, it is necessary to accurately identify whether the communication status is two-way.

[0003] Currently, when detecting communication status, the difference between the energy levels of the microphone signal and the speaker signal is often used to determine whether the communication is in two-way talk mode. However, this method is susceptible to noise, resulting in low accuracy. Therefore, improving the accuracy of communication status detection, thereby enhancing echo cancellation precision and improving the user's communication experience, is a problem that needs to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a communication status detection method, apparatus, computer equipment, and storage medium that can improve the accuracy of communication status detection in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a communication status detection method, the method comprising:

[0006] Determine whether a remote voice signal exists based on the remote reference signal;

[0007] If so, then based on the microphone-collected signal, determine the probability of the existence of the first speech signal at the far-end canceled-tone point and the probability of the existence of the second speech signal at the adjacent frequency point of the far-end canceled-tone point;

[0008] Determine the first signal energy of the far-end noise-canceling point and the second signal energy of the adjacent frequency point;

[0009] The frequency energy of the far-end noise point is determined based on the probability of the first speech signal and the energy of the first signal, and the frequency energy of the adjacent frequency point is determined based on the probability of the second speech signal and the energy of the second signal.

[0010] The communication status is determined based on the frequency energy of the distant anechoic point and the frequency energy of the adjacent frequency points.

[0011] In one embodiment, determining the frequency energy of a distant canceled-out frequency point based on the presence probability of the first speech signal and the first signal energy, and determining the frequency energy of a neighboring frequency point based on the presence probability of the second speech signal and the second signal energy, includes:

[0012] The first spectral energy is determined by the product of the probability of the first speech signal and the energy of the first signal, and the second spectral energy is determined by the product of the probability of the second speech signal and the energy of the second signal.

[0013] The frequency energy of the far-end noise-canceling point is determined based on the first spectral energy, and the frequency energy of the adjacent frequency point is determined based on the second spectral energy.

[0014] In one embodiment, determining the frequency energy of the far-end noise-canceling point based on the first spectral energy and determining the frequency energy of the neighboring frequency point based on the second spectral energy includes:

[0015] Based on the energy of the first spectral energy of consecutive frames and the energy of the frequency point of the far-end noise cancellation point;

[0016] The energy of the second spectral energy of consecutive frames and the energy of the frequency points of neighboring frequencies are determined.

[0017] In one embodiment, determining the communication state based on the frequency energy of the distant noise point and the frequency energy of neighboring frequencies includes:

[0018] Determine the frequency energy ratio between the frequency energy of the far-end noise-canceling point and the frequency energy of the adjacent frequency points;

[0019] The communication status is determined based on the frequency-to-energy ratio.

[0020] In one embodiment, determining the communication state based on the frequency-to-energy ratio includes:

[0021] Determine whether the energy ratios at the specified frequency points are all less than or equal to the energy ratio threshold;

[0022] If so, then the communication status is determined to be remote one-way communication status;

[0023] If not, then the communication status is determined to be dual-talk mode.

[0024] In one embodiment, determining the probability of the presence of a first speech signal at a far-end canceled-tone point and the probability of the presence of a second speech signal at a neighboring frequency point of the far-end canceled-tone point, based on the microphone-collected signal, includes:

[0025] Determine whether a far-end audio signal with a far-end noise point exists in the microphone's acquired signal;

[0026] If so, the prior signal-to-noise ratio of the speech at the far-end canceled audio point is determined by the noise reduction algorithm based on the far-end audio signal in the microphone acquisition signal, and the prior signal-to-noise ratio of the speech at the neighboring frequency point is determined based on the neighboring audio signal at the neighboring frequency point of the far-end canceled audio point.

[0027] The probability of the presence of the first speech signal at the far-end canceled-tone point is determined based on the prior signal-to-noise ratio of the speech at the far-end canceled-tone point, and the probability of the presence of the second speech signal at the adjacent frequency point is determined based on the prior signal-to-noise ratio of the speech at the adjacent frequency point.

[0028] In one embodiment, determining the presence of a far-end voice signal based on a far-end reference signal includes:

[0029] Receive a remote reference signal and perform speech recognition on the remote reference signal to determine the speech endpoint information in the remote reference signal;

[0030] The presence of a remote voice signal is determined based on the voice endpoint information.

[0031] Secondly, this application also provides a communication status detection device, the device comprising:

[0032] The remote voice signal determination module is used to determine whether a remote voice signal exists based on a remote reference signal;

[0033] The signal presence probability determination module is used to determine, if yes, the presence probability of the first voice signal at the far-end canceled-tone point and the presence probability of the second voice signal at the adjacent frequency point of the far-end canceled-tone point based on the signal collected by the microphone.

[0034] A signal energy determination module is used to determine the first signal energy of the far-end noise point and the second signal energy of the adjacent frequency point;

[0035] The frequency point energy determination module is used to determine the frequency point energy of the far-end noise point based on the existence probability of the first speech signal and the energy of the first signal, and to determine the frequency point energy of the neighboring frequency points based on the existence probability of the second speech signal and the energy of the second signal.

[0036] The communication status determination module is used to determine the communication status based on the frequency energy of the remote anechoic point and the frequency energy of the adjacent frequency points.

[0037] Thirdly, this application also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0038] Determine whether a remote voice signal exists based on the remote reference signal;

[0039] If so, then based on the microphone-collected signal, determine the probability of the existence of the first speech signal at the far-end canceled-tone point and the probability of the existence of the second speech signal at the adjacent frequency point of the far-end canceled-tone point;

[0040] Determine the first signal energy of the far-end noise-canceling point and the second signal energy of the adjacent frequency point;

[0041] The frequency energy of the far-end noise point is determined based on the probability of the first speech signal and the energy of the first signal, and the frequency energy of the adjacent frequency point is determined based on the probability of the second speech signal and the energy of the second signal.

[0042] The communication status is determined based on the frequency energy of the distant anechoic point and the frequency energy of the adjacent frequency points.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0044] Determine whether a remote voice signal exists based on the remote reference signal;

[0045] If so, then based on the microphone-collected signal, determine the probability of the existence of the first speech signal at the far-end canceled-tone point and the probability of the existence of the second speech signal at the adjacent frequency point of the far-end canceled-tone point;

[0046] Determine the first signal energy of the far-end noise-canceling point and the second signal energy of the adjacent frequency point;

[0047] The frequency energy of the far-end noise point is determined based on the probability of the first speech signal and the energy of the first signal, and the frequency energy of the adjacent frequency point is determined based on the probability of the second speech signal and the energy of the second signal.

[0048] The communication status is determined based on the frequency energy of the distant anechoic point and the frequency energy of the adjacent frequency points.

[0049] The aforementioned communication status detection method, apparatus, computer equipment, and storage medium, when determining the presence of a far-end voice signal in the far-end reference signal, determine the probability of the presence of a first voice signal at a far-end silenced frequency point in the microphone-collected signal, the first signal energy of the far-end silenced frequency point, the probability of the presence of a second voice signal at a neighboring frequency point of the far-end silenced frequency point, and the second signal energy of the neighboring frequency point. The communication status is then determined based on the probability of the first voice signal presence, the first signal energy, the probability of the second voice signal presence, and the second signal energy. This solves the problem that directly determining whether the communication status is in a two-way communication state by the difference between the microphone signal frequency energy and the speaker signal frequency energy during communication status detection results in low accuracy due to noise interference. The above method, when performing communication status detection, generates a far-end canceled audio point by canceling the far-end signal at a fixed frequency. This amplifies the difference between far-end and near-end speech, improving the accuracy of communication status detection to some extent. Simultaneously, based on the probability of the speech signal presence at the far-end canceled audio point, the signal energy at the far-end canceled audio point, the probability of the speech signal presence at neighboring frequencies of the far-end canceled audio point, and the signal energy at neighboring frequencies, it determines whether the communication status is a two-way communication state. This eliminates noise interference during communication status detection, further improving the accuracy of communication status detection, thereby increasing echo cancellation accuracy and enhancing the user's communication experience. Attached Figure Description

[0050] Figure 1 This is an application environment diagram of the communication status detection method in one embodiment;

[0051] Figure 2 This is a flowchart illustrating a communication status detection method in one embodiment;

[0052] Figure 3 This is a flowchart illustrating the communication status detection method in another embodiment;

[0053] Figure 4 This is a flowchart illustrating the communication status detection method in another embodiment;

[0054] Figure 5 This is a flowchart illustrating the communication status detection method in another embodiment;

[0055] Figure 6 This is a flowchart illustrating the communication status detection method in another embodiment;

[0056] Figure 7 This is a structural block diagram of a communication status detection device in one embodiment;

[0057] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] The communication status detection method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed in the cloud or on another network server. Server 104 determines whether a remote voice signal exists based on a remote reference signal; if so, it determines the probability of the presence of a first voice signal at a remote noise point and the probability of the presence of a second voice signal at a neighboring frequency point based on the microphone signal; it determines the first signal energy of the remote noise point and the second signal energy of the neighboring frequency point; it determines the frequency energy of the remote noise point based on the probability of the first voice signal and the first signal energy, and determines the frequency energy of the neighboring frequency point based on the probability of the second voice signal and the second signal energy; it determines the communication state based on the frequency energy of the remote noise point and the frequency energy of the neighboring frequency point; it processes the voice signal during communication according to the communication state, and sends the processed voice signal to terminal 102 through the communication network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0060] In one embodiment, such as Figure 2 As shown, a communication status detection method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server and an echo cancellation system. The method includes the following steps:

[0061] S210. Determine whether a remote voice signal exists based on the remote reference signal.

[0062] It should be noted that in multi-person voice communication scenarios, real-time transmission of voice signals is required. At this time, the local computer device can collect near-end signals through the microphone and transmit them outwards. At the same time, it can also play the received audio signals through the speaker. Thus, while the microphone is collecting near-end signals, it may also actually collect the audio signals played by the speaker, thereby producing an echo.

[0063] The remote voice signal refers to the voice signal emitted by the remote communication user, which is acquired by the echo cancellation system through the remote communication equipment. The audio signal played by the loudspeaker is the remote reference signal.

[0064] Specifically, when the far-end reference signal is acquired, a speech recognition algorithm is used to perform speech recognition on the far-end reference signal in order to determine whether there is a far-end speech signal in the far-end reference signal based on the speech recognition results.

[0065] S220. If so, then based on the microphone-collected signal, determine the probability of the existence of the first speech signal at the far-end canceled-tone point and the probability of the existence of the second speech signal at the adjacent frequency point of the far-end canceled-tone point.

[0066] It should be noted that during communication status detection, the fixed frequency point of the remote reference signal needs to be zeroed out. The fixed frequency point requiring zeroing is identified as the remote noise level point, which can be preset as needed. Zeroing the fixed frequency point of the remote reference signal can be achieved by using a filter to suppress the signal, reducing its energy to near zero. The neighboring frequencies of the remote noise level point refer to frequencies near it. The positional relationship between these neighboring frequencies and the remote noise level point can be preset, and the neighboring frequencies can be determined based on this relationship. The first voice signal refers to the voice signal corresponding to the remote noise level point, and the second voice signal refers to the voice signal corresponding to the neighboring frequency point of the remote noise level point.

[0067] Among them, the microphone signal acquisition is the audio signal acquired by the microphone of the near-end communication device. When the microphone of the near-end communication device is turned on, the near-end communication device can acquire the far-end audio signal emitted by the speaker on the near-end communication device, at which time an echo phenomenon may occur.

[0068] The first probability of speech presence can characterize the likelihood that a speech signal exists at a far-end anechoic point in the microphone's signal acquisition, while the second probability of speech presence can characterize the likelihood that a speech signal exists at a neighboring frequency point.

[0069] Specifically, if a remote voice signal exists in the remote reference signal, then there is a remote user communicating. Therefore, if it is determined that a remote voice signal exists in the remote reference signal, the microphone acquisition signal is obtained, and the probability of the existence of the first voice signal at the remote canceled frequency point in the microphone acquisition signal and the probability of the existence of the second voice signal at the adjacent frequency point of the remote canceled frequency point are determined.

[0070] S230, determine the first signal energy of the far-end noise point and the second signal energy of the adjacent frequency point.

[0071] The first signal energy refers to the set of reference signal energy for each frame at the far-end anechoic point. The second signal energy refers to the set of reference signal energy for each frame at the adjacent frequency point.

[0072] Specifically, the first signal energy of the far-end anechoic point in the microphone acquisition signal and the second signal energy of the adjacent frequency point in the microphone acquisition signal are determined.

[0073] S240. Determine the frequency point energy of the far-end silenced audio point based on the existence probability of the first speech signal and the energy of the first signal, and determine the frequency point energy of the adjacent frequency point based on the existence probability of the second speech signal and the energy of the second signal.

[0074] The frequency energy of the far-end anechoic point refers to the total spectral energy of the far-end anechoic point. The frequency energy of the neighboring frequency points refers to the total spectral energy of the neighboring frequency points.

[0075] Specifically, based on the presence probability and energy of the first speech signal, the total spectral energy of the far-end canceled-out audio point is determined, and this total spectral energy is used as the frequency point energy of the far-end canceled-out audio point. Based on the presence probability and energy of the second speech signal, the total spectral energy of the neighboring frequency points is determined, and this total spectral energy is used as the frequency point energy of the neighboring frequency points.

[0076] S250. Determine the communication status based on the frequency energy of the remote noise level point and the frequency energy of the adjacent frequency points.

[0077] The communication status is either dual-talk or remote single-talk.

[0078] Specifically, the communication state is determined based on the difference between the frequency energy of the far-end noise cancellation point and the frequency energy of neighboring frequencies. If the communication state is a two-way communication state, the echo cancellation mode is activated to eliminate communication echoes. If the communication state is a far-end one-way communication state, echo cancellation is not required.

[0079] In the aforementioned communication status detection method, when it is determined that a far-end voice signal exists in the far-end reference signal, the probability of the first voice signal at the far-end canceled-tone point in the microphone acquisition signal, the first signal energy of the far-end canceled-tone point, the probability of the second voice signal at the adjacent frequency point of the far-end canceled-tone point, and the second signal energy of the adjacent frequency point are determined. The communication status is then determined based on the probability of the first voice signal, the first signal energy, the probability of the second voice signal, and the second signal energy. This solves the problem that directly determining whether the communication status is in a two-way talk state by the difference between the microphone signal frequency energy and the speaker signal frequency energy during communication status detection is susceptible to noise, resulting in low accuracy of the determined communication status. The above method, when performing communication status detection, generates a far-end canceled audio point by canceling the far-end signal at a fixed frequency. This amplifies the difference between far-end and near-end speech, improving the accuracy of communication status detection to some extent. Simultaneously, based on the probability of the speech signal presence at the far-end canceled audio point, the signal energy at the far-end canceled audio point, the probability of the speech signal presence at neighboring frequencies of the far-end canceled audio point, and the signal energy at neighboring frequencies, it determines whether the communication status is a two-way communication state. This eliminates noise interference during communication status detection, further improving the accuracy of communication status detection, thereby increasing echo cancellation accuracy and enhancing the user's communication experience.

[0080] In one embodiment, such as Figure 3 As shown, the frequency energy of the far-end canceled frequency point is determined based on the presence probability and energy of the first speech signal, and the frequency energy of the adjacent frequency point is determined based on the presence probability and energy of the second speech signal, including:

[0081] S310. Determine the first spectral energy based on the product of the existence probability of the first speech signal and the energy of the first signal, and determine the second spectral energy based on the product of the existence probability of the second speech signal and the energy of the second signal.

[0082] Specifically, the product of the probability of the presence of the first speech signal and the energy of the first signal at each far-end censored frequency point is taken as the first spectral energy of each far-end censored frequency point. The product of the probability of the presence of the second speech signal and the energy of the second signal at each neighboring frequency point is taken as the second spectral energy of each neighboring frequency point.

[0083] S320. Determine the frequency energy of the far-end noise-canceling point based on the first spectral energy, and determine the frequency energy of the adjacent frequency point based on the second spectral energy.

[0084] Specifically, the frequency energy of each far-end noise point is determined based on the first spectral energy corresponding to each far-end noise point; and the frequency energy of each neighboring frequency point is determined based on the second spectral energy corresponding to each neighboring frequency point.

[0085] For example, the frequency energy of the far-end noise-canceling point and the frequency energy of the neighboring frequency points can be determined by: determining the frequency energy of the far-end noise-canceling point based on the energy of the first spectral energy of the consecutive frames; and determining the frequency energy of the neighboring frequency points based on the energy of the second spectral energy of the consecutive frames.

[0086] Determining the frequency energy of the far-end anechoic point based on the first spectral energy of consecutive frames, and determining the frequency energy of the adjacent frequency point based on the second spectral energy of consecutive frames, can improve the efficiency and accuracy of frequency energy acquisition.

[0087] This embodiment provides a method for determining the frequency energy of a distant noise-canceling point and the frequency energy of neighboring frequencies, which can improve the calculation efficiency of the frequency energy of the distant noise-canceling point and the frequency energy of neighboring frequencies.

[0088] In one embodiment, such as Figure 4 As shown, the communication state is determined based on the frequency energy of the distant noise point and the frequency energy of neighboring frequencies, including:

[0089] S410. Determine the frequency energy ratio between the frequency energy of the far-end noise point and the frequency energy of the adjacent frequency points.

[0090] Specifically, the frequency energy ratio is the ratio of the frequency energy of the far-end anechoic point to the frequency energy of the neighboring frequency points.

[0091] S420. Determine the communication status based on the frequency-to-energy ratio.

[0092] Specifically, the difference between the frequency energy of the far-end anechoic point and the frequency energy of the neighboring frequency points is determined based on the frequency energy ratio, and the communication status is determined based on the difference between the frequency energy of the far-end anechoic point and the frequency energy of the neighboring frequency points.

[0093] For example, a method for determining the communication state based on the frequency point energy ratio can be as follows: determine whether the frequency point energy ratios are all less than or equal to the energy ratio threshold; if so, the difference between the frequency point energy of each far-end anechoic point and the frequency point energy of its corresponding neighboring frequency point is small, and it can be determined that there is no far-end voice signal in the microphone acquisition signal, so the communication state is determined to be a far-end single-talk state; if not, there is a large difference between the frequency point energy of at least one far-end anechoic point and the frequency point energy of its corresponding neighboring frequency point, and it can be determined that there is a far-end voice signal in the microphone acquisition signal, so the communication state is a dual-talk state.

[0094] The energy ratio threshold can be set according to the actual situation.

[0095] Specifically, the frequency energy ratio corresponding to each remote anechoic point is compared with the energy ratio threshold. If the frequency energy ratio corresponding to each remote anechoic point is less than or equal to the energy ratio threshold, the communication state is determined to be remote single-talk state; if there is at least one remote anechoic point whose frequency energy ratio is greater than the energy ratio threshold, the communication state is determined to be dual-talk state.

[0096] Determining whether the communication status is dual-talk mode based on the comparison results of the frequency point energy ratio and the energy ratio threshold can improve the efficiency of communication status determination.

[0097] The above scheme obtains the communication status based on the frequency energy ratio between the frequency energy of the far-end noise point and the frequency energy of the neighboring frequency points, which can improve the efficiency of determining the communication status while ensuring the accuracy of the communication status.

[0098] In one embodiment, such as Figure 5 As shown, based on the microphone-collected signals, the probability of the presence of the first speech signal at the far-end canceled-out frequency point and the probability of the presence of the second speech signal at the neighboring frequency point of the far-end canceled-out frequency point are determined, including:

[0099] S510. Determine whether there is a remote audio signal at a remote noise point in the microphone's acquired signal.

[0100] Among them, the far-end audio signal refers to the far-end sound signal emitted by the speaker and collected by the microphone.

[0101] Specifically, a filter is used to determine whether there is a far-end audio signal with a far-end noise cancellation point in the signal collected by the microphone.

[0102] S520. If so, then the prior signal-to-noise ratio of the speech at the far-end canceled audio point is determined by the noise reduction algorithm based on the far-end audio signal in the microphone acquisition signal, and the prior signal-to-noise ratio of the speech at the neighboring frequency point is determined based on the neighboring audio signal at the neighboring frequency point of the far-end canceled audio point.

[0103] For example, a noise reduction algorithm can be an audio compression format noise reduction algorithm, namely the SPEEX noise reduction algorithm.

[0104] The signal-to-noise ratio (SNR) is the ratio of the strength of the received valid signal to the strength of the received interference signal. The far-end speech signal contains both valid and interference signals, while the speech signal in the far-end reference signal is the valid signal. The higher the ratio of the valid signal strength to the interference signal strength at the far-end canceled-out point, the higher the probability of the presence of a speech signal at that point. Therefore, the probability of the presence of the first speech signal at the far-end canceled-out point and the probability of the presence of the second speech signal at a neighboring frequency point can be determined using the prior signal-to-noise ratio.

[0105] Specifically, if the microphone-acquired signal contains a distant audio signal at a distant canceled-out point, a speech signal recognition algorithm is used to determine the distant audio signal at the distant canceled-out point and the adjacent audio signals at neighboring frequencies from the microphone-acquired signal. A noise reduction algorithm is then used to determine the a priori signal-to-noise ratio (SNR) of the distant canceled-out point based on the ratio of valid to invalid signals in the distant audio signal from the microphone-acquired signal, and the a priori SNR of the adjacent frequencies is determined based on the ratio of valid to invalid signals in the adjacent audio signals of the distant audio signal.

[0106] S530. Determine the probability of the existence of the first speech signal at the far-end canceled-tone point based on the prior signal-to-noise ratio of the speech at the far-end canceled-tone point, and determine the probability of the existence of the second speech signal at the adjacent frequency point based on the prior signal-to-noise ratio of the speech at the adjacent frequency point.

[0107] Specifically, the average of the prior signal-to-noise ratio of the speech at the far-end canceled-out point is taken as the first speech signal existence probability at the far-end canceled-out point, and the average of the prior signal-to-noise ratio of the speech at the neighboring frequency point is taken as the second speech signal existence probability at the neighboring frequency point.

[0108] The above scheme provides a method for determining the probability of the presence of a first voice signal at a distant noise point and the probability of the presence of a second voice signal at a neighboring frequency point. This method can improve the accuracy of the probability of the presence of the first voice signal and the second voice signal. Determining the communication status based on the probability of the presence of the first voice signal and the second voice signal can avoid noise interference and improve the reliability of the communication status.

[0109] In one embodiment, such as Figure 6 As shown, determining the presence of a distant speech signal based on a distant reference signal includes:

[0110] S610: Receive the remote reference signal and perform speech recognition on the remote reference signal to determine the speech endpoint information in the remote reference signal.

[0111] The voice endpoint information includes the voice start endpoint and the voice end endpoint.

[0112] Specifically, the system receives a remote reference signal and performs speech recognition on the remote reference signal using a speech signal endpoint detection method to determine the speech endpoint information in the remote reference signal.

[0113] Speech signal endpoint detection is a crucial step in speech signal analysis, synthesis, and recognition. Its purpose is to identify the start and end points of a speech signal within a given segment, ensuring that only valid speech signals are stored and processed. Effective endpoint detection not only reduces data storage and processing time but also eliminates noise interference from silent segments.

[0114] S620. Determine whether a remote voice signal exists based on the voice endpoint information.

[0115] Specifically, the presence of a valid voice signal in the far-end reference signal is determined based on the voice endpoint information; if so, the presence of a far-end voice signal is confirmed.

[0116] The above scheme can improve the accuracy of determining whether a remote voice signal exists by determining the voice endpoint information. After determining the existence of a remote voice signal, further judgment on the communication status can avoid wasting resources.

[0117] For example, based on the above embodiments, the communication status detection method includes:

[0118] An echo cancellation system receives a remote reference signal via a microphone on a remote communication device. It then performs speech recognition on the remote reference signal using a speech signal endpoint detection method to determine the speech endpoint information within the signal. Based on this information, it determines whether a remote speech signal exists. If so, it confirms the presence of a communication user in a call at the remote location and identifies the remote cancellation point of the remote reference signal. When an echo is present, the echo cancellation system needs to cancel the speech signal at the remote cancellation point.

[0119] Determine whether a far-end audio signal from a distant-end noise-canceling point exists in the microphone signal acquired by the near-end communication device. If so, determine the far-end audio signal from the microphone signal acquired by the near-end communication device, as well as the neighboring audio signals of neighboring frequencies. Using a noise reduction algorithm, determine the prior signal-to-noise ratio (SNR) of the far-end noise-canceling point based on the far-end audio signal, and determine the prior SNR of the neighboring frequencies based on the neighboring audio signals. Use the average of the prior SNR of the far-end noise-canceling point as the first probability of the presence of the far-end noise-canceling point's speech signal, and use the average of the prior SNR of the neighboring frequencies as the second probability of the presence of the neighboring frequencies' speech signal.

[0120] The first signal energy of the far-end canceled audio point in the microphone acquisition signal and the second signal energy of the adjacent frequency points in the microphone acquisition signal are determined. The product of the existence probability of the first speech signal corresponding to each far-end canceled audio point and the first signal energy is taken as the first spectral energy of each far-end canceled audio point. The product of the existence probability of the second speech signal of each adjacent frequency point and the second signal energy is taken as the second spectral energy of each adjacent frequency point. The frequency point energy of the far-end canceled audio point is determined based on the energy of the first spectral energy of consecutive frames; the frequency point energy of the adjacent frequency points is determined based on the energy of the second spectral energy of consecutive frames.

[0121] Determine the frequency energy ratio between the frequency energy of the far-end canceled audio point and the frequency energy of its neighboring frequencies. Compare the frequency energy ratio corresponding to each far-end canceled audio point with an energy ratio threshold. If the frequency energy ratio corresponding to each far-end canceled audio point is less than or equal to the energy ratio threshold, it can be considered that the microphone of the near-end communication device is in a closed state. At this time, the microphone of the near-end communication device cannot capture the far-end voice signal emitted by the speaker of the near-end communication device, and the echo cancellation system can only acquire the far-end reference signal transmitted by the far-end communication device and cannot acquire the near-end audio signal. It can be determined that the communication state is a far-end one-way state. If at least one far-end canceled audio point exists... If the frequency-to-energy ratio of the audio point is greater than the energy ratio threshold, then the microphone of the near-end communication device can be considered to be in the on state. At this time, the microphone of the near-end communication device can capture the far-end voice signal emitted by the speaker of the near-end communication device. That is, the microphone of the near-end communication device can acquire both the voice signal emitted by the near-end communicator and the voice signal emitted by the far-end communicator emitted by the speaker of the near-end communication device. At this time, the echo cancellation system can acquire the far-end reference signal transmitted by the far-end communication device and the far-end audio signal emitted by the speaker collected by the microphone of the near-end communication device. Therefore, the communication state can be determined to be a two-way communication state.

[0122] In the aforementioned communication status detection method, when it is determined that a far-end voice signal exists in the far-end reference signal, the probability of the first voice signal at the far-end canceled-tone point in the microphone acquisition signal, the first signal energy of the far-end canceled-tone point, the probability of the second voice signal at the adjacent frequency point of the far-end canceled-tone point, and the second signal energy of the adjacent frequency point are determined. The communication status is then determined based on the probability of the first voice signal, the first signal energy, the probability of the second voice signal, and the second signal energy. This solves the problem that directly determining whether the communication status is in a two-way talk state by the difference between the microphone signal frequency energy and the speaker signal frequency energy during communication status detection is susceptible to noise, resulting in low accuracy of the determined communication status. The above method, when performing communication status detection, generates a far-end canceled audio point by canceling the far-end signal at a fixed frequency. This amplifies the difference between far-end and near-end speech, improving the accuracy of communication status detection to some extent. Simultaneously, based on the probability of the speech signal presence at the far-end canceled audio point, the signal energy at the far-end canceled audio point, the probability of the speech signal presence at neighboring frequencies of the far-end canceled audio point, and the signal energy at neighboring frequencies, it determines whether the communication status is a two-way communication state. This eliminates noise interference during communication status detection, further improving the accuracy of communication status detection, thereby increasing echo cancellation accuracy and enhancing the user's communication experience.

[0123] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0124] Based on the same inventive concept, this application also provides a communication status detection device for implementing the communication status detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more communication status detection device embodiments provided below can be found in the limitations of the communication status detection method described above, and will not be repeated here.

[0125] In one embodiment, such as Figure 7 As shown, a communication status detection device is provided, including: a remote voice signal determination module 701, a signal presence probability determination module 702, a signal energy determination module 703, a frequency point energy determination module 704, and a communication status determination module 705, wherein:

[0126] The remote voice signal determination module 701 is used to determine whether a remote voice signal exists based on a remote reference signal.

[0127] The signal existence probability determination module 702 is used to determine, if yes, the existence probability of the first voice signal at the far-end canceled audio point and the existence probability of the second voice signal at the adjacent frequency point of the far-end canceled audio point based on the signal collected by the microphone.

[0128] The signal energy determination module 703 is used to determine the first signal energy of the far-end noise point and the second signal energy of the adjacent frequency point;

[0129] The frequency point energy determination module 704 is used to determine the frequency point energy of the far-end canceled audio point based on the existence probability of the first speech signal and the first signal energy, and to determine the frequency point energy of the neighboring frequency point based on the existence probability of the second speech signal and the second signal energy.

[0130] The communication status determination module 705 is used to determine the communication status based on the frequency energy of the remote anechoic point and the frequency energy of the adjacent frequency points.

[0131] The aforementioned communication status detection device, upon determining the presence of a far-end voice signal in the far-end reference signal, determines the probability of the presence of a first voice signal at a far-end silenced frequency point in the microphone-collected signal, the first signal energy of the far-end silenced frequency point, the probability of the presence of a second voice signal at a neighboring frequency point, and the second signal energy of the neighboring frequency point. Based on the probability of the first voice signal presence, the first signal energy, the probability of the second voice signal presence, and the second signal energy, the communication status is determined. This solves the problem that directly determining whether the communication status is in a two-way talk state by the difference between the microphone signal frequency energy and the speaker signal frequency energy during communication status detection is susceptible to noise interference, resulting in low accuracy in determining the communication status. The above method, when performing communication status detection, generates a far-end canceled audio point by canceling the far-end signal at a fixed frequency. This amplifies the difference between far-end and near-end speech, improving the accuracy of communication status detection to some extent. Simultaneously, based on the probability of the speech signal presence at the far-end canceled audio point, the signal energy at the far-end canceled audio point, the probability of the speech signal presence at neighboring frequencies of the far-end canceled audio point, and the signal energy at neighboring frequencies, it determines whether the communication status is a two-way communication state. This eliminates noise interference during communication status detection, further improving the accuracy of communication status detection, thereby increasing echo cancellation accuracy and enhancing the user's communication experience.

[0132] For example, the frequency point energy determination module 704 is specifically used for:

[0133] The first spectral energy is determined by the product of the probability of the first speech signal and the energy of the first signal, and the second spectral energy is determined by the product of the probability of the second speech signal and the energy of the second signal.

[0134] The frequency energy of the far-end noise-canceling point is determined based on the first spectral energy, and the frequency energy of the adjacent frequency point is determined based on the second spectral energy.

[0135] Furthermore, the frequency point energy determination module 704 is also specifically used for:

[0136] Based on the energy of the first spectral energy of consecutive frames and the energy of the frequency point of the far-end noise cancellation point;

[0137] The energy of the second spectral energy of consecutive frames and the energy of the frequency points of neighboring frequencies are determined.

[0138] For example, the communication status determination module 705 is specifically used for:

[0139] Determine the frequency energy ratio between the frequency energy of the far-end noise-canceling point and the frequency energy of the adjacent frequency points;

[0140] The communication status is determined based on the frequency-to-energy ratio.

[0141] Furthermore, the communication status determination module 705 is also specifically used for:

[0142] Determine whether the frequency point energy ratio is less than or equal to the energy ratio threshold;

[0143] If so, then the communication status is determined to be remote one-way communication status;

[0144] If not, then the communication status is determined to be dual-talk mode.

[0145] Furthermore, the signal presence probability determination module 702 is specifically used for:

[0146] Determine whether a far-end audio signal with a far-end noise point exists in the microphone's acquired signal;

[0147] If so, the noise reduction algorithm is used to determine the prior signal-to-noise ratio of the speech at the far-end canceled audio point based on the far-end audio signal in the microphone acquisition signal, and the prior signal-to-noise ratio of the speech at the neighboring frequency point is determined based on the neighboring audio signal at the neighboring frequency point of the far-end canceled audio point.

[0148] The probability of the first speech signal at the far-end canceled-tone point is determined based on the prior signal-to-noise ratio of the speech at the far-end canceled-tone point, and the probability of the second speech signal at the adjacent frequency point is determined based on the prior signal-to-noise ratio of the speech at the adjacent frequency point.

[0149] For example, the remote voice signal determination module 701 is specifically used for:

[0150] Receive the remote reference signal and perform speech recognition on the remote reference signal to determine the speech endpoint information in the remote reference signal;

[0151] Determine whether a remote voice signal exists based on the voice endpoint information.

[0152] Each module in the aforementioned communication status detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0153] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a communication status detection method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0154] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0156] Step 1: Determine whether a remote voice signal exists based on the remote reference signal;

[0157] Step 2: If yes, then based on the microphone signal, determine the probability of the existence of the first speech signal at the far-end canceled-out frequency point and the probability of the existence of the second speech signal at the neighboring frequency point of the far-end canceled-out frequency point.

[0158] Step 3: Determine the first signal energy at the far-end noise-canceling point and the second signal energy at the adjacent frequency point;

[0159] Step 4: Determine the frequency energy of the far-end canceled-tone point based on the existence probability of the first speech signal and the energy of the first signal, and determine the frequency energy of the adjacent frequency points based on the existence probability of the second speech signal and the energy of the second signal.

[0160] Step 5: Determine the communication status based on the frequency energy of the distant noise point and the frequency energy of the adjacent frequency points.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0162] Step 1: Determine whether a remote voice signal exists based on the remote reference signal;

[0163] Step 2: If yes, then based on the microphone signal, determine the probability of the existence of the first speech signal at the far-end canceled-out frequency point and the probability of the existence of the second speech signal at the neighboring frequency point of the far-end canceled-out frequency point.

[0164] Step 3: Determine the first signal energy at the far-end noise-canceling point and the second signal energy at the adjacent frequency point;

[0165] Step 4: Determine the frequency energy of the far-end canceled-tone point based on the existence probability of the first speech signal and the energy of the first signal, and determine the frequency energy of the adjacent frequency points based on the existence probability of the second speech signal and the energy of the second signal.

[0166] Step 5: Determine the communication status based on the frequency energy of the distant noise point and the frequency energy of the adjacent frequency points.

[0167] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0168] Step 1: Determine whether a remote voice signal exists based on the remote reference signal;

[0169] Step 2: If yes, then based on the microphone signal, determine the probability of the existence of the first speech signal at the far-end canceled-out frequency point and the probability of the existence of the second speech signal at the neighboring frequency point of the far-end canceled-out frequency point.

[0170] Step 3: Determine the first signal energy at the far-end noise-canceling point and the second signal energy at the adjacent frequency point;

[0171] Step 4: Determine the frequency energy of the far-end canceled-tone point based on the existence probability of the first speech signal and the energy of the first signal, and determine the frequency energy of the adjacent frequency points based on the existence probability of the second speech signal and the energy of the second signal.

[0172] Step 5: Determine the communication status based on the frequency energy of the distant noise point and the frequency energy of the adjacent frequency points.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A communication status detection method, characterized in that, include: Determine whether a remote voice signal exists based on the remote reference signal; If so, then based on the microphone-collected signal, determine the probability of the existence of the first speech signal at the far-end canceled-tone point and the probability of the existence of the second speech signal at the adjacent frequency point of the far-end canceled-tone point; Determine the first signal energy of the far-end noise-canceling point and the second signal energy of the adjacent frequency point; The frequency energy of the far-end noise point is determined based on the probability of the first speech signal and the energy of the first signal, and the frequency energy of the adjacent frequency point is determined based on the probability of the second speech signal and the energy of the second signal. The communication status is determined based on the frequency energy of the distant anechoic point and the frequency energy of the adjacent frequency points.

2. The method according to claim 1, characterized in that, Determining the frequency energy of the far-end canceled frequency point based on the presence probability of the first speech signal and the energy of the first signal, and determining the frequency energy of the neighboring frequency point based on the presence probability of the second speech signal and the energy of the second signal, including: The first spectral energy is determined by the product of the probability of the first speech signal and the energy of the first signal, and the second spectral energy is determined by the product of the probability of the second speech signal and the energy of the second signal. The frequency energy of the far-end noise-canceling point is determined based on the first spectral energy, and the frequency energy of the adjacent frequency point is determined based on the second spectral energy.

3. The method according to claim 2, characterized in that, Determining the frequency energy of the far-end noise-canceling point based on the first spectral energy, and determining the frequency energy of the neighboring frequency point based on the second spectral energy, includes: Based on the energy of the first spectral energy of consecutive frames and the energy of the frequency point of the far-end noise cancellation point; The energy of the second spectral energy of consecutive frames and the energy of the frequency points of neighboring frequencies are determined.

4. The method according to claim 1, characterized in that, The communication status is determined based on the frequency energy of the distant noise level point and the frequency energy of neighboring frequencies, including: Determine the frequency energy ratio between the frequency energy of the far-end noise-canceling point and the frequency energy of the adjacent frequency points; The communication status is determined based on the frequency-to-energy ratio.

5. The method according to claim 4, characterized in that, Determining the communication status based on the frequency-to-energy ratio includes: Determine whether the energy ratios at the specified frequency points are all less than or equal to the energy ratio threshold; If so, then the communication status is determined to be remote one-way communication status; If not, then the communication status is determined to be dual-talk mode.

6. The method according to claim 1, characterized in that, Based on the signals collected by the microphone, the probability of the presence of a first speech signal at a far-end canceled-tone point and the probability of the presence of a second speech signal at a neighboring frequency point of the far-end canceled-tone point are determined, including: Determine whether a far-end audio signal with a far-end noise point exists in the microphone's acquired signal; If so, the prior signal-to-noise ratio of the speech at the far-end canceled audio point is determined by the noise reduction algorithm based on the far-end audio signal in the microphone acquisition signal, and the prior signal-to-noise ratio of the speech at the neighboring frequency point is determined based on the neighboring audio signal at the neighboring frequency point of the far-end canceled audio point. The probability of the presence of the first speech signal at the far-end canceled-tone point is determined based on the prior signal-to-noise ratio of the speech at the far-end canceled-tone point, and the probability of the presence of the second speech signal at the adjacent frequency point is determined based on the prior signal-to-noise ratio of the speech at the adjacent frequency point.

7. The method according to claim 1, characterized in that, Determining the presence of a remote voice signal based on a remote reference signal includes: Receive a remote reference signal and perform speech recognition on the remote reference signal to determine the speech endpoint information in the remote reference signal; The presence of a remote voice signal is determined based on the voice endpoint information.

8. A communication status detection device, characterized in that, The communication status detection device includes: The remote voice signal determination module is used to determine whether a remote voice signal exists based on a remote reference signal; The signal presence probability determination module is used to determine, if yes, the presence probability of the first voice signal at the far-end canceled-tone point and the presence probability of the second voice signal at the adjacent frequency point of the far-end canceled-tone point based on the signal collected by the microphone. A signal energy determination module is used to determine the first signal energy of the far-end noise point and the second signal energy of the adjacent frequency point; The frequency point energy determination module is used to determine the frequency point energy of the far-end noise point based on the existence probability of the first speech signal and the energy of the first signal, and to determine the frequency point energy of the neighboring frequency points based on the existence probability of the second speech signal and the energy of the second signal. The communication status determination module is used to determine the communication status based on the frequency energy of the remote anechoic point and the frequency energy of the adjacent frequency points.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images