An echo state detection method, device, computer storage medium and chip

Abstract

An embodiment of the present application provides an echo state detection method, apparatus, computer storage medium, and chip. According to the echo state detection solution provided by the embodiment of the present application, by estimating impulse response parameters and filter parameters, and acquiring a pick-up signal and a remote signal, an output signal and echo information are respectively determined, and then the energy density E1 of the output signal and the energy density E2 of the echo information are determined, so as to determine the ratio r2 of E1 to E2. When it is determined that the echo state is a double-talk state or an impulse response state, if the r2 is greater than the echo information ratio threshold, it is determined that the echo state is a double-talk state; otherwise, it is determined that the echo state is an impulse response state.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technologies, and in particular, to an echo state detection method, apparatus, computer storage medium, and chip. Background Art

[0002] In scenarios involving voice communication, since there are usually multiple parties participating, acoustic echo cancellation (AEC) is an essential technical means. Acoustic echo cancellation is to estimate the magnitude of the echo signal through an adaptive method, and then subtract this estimated value from the received signal to cancel the echo, so that the speaker at the far end will not hear his own echo. The premise of performing acoustic echo cancellation is to accurately judge the echo state of the sound signal.

[0003] In current acoustic echo cancellation, due to inaccurate discrimination of the double-talk state (that is, there is both far-end voice and near-end voice) and the impulse response state (changes in the impulse response between the near-end speaker and the near-end microphone, such as the device being moved, an obstacle suddenly appearing between the near-end speaker and the near-end microphone, etc.), AEC cannot select an accurate echo cancellation strategy, which will reduce the echo cancellation performance of AEC, resulting in the user being basically unable to hear the other party's voice when speaking simultaneously, and reducing the user experience.

[0004] Based on this, a solution for more accurately discriminating the double-talk state and the impulse response state in AEC is needed. Summary of the Invention

[0005] In view of this, embodiments of the present application provide an echo state detection solution to at least partially solve the above problems.

[0006] According to a first aspect of embodiments of the present application, an echo state detection method is provided. The method includes: obtaining a picked-up signal and a far-end signal; determining an output signal according to pre-estimated impulse response parameters, filter parameters, the picked-up signal, and the far-end signal, and determining an energy density E1 of the output signal; determining echo information as the product of the far-end signal and the filter parameters, and determining an energy density E2 of the echo information; when determining that the echo state is a double-talk state or an impulse response state, determining a ratio r2 of E1 and E2; if the r2 is greater than an echo information ratio threshold, determining that the echo state is a double-talk state; otherwise, determining that the echo state is an impulse response state.

[0007] According to the second aspect of the embodiments of the present application, an echo state detection method is provided, which is applied to the scenario of a multi-person meeting. The method includes: obtaining a pick-up signal of a local user terminal and a remote signal of other user terminals, where the other user terminals include a host terminal or other meeting participant user terminals; determining an output signal according to pre-estimated impulse response parameters, filter parameters, the pick-up signal and the remote signal, and determining an energy density E1 of the output signal; determining echo information according to the product of the remote signal and the filter parameters, and determining an energy density E2 of the echo information; when it is determined that the echo state is a double-talking state or an impulse response state, determining a ratio r2 of the E1 and the E2. If the r2 is greater than an echo information ratio threshold, determining that the echo state is a double-talking state; otherwise, determining that the echo state is an impulse response state, where the double-talking state is a state in which the user of the local user terminal and the user of the other user terminal speak simultaneously.

[0008] According to the third aspect of the embodiments of the present application, an echo state detection method is provided, which is applied to the scenario of online education. The method includes: obtaining a pick-up signal of a local user terminal and a remote signal of other user terminals, where the other user terminals include a teacher terminal or other student terminals; determining an output signal according to pre-estimated impulse response parameters, filter parameters, the pick-up signal and the remote signal, and determining an energy density E1 of the output signal; determining echo information according to the product of the remote signal and the filter parameters, and determining an energy density E2 of the echo information; when it is determined that the echo state is a double-talking state or an impulse response state, determining a ratio r2 of the E1 and the E2. If the r2 is greater than an echo information ratio threshold, determining that the echo state is a double-talking state; otherwise, determining that the echo state is an impulse response state, where the double-talking state includes a state in which the user of the local user terminal and the user of the other user terminal speak simultaneously.

[0009] According to a fourth aspect of the embodiments of the present application, there is provided an echo state detection method applied to the scenario of vehicle-mounted communication. The method includes: obtaining a pick-up signal at the vehicle-mounted device end and a remote signal corresponding to the other end that conducts voice communication with the vehicle-mounted device, where the pick-up signal at the vehicle-mounted device end includes a signal corresponding to user speech or a signal corresponding to a vehicle-mounted playback device; determining an output signal based on pre-estimated impulse response parameters, filter parameters, the pick-up signal, and the remote signal, and determining the energy density E1 of the output signal; determining echo information based on the product of the remote signal and the filter parameters, and determining the energy density E2 of the echo information; when determining that the echo state is a double-talk state or an impulse response state, determining the ratio r2 of E1 and E2, and if the r2 is greater than an echo information ratio threshold, determining that the echo state is a double-talk state, otherwise, determining that the echo state is an impulse response state, where the double-talk state includes a state where the other end and the user speak together, or a state where the other end and the vehicle-mounted device produce sound together.

[0010] According to a fifth aspect of the embodiments of the present application, there is provided an echo state detection device, including: a signal acquisition module that acquires a pick-up signal and a remote signal; an output signal determination module that determines an output signal based on pre-estimated impulse response parameters, filter parameters, the pick-up signal, and the remote signal, and determines the energy density E1 of the output signal; an echo information determination module that determines echo information based on the product of the remote signal and the filter parameters, and determines the energy density E2 of the echo information; an echo state determination module that determines the ratio r2 of E1 and E2 when determining that the echo state is a double-talk state or an impulse response state; and if the r2 is greater than an echo information ratio threshold, determining that the echo state is a double-talk state, otherwise, determining that the echo state is an impulse response state.

[0011] According to a sixth aspect of the embodiments of the present application, there is provided a chip including the echo state detection device as described above.

[0012] According to a seventh aspect of the embodiments of the present application, there is provided a computer storage medium having a computer program stored thereon, and when the program is executed by a processor, it implements the echo state detection method as described in the first aspect.

[0013] According to an eighth aspect of the embodiments of the present application, there is provided a computer program product including computer instructions, and the computer instructions direct a computing device to perform operations corresponding to the above echo state detection method.

[0014] According to the echo state detection solution provided by the embodiments of the present application, by estimating impulse response parameters and filter parameters, and acquiring a pick-up signal and a far-end signal, an output signal and echo information are respectively determined, and then the energy density E1 of the output signal and the energy density E2 of the echo information are determined, so as to determine the ratio r2 of E1 to E2. When determining that the echo state is a double-talk state or an impulse response state, if the r2 is greater than the echo information ratio threshold, it is determined that the echo state is a double-talk state; otherwise, it is determined that the echo state is an impulse response state. Since in the double-talk state, the value of r2 will become relatively large, while in the impulse response state, the value of r2 will not become very large, therefore, the double-talk state or the impulse response state can be accurately distinguished based on the pre-determined echo information ratio threshold, providing an accurate basis for strategy selection for subsequent echo cancellation, improving the echo cancellation effect, and enhancing the user experience. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0016] Figure 1 It is a schematic flowchart of a method for detecting an echo state provided by an embodiment of the present application;

[0017] Figure 2 It is a schematic diagram of an echo cancellation scenario involved in an embodiment of the present application;

[0018] Figure 3 It is a schematic flowchart of a method for distinguishing multiple echo states provided by an embodiment of the present application;

[0019] Figure 4 It is a schematic flowchart of a method for detecting an echo state provided by an embodiment of the present application;

[0020] Figure 5 It is a schematic flowchart of a method for detecting an echo state provided by an embodiment of the present application;

[0021] Figure 6 It is a schematic flowchart of a method for detecting an echo state provided by an embodiment of the present application;

[0022] Figure 7 It is a schematic diagram of an echo state detection device provided by an embodiment of the present application;

[0023] Figure 8 It is a schematic structural diagram of an electronic device provided according to an embodiment of the present application. Detailed implementation manners

[0024] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application shall fall within the protection scope of the embodiments of the present application.

[0025] In scenarios involving voice communication, different echo cancellation strategies need to be selected for different echo states. The current AEC algorithm can judge single-talk or double-talk based on the signal energy received by the proximal microphone. However, since the signal energy received by the proximal microphone will increase in both the double-talk state and the impulse response state, different processing strategies are required for AEC in these two states. If the echo state detection is inaccurate, words may be lost during double-talk in the double-talk state, and echo leakage may occur in the impulse response state, reducing the user experience. Based on this, the embodiments of the present application provide a solution for accurately distinguishing the double-talk state and the impulse response state in AEC.

[0026] The following further illustrates the specific implementation of the embodiments of the present application with reference to the accompanying drawings of the embodiments of the present application. As Figure 1 shown, Figure 1 is a schematic flowchart of an echo state detection method provided by an embodiment of the present application, and the method includes:

[0027] S101, obtain a pickup signal and a remote signal.

[0028] Referring to Figure 2 , Figure 2 is a schematic diagram of an echo cancellation scenario involved in an embodiment of the present application. This scenario may include multiple devices connected and communicating through a network. The device can be various electronic devices such as mobile phones, personal computers, or tablets, etc., which include speakers and microphones, and communicate with other similar devices (not shown in the figure) through the network.

[0029] For the device, it is usually in a fixed environment or in a fixed position, that is, the position of the speaker and microphone on the device is relatively fixed. At this time, for the device, the sound signal propagation path from the speaker to the microphone is determined, and the impulse response parameter h and the filter parameter g can be estimated based on the relevant performance of the device. That is, the impulse response parameter h is related to the sound signal propagation path from the speaker to the microphone in the environment where the device is located, and the filter parameter g always approaches the impulse response parameter h. That is, in the echo state detection module, the impulse response parameters and filter parameters can be estimated based on the environment.

[0030] At the same time, for the device, since the echo state detection module is directly connected to the far-end signal, it can directly obtain the far-end signal sf. The far-end signal is the corresponding electrical signal generated by the sound signal based on the far-end device received by the echo state detection module during the speech process. The pickup signal refers to the signal obtained by the echo state detection module through the microphone. It should be noted that the sound received by the microphone actually includes the sound generated in the speaker based on the far-end signal and the sound corresponding to the near-speaking signal sn. In other words, the pickup signal x can be expressed as follows: x = sf × h + sn.

[0031] S103, determining an output signal according to pre-estimated impulse response parameters, filter parameters, a picked-up signal and a far-end signal, and determining an energy density E1 of the output signal.

[0032] The output signal e is based on the filter parameter g to correct the sound pickup signal, that is, the output signal e is = x-sf×g, or, e=sf×(hg)+sn. Then, the energy density of the output signal e can be calculated to be E1=E{e 2}.

[0033] S105, determining echo information according to the product of the far-end signal and the filter parameter, and determining energy density E2 of the echo information.

[0034] At the same time, the echo information echo = sf × g can be obtained, and then the energy density of the echo information E2 = E{(sf × g) 2 And, if possible, a corresponding scaling factor may be added to sf or g to adjust the echo information, for example, echo = k1 × (sf k2 ×g k3 ), where k1, k2 and k3 are scaling factors determined according to actual needs.

[0035] S107. When it is determined that the echo state is the double-talk state or the impulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state; otherwise, determine that the echo state is the impulse response state.

[0036] The echo detection state can detect the echo state in advance. When it is determined that the echo state is not the single-talk state but one of the double-talk state or the impulse response state, the echo information ratio threshold and can be used for judgment.

[0037] Actually, if double-talk occurs, then there will be both the sf signal and the sn signal at this time. Then, the energy density E1 of the molecular output signal e of the corresponding r2 = E{e 2} will be relatively large, while the echo information only contains the far-end signal sf and does not contain the near-end signal. Therefore, the energy density E2 of the denominator echo information = E{(sf × g) 2} will be relatively small compared to E1. In other words, the ratio r2 of E1 and E2 at this time will become relatively large.

[0038] If impulse response occurs instead of double-talk (sn = 0, h and g change), then in such a situation, the impulse response parameter h suddenly changes (h1 → h2), which in turn causes the filter parameter to also change (g1 → g2). However, since the change in the impulse response parameter h is actually not large, the change in the filter parameter g will not be large either.

[0039] At this time, the energy density of the output signal e still only contains the far-end signal sf (sn = 0), and at the same time, the change in the echo information is very small, so the change in the energy density of the echo information is also very small. At this time, r2 = E1 / E2 can be considered to have a small oscillation and will not change greatly.

[0040] Therefore, the echo information ratio threshold can be preset based on experience. If r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state; otherwise, determine that the echo state is the impulse response state.

[0041] According to the echo state detection solution provided by the embodiments of the present application, by estimating the impulse response parameters and filter parameters, and acquiring the pickup signal and the far-end signal, the output signal and the echo information are respectively determined, and then the energy density E1 of the output signal and the energy density E2 of the echo information are determined, so as to determine the ratio r2 of E1 and E2. When determining that the echo state is the double-talk state or the impulse response state, if the r2 is greater than the echo information ratio threshold, it is determined that the echo state is the double-talk state; otherwise, it is determined that the echo state is the impulse response state. Since in the double-talk state, the value of r2 will become relatively large, while in the impulse response state, the value of r2 will not become very large, therefore, the double-talk state or the impulse response state can be accurately distinguished based on the pre-determined echo information ratio threshold, providing an accurate basis for strategy selection for subsequent echo cancellation, improving the echo cancellation effect, and enhancing the user experience.

[0042] In one embodiment, other echo state related determinations can also be performed based on the foregoing obtained related parameters. Specifically:

[0043] If a possible far-end signal is detected (i.e., it indicates that the network is connected and multi-person connection is entered, that is, as shown in Figure 2 the far-end signal sf can be received), then relevant detection of the echo state is initiated at this time.

[0044] First, it is determined that the residual echo signal is sf×(h - g). The residual echo signal characterizes the residual signal when the echo state detection module eliminates the far-end signal, and then the energy density E3 = E{(sf×(h - g)) 2} of the residual echo signal is determined, and the ratio of E1 and E3 is determined

[0045] Since e = sf×(h - g)+sn, for the two states of single talk (sn = 0) or double talk (sn>0), the change of e is relatively large, while the change of the residual echo signal sf×(h - g) is not large, which results in different value ranges of r1 corresponding to single talk or double talk.

[0046] It is easy to understand that in single talk (sn = 0), the value of e is relatively small, so the value of r1 will also be small, while in double talk (sn>0), the value of e is relatively large, so the value of r1 will also be large. Therefore, a residual echo signal ratio threshold can be preset to detect single talk or double talk of the echo state.

[0047] Specifically, an energy density threshold can be set to distinguish between the far talk state and the near talk state.

[0048] If the energy density of the far - end signal does not exceed the energy density threshold, that is, far - field speech does not occur at this time. Then, at this time, further judgment can be made based on r1. Specifically, if no one is speaking at the near - end at this time, then obviously sn = 0, and the value of r1 will be very small. If someone is speaking at the near - end at this time, then obviously sn>0. At this time, E1 is larger and E3 is smaller, resulting in a larger r1.

[0049] In other words, if the energy density of the far - end signal does not exceed the energy density threshold and the r1 does not exceed the echo residual signal ratio threshold, then the echo state is determined to be the mute state; if the energy density of the far - end signal does not exceed the energy density threshold and the r1 exceeds the echo residual signal ratio threshold, then the echo state is determined to be the near - end single - talk state.

[0050] If the energy density of the far - end signal exceeds the energy density threshold, it means that either far - field speech (including the far - end single - talk state or the double - talk state) occurs at this time, or a pulse response occurs.

[0051] Further, if r1 does not exceed the echo residual signal ratio threshold, that is, the values of the numerator part E1 and the denominator part E3 of r1 may be both large at this time (that is, the value of sf is large and sn is approximately equal to 0), resulting in a not - large value of r1, then the echo state is determined to be the far - end single - talk state. Correspondingly, if r1 exceeds the echo residual signal ratio threshold at this time, that is, the numerator part E1 of r1 is relatively large compared to the denominator E3 at this time, this may be caused by double - talk (sn>0) or may be caused by a pulse response. That is, the echo state is determined to be the double - talk state or the pulse - response state at this time.

[0052] In one embodiment, when distinguishing the double - talk state or the pulse - response state by the ratio r2 of E1 and E2, partial signals of the output signal and the echo information in the low - frequency band (for example, 30Hz - 150Hz) can also be selected to calculate the corresponding E1 and E2, and calculate the corresponding ratio r2. The pulse response does not cause much change in the low - frequency band (that is, the pulse responses h and g caused by the change of the acoustic environment are basically equal before and after the change, resulting in E2 = E{(sf×g) 2} being basically equal before and after the change). Therefore, r2 in the pulse - response state basically does not change, and since r2 significantly increases in the double - talk state, therefore, selecting the signals in the low - frequency band to calculate the ratio r2 of their energy densities for state discrimination will be more accurate.

[0053] For the specific discrimination process of various echo states (mute state, near - end single - talk state, far - end single - talk state, double - talk state, and pulse - response state) in the embodiments of the present application, refer to Figure 3 , Figure 3 which is a schematic flow chart for distinguishing multiple echo states provided by the embodiments of the present application.

[0054] According to a second aspect of the embodiments of the present application, an echo state detection method is provided, which is applied to the scenario of a multi-person meeting. Figure 4 As shown in the flowchart of an echo state detection method provided by the embodiments of the present application, the method includes:

[0055] S401, obtain the pickup signal of the local user terminal and the remote signals of other user terminals, where the other user terminals include the host terminal or other meeting participant user terminals.

[0056] In the scenario of a multi-person meeting, the host usually has the permission to turn on or off the microphone. In other words, any user terminal always receives the remote voice signal from the host terminal, and, according to the needs of the meeting, may also receive the voice signal from the speaker.

[0057] Of course, the local user terminal may also be the host terminal, and in this case, the other user terminals are other meeting participant user terminals.

[0058] S403, determine the output signal according to the pre-estimated impulse response parameters, filter parameters, the pickup signal, and the remote signals, and determine the energy density E1 of the output signal.

[0059] S405, determine the echo information according to the product of the remote signal and the filter parameters, and determine the energy density E2 of the echo information.

[0060] S407, when it is determined that the echo state is the double-talk state or the impulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state; otherwise, determine that the echo state is the impulse response state.

[0061] Wherein, the double-talk state is the state where the user of the local user terminal and the user of the other user terminal (the host corresponding to the host terminal or other meeting participants corresponding to the other meeting participant user terminals, etc.) speak simultaneously.

[0062] It should be noted that the specific implementation of the above steps S403 - S407 can refer to the description of the relevant parts in the foregoing embodiments, and will not be elaborated here.

[0063] According to the embodiments of the present application, the echo state detection solution can be applied to the meeting scenario, so as to accurately distinguish the double-talk state or the impulse response state in the meeting scenario, provide an accurate basis for policy selection for subsequent echo cancellation, improve the echo cancellation effect, and enhance the user experience.

[0064] According to a third aspect of the embodiments of the present application, another echo state detection method is provided, which is applied to the scenario of online education, such as Figure 5 As shown in

[0065] S501, obtain the pick-up signal of the local user terminal and the remote signal of other user terminals, where the other user terminals include the teacher terminal or other student terminals.

[0066] In the scenario of online education, usually the teacher terminal has the permission to turn on or off the microphone. For any student terminal, it may always receive the remote voice signal from the teacher terminal, and, according to the needs during the teaching process, it may also receive the voice signal from other student terminals.

[0067] When the local user terminal is the teacher terminal, the other user terminals may be other student terminals of any participating user, or may also be the parent terminal participating in the online education.

[0068] S503, determine the output signal according to the pulse response parameters, filter parameters, the pick-up signal and the remote signal estimated in advance, and determine the energy density E1 of the output signal.

[0069] S505, determine the echo information according to the product of the remote signal and the filter parameters, and determine the energy density E2 of the echo information.

[0070] S507, when it is determined that the echo state is the double-talk state or the pulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state, otherwise, determine that the echo state is the pulse response state.

[0071] Wherein, the double-talk state includes the state where the user of the local user terminal and the user of the other user terminal (such as the teacher of the teacher terminal or the student of the other student terminal, etc.) speak together.

[0072] It should be noted that the specific implementation of the above steps S503-S507 can refer to the description of the relevant parts in the foregoing embodiments, and will not be elaborated herein.

[0073] According to the embodiments of the present application, the echo state detection scheme can be applied to the online teaching scenario, so as to accurately distinguish the double-talk state or the pulse response state in the online teaching scenario, provide an accurate basis for policy selection for subsequent echo cancellation, improve the echo cancellation effect, and enhance the user experience.

[0074] According to a fourth aspect of the embodiments of the present application, an echo state detection method is provided, which is applied to the scenario of vehicle-mounted communication, such asFigure 6 As shown in Figure 6 FIG. Figure 6 is a schematic flowchart of an echo state detection method provided by an embodiment of the present application. The method includes:

[0075] S601, obtaining a pickup signal at the in-vehicle device end and a remote signal corresponding to the other end that communicates with the in-vehicle device by voice. The pickup signal at the in-vehicle device end includes a signal corresponding to the user's speech or a signal corresponding to the in-vehicle playback device.

[0076] When the in-vehicle device is also playing media, if someone communicates with the user through the in-vehicle device at this time, the remote signal at this time includes a signal for voice communication with the in-vehicle device, and the pickup signal will at least include a signal of the in-vehicle playback device, and, it is also possible that the pickup signal contains a signal generated when the in-vehicle user speaks.

[0077] S603, determining an output signal according to the pulse response parameters, filter parameters estimated in advance, and the pickup signal and the remote signal, and determining the energy density E1 of the output signal.

[0078] S605, determining echo information according to the product of the remote signal and the filter parameters, and determining the energy density E2 of the echo information.

[0079] S607, when determining that the echo state is a double-talk state or a pulse response state, determining the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determining that the echo state is a double-talk state; otherwise, determining that the echo state is a pulse response state.

[0080] Wherein, the double-talk state includes a state where the other end and the user speak together, or a state where the other end and the in-vehicle device make a sound together.

[0081] It should be noted that the specific implementation of the above steps S503-S507 can refer to the description of the relevant parts in the foregoing embodiments, and will not be elaborated herein.

[0082] According to the embodiment of the present application, the echo state detection scheme can be applied to the in-vehicle communication scenario, so as to accurately distinguish the double-talk state or the pulse response state in the in-vehicle communication scenario, provide an accurate basis for strategy selection for subsequent echo cancellation, improve the echo cancellation effect, and enhance the user experience.

[0083] The echo state detection method of the embodiment of the present application can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as mobile phones, PADs, etc.) and PC machines, etc.

[0084] In the fifth aspect of the embodiments of the present application, an echo state detection device is further provided, as Figure 7 shown, Figure 7 which is a schematic diagram of an echo state detection device provided by the embodiments of the present application. The device includes:

[0085] A signal acquisition module 701, which acquires a pick-up signal and a remote signal;

[0086] An output signal determination module 703, which determines an output signal according to the pulse response parameters, filter parameters, and the pick-up signal and remote signal estimated in advance, and determines the energy density E1 of the output signal;

[0087] An echo information determination module 705, which determines echo information according to the product of the remote signal and the filter parameters, and determines the energy density E2 of the echo information;

[0088] An echo state determination module 707, when determining that the echo state is a double-talk state or a pulse response state, determines the ratio r2 of E1 and E2; if the r2 is greater than the echo information ratio threshold, it determines that the echo state is a double-talk state, otherwise, it determines that the echo state is a pulse response state.

[0089] Further, the echo state determination module 707 determines an echo residual signal according to the pulse response parameters, filter parameters, and the remote signal, and determines the energy density E3 of the echo residual signal; determines the ratio r1 of E1 and E3; if the energy density of the remote signal exceeds the energy density threshold, and the r1 exceeds the echo residual signal ratio threshold, it determines that the echo state is a double-talk state or a pulse response state.

[0090] Further, the echo state determination module 707, if the energy density of the remote signal exceeds the energy density threshold, and the r1 does not exceed the echo residual signal ratio threshold, determines that the echo state is a remote single-talk state.

[0091] Further, the echo state determination module 707, if the energy density of the remote signal does not exceed the energy density threshold, and the r1 exceeds the echo residual signal ratio threshold, determines that the echo state is a proximal single-talk state.

[0092] Further, the echo state determination module 707, if the energy density of the remote signal does not exceed the energy density threshold, and the r1 does not exceed the echo residual signal ratio threshold, determines that the echo state is a mute state.

[0093] Further, the echo state determination module 707 determines the ratio r2 of E1 and E2 in the low-frequency band, and the low-frequency band includes the 30 Hz - 150 Hz band.

[0094] Further, the device further includes an echo cancellation module 709, which determines a corresponding echo cancellation strategy according to the echo state and performs echo cancellation.

[0095] The echo state detection device of this embodiment is used to implement the corresponding echo state detection method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here. In addition, the function implementation of each module in the echo state detection device of this embodiment can refer to the description of the corresponding part in the foregoing method embodiments, which will not be elaborated here either.

[0096] According to the sixth aspect of the embodiments of the present application, there is provided a chip including the echo state detection device as described in the first aspect.

[0097] According to the seventh aspect of the embodiments of the present application, there is provided a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the Figures 1 to 4 echo state detection method as described above.

[0098] According to the eighth aspect of the embodiments of the present application, there is provided a computer program product including computer instructions, and the computer instructions direct a computing device to perform operations corresponding to the foregoing echo state detection method.

[0099] The embodiments of the present application further provide an electronic device, as Figure 8 shown. Figure 8 This is a schematic structural diagram of an electronic device provided by the embodiments of the present application. The specific implementation of the electronic device is not limited in the specific embodiments of the present invention.

[0100] As Figure 8 shown, the electronic device may include: a processor 802, a communication interface 804, a memory 806, and a communication bus 808.

[0101] Among them:

[0102] The processor 802, the communication interface 804, and the memory 806 communicate with each other through the communication bus 808.

[0103] The communication interface 804 is used to communicate with other electronic devices or servers.

[0104] The processor 802 is used to execute the program 810, and specifically may execute the relevant steps in the foregoing echo state detection method embodiments.

[0105] Specifically, the program 810 may include program code, and the program code includes computer operation instructions.

[0106] The processor 802 may be a central processing unit (CPU), or a specific application integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0107] A memory 806 is used to store a program 810. The memory 806 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0108] The program 810 is specifically configured to cause the processor 802 to perform the following operations:

[0109] Obtain a pick-up signal and a remote signal;

[0110] Determine an output signal according to the pulse response parameters, filter parameters, the pick-up signal, and the remote signal obtained in advance, and determine the energy density E1 of the output signal;

[0111] Determine echo information according to the product of the remote signal and the filter parameters, and determine the energy density E2 of the echo information;

[0112] When it is determined that the echo state is a double-talk state or a pulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is a double-talk state; otherwise, determine that the echo state is a pulse response state.

[0113] For the specific implementation of each step in the program 510, reference may be made to the corresponding steps and descriptions in the units in the above-mentioned embodiments of the echo state detection method, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated here.

[0114] It should be noted that according to the needs of implementation, each component / step described in the embodiments of the present application may be split into more components / steps, or two or more components / steps or partial operations of components / steps may be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0115] The method according to the embodiments of the present application can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or be implemented as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and will be stored in a local recording medium, so that the method described herein can be stored on such a software process on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the echo state detection method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the echo state detection method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the echo state detection method shown herein.

[0116] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such an implementation should not be considered to exceed the scope of the embodiments of the present application.

[0117] The above embodiments are only used to illustrate the embodiments of the present application, rather than to limit the embodiments of the present application. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The patent protection scope of the embodiments of the present application shall be defined by the claims.

Claims

1. An echo state detection method, the method comprising: Obtaining a pick-up signal and a far-end signal; Determining an output signal according to pre-estimated impulse response parameters, filter parameters, the pick-up signal and the far-end signal, and determining the energy density E1 of the output signal; Determining echo information according to the product of the far-end signal and the filter parameters, and determining the energy density E2 of the echo information, wherein the energy density E2 is used to characterize the energy density calculated based on the echo information; When it is determined that the echo state is a double-talk state or an impulse response state, determining the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determining that the echo state is a double-talk state; otherwise, determining that the echo state is an impulse response state.

2. The method according to claim 1, wherein The echo state being a double-talk state or an impulse response state is determined by the following method: Determining an echo residual signal according to the impulse response parameters, filter parameters and the far-end signal, and determining the energy density E3 of the echo residual signal; Determining the ratio r1 of E1 and E3; If the energy density of the far-end signal exceeds the energy density threshold and the r1 exceeds the echo residual signal ratio threshold, determining that the echo state is a double-talk state or an impulse response state.

3. The method according to claim 2, further comprising: If the energy density of the far-end signal exceeds the energy density threshold and the r1 does not exceed the echo residual signal ratio threshold, then determining that the echo state is a far-end single-talk state.

4. The method according to claim 2, further comprising: If the energy density of the far-end signal does not exceed the energy density threshold and the r1 exceeds the echo residual signal ratio threshold, then determining that the echo state is a near-end single-talk state.

5. The method according to claim 2, further comprising: If the energy density of the far-end signal does not exceed the energy density threshold and the r1 does not exceed the echo residual signal ratio threshold, then determining that the echo state is a mute state.

6. The method according to claim 1, wherein, Determining the ratio r2 of E1 and E2 includes: Determining the ratio r2 of E1 and E2 in the low-frequency band, where the low-frequency band includes the 30 Hz - 150 Hz band.

7. The method according to any one of claims 1 to 6, further comprising: Determining a corresponding echo cancellation strategy according to the echo state and performing echo cancellation.

8. An echo state detection method applied to a multi-person conference scenario, the method comprising: Obtaining a pick-up signal of a local user terminal and far-end signals of other user terminals, where the other user terminals include a host terminal or other conference participant terminals; Determining an output signal according to pre-estimated impulse response parameters, filter parameters, the pick-up signal and the far-end signals, and determining the energy density E1 of the output signal; Determining echo information according to the product of the far-end signal and the filter parameters, and determining the energy density E2 of the echo information, wherein the energy density E2 is used to characterize the energy density calculated based on the echo information; When it is determined that the echo state is the double-talk state or the impulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state; otherwise, determine that the echo state is the impulse response state. Herein, the double-talk state is the state where the user of the local client and the user of the other client speak simultaneously.

9. An echo state detection method applied to the scenario of online education, the method comprising: Obtain the pick-up signal of the local client and the remote signal of the other client, where the other client includes the teacher client or other student clients; Determine the output signal according to the pre-estimated impulse response parameters, filter parameters, the pick-up signal and the remote signal, and determine the energy density E1 of the output signal; Determine the echo information according to the product of the remote signal and the filter parameters, and determine the energy density E2 of the echo information, where the energy density E2 is used to represent the energy density calculated based on the echo information; When it is determined that the echo state is the double-talk state or the impulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state; otherwise, determine that the echo state is the impulse response state. Herein, the double-talk state includes the state where the user of the local client and the user of the other client speak simultaneously.

10. An echo state detection method applied to the scenario of vehicle-mounted communication, the method comprising: Obtain the pick-up signal of the vehicle-mounted device and the remote signal corresponding to the other end that conducts voice communication with the vehicle-mounted device. The pick-up signal of the vehicle-mounted device includes the signal corresponding to the user's speech or the signal corresponding to the vehicle-mounted playback device; Determine the output signal according to the pre-estimated impulse response parameters, filter parameters, the pick-up signal and the remote signal, and determine the energy density E1 of the output signal; Determine the echo information according to the product of the remote signal and the filter parameters, and determine the energy density E2 of the echo information, where the energy density E2 is used to represent the energy density calculated based on the echo information; When it is determined that the echo state is the double-talk state or the impulse response state, determine the ratio r2 of E1 and E2. If the r2 is greater than the echo information ratio threshold, determine that the echo state is the double-talk state; otherwise, determine that the echo state is the impulse response state. Herein, the double-talk state includes the state where the other end and the user speak simultaneously, or the state where the other end and the vehicle-mounted device make sounds simultaneously.

11. An echo state detection device, comprising: A signal acquisition module, which acquires the pick-up signal and the remote signal; An output signal determination module, which determines the output signal according to the pre-estimated impulse response parameters, filter parameters, the pick-up signal and the remote signal, and determines the energy density E1 of the output signal; An echo information determination module determines echo information based on the product of the remote signal and the filter parameter, and determines the energy density E2 of the echo information, where the energy density E2 is used to characterize the energy density calculated based on the echo information. An echo state determination module determines the ratio r2 of E1 and E2 when the echo state is determined to be the double-talk state or the impulse response state; if the r2 is greater than the echo information ratio threshold, it determines that the echo state is the double-talk state, otherwise, it determines that the echo state is the impulse response state.

12. A chip comprising the echo state detection device as claimed in claim 11.

13. A computer storage medium having stored thereon a computer program which, when executed by a processor, implements the echo state detection method as claimed in any one of claims 1-10.

14. A computer program product comprising computer instructions which direct a computing device to perform operations corresponding to the echo state detection method as claimed in any one of claims 1-10.

Images