Method for detecting the device status of audio devices in audio / video conferencing and audio devices.

Abstract

An audio device is provided. The audio device includes processing circuitry connected to a speaker and a microphone. The processing circuitry is configured to play an echo reference signal from a remote location onto the speaker, and to perform acoustic echo cancellation (AEC) processing using the echo reference signal and a sound frequency signal received by the microphone via an AEC adaptive filter. The processing circuitry repeatedly determines a first state of the speaker based on the relationship between the played echo reference signal and the received sound frequency signal, and transmits a first state signal indicating the first state of the speaker to a remote location via a cloud network.

Description

TECHNICAL FIELD

[0001] The present application relates to video conferencing, and in particular, to a method for detecting a device state of an audio device in an audio / video conference and an audio device. BACKGROUND

[0002] The questions "Can you hear me?" and "What did you say?" are often asked in an audio / video conference because a speaker needs to know whether other participants are online and whether they can hear the speaker's voice. However, it is frustrating for a speaker to constantly ask these questions in an audio / video conference.

[0003] Therefore, there is a need for a method for detecting a device state in an audio / video conference and a video conference audio device to solve the above problems. SUMMARY

[0004] The following embodiments will be described in detail with reference to the accompanying drawings.

[0005] In an example embodiment, an audio device is provided. The audio device includes a processing circuit connected to a speaker and a microphone. The processing circuit is configured to play an echo reference signal from a remote on the speaker, and perform an acoustic echo cancellation (AEC) process using the echo reference signal and a sound signal received by the microphone through an AEC adaptive filter. The processing circuit repeatedly determines a first state of the speaker according to a relationship between the played echo reference signal and the received sound signal, and transmits a first state signal indicating the first state of the speaker to the remote through a cloud network.

[0006] In some embodiments, in response to the processing circuit determining that a signal level of the microphone is lower than or equal to a threshold value, the processing circuit determines that the microphone is muted. In response to the processing circuit determining that the signal level of the microphone is higher than the threshold value, the processing circuit determines a second state that the microphone is working normally, transmits a second state signal indicating the second state of the microphone to the remote through the cloud network, obtains filter coefficients from the AEC adaptive filter, and calculates a similarity of the obtained filter coefficients to reference filter coefficients.

[0007] In some embodiments, in response to the processing circuit determining that the calculated similarity is lower than a default threshold value, the processing circuit determines that the first state of the speaker is not working. In response to the processing circuit determining that the calculated similarity is higher than or equal to the default threshold value, the processing circuit determines that the first state of the speaker is that the speaker is working normally.

[0008] In some embodiments, the reference filter coefficients are calculated by using white noise and a sweep tone played on the speaker for a first predetermined time period through the AEC adaptive filter, and the calculated reference filter coefficients are pre-stored in a non-volatile memory of the audio device in a process of manufacturing the audio device in a factory.

[0009] In some embodiments, the processing circuit initializes filter coefficients of the AEC adaptive filter to zeros, and obtains filter coefficients of the AEC adaptive filter at runtime as the reference filter coefficients by calculating an average of the filter coefficients of the AEC adaptive filter over a second predetermined time period.

[0010] In some embodiments, the processing circuit calculates a cosine similarity between the filter coefficients and the reference filter coefficients as the similarity.

[0011] In some embodiments, the processing circuit receives third and fourth status signals via the cloud network, the third and fourth status signals respectively indicating a third status of a speaker of another audio device remotely and a fourth status of a microphone of the another audio device, and displays icons corresponding to the third and fourth statuses on a graphical user interface of a video conferencing application running on the video device on which the audio device is disposed.

[0012] In another exemplary embodiment, a method used in an audio device is provided. The audio device is connected to a speaker and a microphone. The method comprises the following steps: playing an echo reference signal from a remote on the speaker; performing acoustic echo cancellation (AEC) processing on the echo reference signal and a sound signal received by the microphone using an AEC adaptive filter; determining a first status of the speaker according to a relationship between the played echo reference signal and the received sound signal; and transmitting a first status signal indicating the first status of the speaker to the remote via a cloud network. BRIEF DESCRIPTION OF DRAWINGS

[0013] The present application can be more fully understood by reading the following detailed description together with the accompanying drawings, in which:

[0014] Figure 1 a block diagram showing a video conferencing system according to an embodiment of the present application;

[0015] Figure 2 a block diagram showing an audio device according to an embodiment of the present application;

[0016] Figure 3 a flowchart showing an acoustic echo cancellation (AEC) process according to an embodiment of the present application;

[0017] Figure 4 a flowchart showing a method of detecting a device status of an audio device in a video conference according to an embodiment of the present application;

[0018] Figures 5A-5B a diagram showing a graphical user interface with an illustration of different device statuses of an audio device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following description is made for the purpose of illustrating the general principles of the present application and should not be taken in a limiting sense. The scope of the present application is best determined by reference to the appended claims.

[0020] Figure 1 A block diagram of a video conference system according to an embodiment of the present application is shown.

[0021] In one embodiment, the video conference system 10 can include two or more video conference devices 100 connected to each other through a cloud network 20. Each video device 100 can be a device including display function, webcam function, speaker function, and microphone function, such as a desktop computer, a notebook computer, a smartphone, or a tablet computer equipped with a speaker and a microphone, but the present application is not limited thereto. In some embodiments, the speaker function and the microphone function in each video device 100 can be implemented by an audio device 200.

[0022] In some embodiments, each video device 100 can execute a video conference application program that presents a graphical user interface on its display. A user of each video device 100 can see the device status (e.g., including the status of the microphone and the speaker) of the audio device 200 of the other participants in the video conference through the graphical user interface.

[0023] The audio device 200 can include an acoustic echo cancellation (AEC) function in order to provide high-quality sound frequency to each person in the audio / video conference. In some embodiments, the audio device 200 can be an electronic device that simultaneously processes the speaker and microphone functions, such as a desktop audio device, a desktop audio device, a bar-shaped sound bar having a microphone array, a smartphone, a tablet PC, a laptop computer, or a personal computer equipped with a separate microphone (e.g., which can be a 3.5 mm jack microphone, a USB microphone, or a Bluetooth microphone) and a separate speaker, but the present application is not limited thereto. In some embodiments, the audio device 200 can be provided in the video device 100. In other embodiments, the audio device 200 is electrically connected to the video device 100, and the audio device 200 and the video device 100 are independent devices.

[0024] Figure 2 A block diagram of an audio device 200 according to an embodiment of the present application is shown.

[0025] In an embodiment, the audio device 200 can include a processing circuit 210, a memory 215, a digital-to-analog converter (DAC) 220, an amplifier (AMP) 230, one or more speakers 240, and one or more microphones 250. The processing circuit 210, the memory 215, the DAC 220, and the amplifier 230 can be implemented by an integrated circuit (or system on chip) 270. The processing circuit 210 can be implemented by a central processing unit (CPU), a digital-signal processor (DSP), or an application-specific integrated circuit (ASIC), a multi-processor, and / or a processor with multiple cores, but the present application is not limited thereto. The memory 215 can be a computer storage medium and can include volatile memory and non-volatile memory. The memory 215 can include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or other storage technologies.

[0026] The speaker 240 can be configured to emit a speaker signal from other audio devices 200 in the video conferencing system 10. In addition, the speaker 240 can also emit the echo reference signal 212, and the microphone 250 can receive local speech signals and other sounds from the user environment in addition to the echo reference signal. In some embodiments, the microphone 250 can include an analog-to-digital converter (ADC) (not shown in FIG. 2) to convert the received analog sound signals into discrete sound signals for subsequent AEC processing. Figure 2

[0027] The processing circuit 210 can perform AEC processing on the sound signals received by the microphone 250 (i.e., including the echo reference signal, the local speech signals, and other environmental sounds) in order to estimate the condition of the echo path from the speaker 240 to the microphone 250. In some embodiments, the AEC processing can be implemented by an AEC adaptive filter, such as a least mean squares (LMS) filter, a normalized least mean square (NLMS) adaptive filter, or other types with a predetermined number of taps, but the present application is not limited thereto.

[0028] ​In particular, when a user joins a video conference or an audio conference using the audio device 200, the positions of the loudspeaker 240 and the microphone 250 are usually fixed, and the distance between the loudspeaker 240 and the microphone 250 is also fixed. When the loudspeaker 240 and the microphone 250 are working normally, it means that the echo path from the loudspeaker 240 to the microphone 250 is effective, the coefficients of the AEC adaptive filter will converge, and will be close to the predetermined coefficients. When the loudspeaker 240 or the microphone 250 is off or not working normally, the coefficients of the AEC adaptive filter will diverge. The details of the AEC process will be described in the next section.

[0029] Figure 3 A flowchart showing a sound acoustic echo cancellation (AEC) process according to an embodiment of the present application.

[0030] With reference to Figure 3 , the processing circuit 210 can store in the memory 215 a predetermined number of input samples from a remote (e.g., other audio devices 200 in the video conference system 10), where the predetermined number of input samples can be equal to the number of taps of the AEC adaptive filter 214.

[0031] For ease of description, a normalized least mean square (NLMS) algorithm is used in the AEC adaptive filter 214 of the processing circuit 210, and the AEC adaptive filter 214 can find the filter coefficients related to the minimum normalized mean square of the error signal (e.g., the difference between the desired signal and the actual signal). For example, the echo path is an unknown system, which has a transfer function h(n) to be identified, and the AEC adaptive filter 214 tries to adjust its transfer function so that it is as close as possible to the transfer function h(n) of the echo path.

[0032] AEC adaptive filter symbol definition

[0033] In this section, the symbols used in the AEC adaptive filter 214 are defined, where: n is the number of the current input sample; p is the number of filter taps; x(n) is the echo reference signal from a remote (e.g., from other audio devices 200 in the video conference system 10), where x(n) = [x(n), x(n-1),..., x(n-p+1)] T ; y(n) is the echo reference signal received by the microphone 250 through the echo path, where y(n) = h H (n) · x(n); v(n) is the local speech frequency number (i.e., at the near end) plus the ambient sound frequency number; d(n) is the sound frequency number generated by the microphone 250, where d(n) = y(n) + v(n); is the transfer function of the AEC adaptive filter 214; is an output signal of the AEC adaptive filter and can be regarded as an estimated echo signal, wherein e(n) is a residual echo signal or error signal, wherein

[0034] In particular, the echo reference signal x(n) is a matrix of the current input sample (i.e., at time n) and (p-1) previous input samples (i.e., at times = n-1, n-2,..., n-p+1) from the remote, e.g., the other audio device 200 in the video conferencing system 10. The AEC adaptive filter 214 can compute a transfer function and an inner product of the transfer function and a Hermitian transpose of the echo reference signal x(n) results in an output signal The subtractor 216 can subtract the output signal from the sound signal d(n) to obtain a residual echo signal e(n) that is sent to the remote, e.g., the other audio device 200 in the video conferencing system 10.

[0035] In some embodiments, the transfer function of the AEC adaptive filter 214 can be regarded as a filter coefficient matrix of the AEC adaptive filter 214. Furthermore, the residual echo signal e(n) is fed back to the AEC adaptive filter 214. If the residual echo signal e(n) is large, the AEC adaptive filter 214 can significantly adjust its filter coefficients to fit the transfer function h(n) of the echo path. If the residual echo signal e(n) is small, it indicates that the filter coefficients currently used by the AEC adaptive filter 214 are close to the transfer function h(n) of the echo path, and the AEC adaptive filter 214 can adjust its filter coefficients to slightly fit the transfer function h(n) of the echo path.

[0036] In some embodiments, the AEC adaptive filter 214 can compute its transfer function at time n+1, wherein In some other embodiments, the AEC adaptive filter 214 can compute its transfer function at time n+1, wherein Thus, the AEC adaptive filter 214 can compare the transfer functions (i.e., filter coefficients) at time n+1 and time n to determine whether to adjust its filter coefficients to fit the transfer function h(n) of the echo path.

[0037] In particular, as described in the foregoing embodiments, assuming that the positions of the loudspeaker 240 and the microphone 250 are fixed, the distance between the loudspeaker 240 and the microphone 250 is also fixed. In this case, if both the loudspeaker 240 and the microphone 250 are turned on and working properly, the echo path can be quite stable. As a result, the filter coefficients of the AEC adaptive filter 214 can converge, which means that the residual echo signal e(n) can be very close to 0. In addition, if a smartphone is used as the audio device 200, the positions of the loudspeaker 240 and the microphone 250 are fixed, and the distance between the loudspeaker 240 and the microphone 250 is fixed. Therefore, if both the loudspeaker 240 and the microphone 250 are turned on and working properly, the filter coefficients of the AEC adaptive filter 214 can converge and approach the reference filter coefficients previously tested and calibrated in the factory.

[0038] However, if either the loudspeaker 240 or the microphone 250 is turned off or not working properly, the echo path can be invalid. For example, assuming that the microphone 250 is working properly while the loudspeaker 240 is turned off or not working properly, the microphone 250 will not receive the echo reference signal x(n) emitted by the loudspeaker 240. Meanwhile, the AEC adaptive filter 214 still uses the echo reference signal x(n) to generate the output signal Since there is no component y(n) in the sound frequency d(n), the difference between the sound frequency d(n) and the output signal (i.e., the residual echo signal e(n)) can be large. As a result, the AEC adaptive filter 214 can incorrectly estimate the transfer function of the echo path (i.e., the filter coefficient matrix), and this will cause the estimated filter coefficients to diverge.

[0039] In another case, assuming that the loudspeaker 240 is working properly while the microphone 250 is turned off or not working properly, the loudspeaker 240 can emit the echo reference signal x(n), but the microphone 250 will not receive any sound frequency. As a result, the sound frequency d(n) is approximately close to 0. Meanwhile, the AEC adaptive filter 214 still uses the echo reference signal x(n) to generate the output signal Since the sound frequency d(n) is approximately close to 0, the difference between the sound frequency d(n) and the output signal is considered as the residual echo signal e(n). As a result, the AEC adaptive filter 214 can incorrectly estimate the transfer function of the echo path (i.e., the filter coefficient matrix), and this will cause the estimated filter coefficients to diverge.

[0040] In some embodiments, the reference filter coefficients for the AEC adaptive filter 214 can be generated during the manufacturing process of the audio device 200 (e.g., a smartphone, a laptop, a tablet PC, a desktop audio device, etc.) with the loudspeaker 240 and the microphone 250 at fixed positions. For example, during the manufacturing process at the factory, the audio device 200 can play white noise or a sweep tone, and the processing circuit 210 of the audio device 200 can perform AEC processing at the same time. Thus, the reference filter coefficients for the AEC adaptive filter 214 can be obtained after the AEC processing is performed for a predetermined period of time, and the obtained reference filter coefficients can be stored in the non-volatile memory of the audio device 200.

[0041] In some other embodiments, the reference filter coefficients for the AEC adaptive filter 214 can be computed at runtime. For example, during an audio conference, the processing circuit 210 of the audio device 200 can automatically perform AEC processing to obtain the reference filter coefficients for the AEC adaptive filter 214. For example, the user environment can be different from the test environment at the factory, and thus the echo paths and the interference in the user environment can be different from the factory. Thus, the processing circuit 210 can automatically perform AEC processing to obtain the reference filter coefficients in response to detecting that the audio device 200 is being used in an audio or video conference. The processing circuit 210 can first set the initial filter coefficients The runtime filter coefficients for the AEC adaptive filter 214 can then be computed by computing the average of the adaptive filter coefficients for a predetermined period of time (the time when the loudspeaker 240 and the microphone 250 are normally operating).

[0042] In some other embodiments, the non-volatile memory of the audio device 200 can store the predetermined reference filter coefficients that have been tested and calibrated at the factory. However, in some cases, the predetermined reference filter coefficients can not be suitable for the user environment. When the audio device 200 is turned on, the processing circuit 210 can load the default reference filter coefficients from the non-volatile memory as the initial filter coefficients for the AEC adaptive filter 214. The processing circuit 210 can then perform AEC processing and determine whether the predetermined reference filter coefficients are suitable for the user environment. For example, the processing circuit 210 can determine whether the residual echo signal e(n) is less than a predetermined threshold to keep the updated filter coefficients converging for a predetermined period of time when detecting that the audio device 200 is being used in an audio or video conference. If the residual echo signal e(n) is less than the predetermined threshold for the predetermined period of time, the processing circuit 210 can use the default reference filter coefficients as the initial filter coefficients for the AEC adaptive filter 214. If the residual echo signal e(n) is not less than the default threshold for the predetermined period of time, the processing circuit 210 can initialize the filter coefficients i.e. all the components in the matrix are zero. Thus, the AEC adaptive filter 214 can refine the filter coefficients at runtime.

[0043] Figure 4 A flowchart of a method for detecting a device status of an audio device in an audio-video conference according to an embodiment of the present application is shown. Please refer to Figures 2 to 4 .

[0044] In step S410, it is determined whether the signal level of the microphone 250 is higher than a threshold. If it is determined that the signal level of the microphone 250 is higher than the threshold, step S420 is performed. If it is determined that the signal level of the microphone 250 is not higher than the threshold, it indicates that the microphone 250 is in a mute state (step S415) and the process ends. Meanwhile, the local audio device 200 or video device 100 can send an indication signal to the cloud network 20 to inform the remote audio device 200 or video conference apparatus 100 that the microphone 250 of the local user is muted, for example, displaying an icon of the muted microphone on a graphical user interface of a video conference application running on each video device 100 in the video conference system 10.

[0045] In step S420, the filter coefficients of the AEC adaptive filter 214 are obtained. For example, the AEC adaptive filter 214 can update its filter coefficients at runtime, and the processing circuitry 210 can repeatedly obtain the filter coefficients of the AEC adaptive filter 214 every predetermined time period.

[0046] In step S430, the similarity of the obtained filter coefficients and the reference filter coefficients is calculated. For example, the processing circuitry 210 can calculate the cosine similarity between the obtained filter coefficients and the reference filter coefficients. For example, the cosine similarity between two vectors a and b can be represented by equation (1):

[0047]

[0048] Given the obtained filter coefficients h adapt and the reference coefficients h ref , the similarity AdaptSim of the obtained filter coefficients h adapt and the reference coefficients h ref can be represented by equation (2):

[0049] AdaptSim = cos sim(h adapt , h ref ) (2)

[0050] In step S440, it is determined whether the similarity is greater than or equal to a preset threshold. If it is determined that the similarity is less than the preset threshold, it is indicated that the loudspeaker 240 is not working (step S450), and the flow ends. If it is determined that the similarity is greater than or equal to the preset threshold, it is indicated that the loudspeaker 240 and the microphone 250 are working normally (step S460), and the flow returns to step S410.

[0051] In particular, Figure 4 Steps S415, S450 and S460 in FIG. 4 can represent different device states of the audio device 200 during an audio or video conference. The processing circuit 210 of the local audio device 200 can send a status signal to the cloud network 20 to indicate the current device state of the audio device 200, and the cloud network 20 can forward the status signal to each video device 100 in the video conference system 10. Therefore, each video device 100 in the video conference system 10 can display a status icon 100 of the audio device 200 of user A on a graphical user interface of a video conference application running on each video device. If user A is speaking during the video conference, user A can know through the graphical user interface whether user B and C can hear what he or she is saying. For example, if Figure 4 The flow in FIG. 4 proceeds to step S415, and the device state of the audio device 200 indicates that the microphone 250 is muted. If Figure 4 The flow in FIG. 4 proceeds to step S450, and the device state of the audio device 200 indicates that the loudspeaker 240 is not working. If Figure 4 The flow in FIG. 4 proceeds to step S460, and the device state of the audio device 200 indicates that the loudspeaker 240 and the microphone 250 are working normally. In short, during an audio or video conference, the processing circuit 210 can repeatedly determine the first state of the loudspeaker 240 according to the relationship between the played echo reference signal and the received sound signal, and send a first status signal indicating the first state of the loudspeaker 240 to the remote via the cloud network 20. For example, the relationship between the played echo reference signal and the received sound signal can be represented using filter coefficients of an AEC adaptive filter and reference filter coefficients. In some other embodiments, the relationship between the played echo reference signal and the received sound signal can be represented using some other coefficients determined from the played echo reference signal and the received sound signal.

[0052] Figures 5A-5B A diagram showing a graphical user interface with an illustration of different device states of an audio device according to an embodiment of the present application. Please refer to Figure 2 、 Figure 4 and Figures 5A-5B .

[0053] Assuming that users A, B and C join the video conference, the video device 100 of user A can show the graphical user interface 500 including blocks 510, 520 and 530, as shown. Figure 5A For example, the block 510 can contain the username 511 (e.g., user B) of the audio device 200 of user B, a video screen 512, and blocks 513 and 514, where the block 513 shows the status of the microphone 250 of the audio device 200 of user B, and the block 514 shows the status of the speaker 240 of the audio device 200 of user B. The block 520 can contain the username 521 (e.g., user C) of the audio device 200 of user C, a video screen 522, and blocks 523 and 524 of the audio device, where the block 523 shows the status of the microphone 250 of the audio device 200 of user C, and the block 524 shows the status of the speaker 240 of the audio device 200 of user C. The block 530 can show the video screen of user A (i.e., the local user).

[0054] Referring to Figure 5A , assuming that the speaker 240 and the microphone 250 of the audio device 200 of users B and C are working properly, therefore the blocks 513 and 523 can display the microphone pattern with a specific color (e.g., green), and the speaker-status icons 514 and 524 can display the speaker pattern with a specific color (e.g., green). Thus, user A can know that the speaker 240 and the microphone 250 of the audio device 200 of users B and C are working properly through the microphone status icons 513 and 523 and the speaker status icons 514 and 524.

[0055] Referring to Figure 5B , if the audio device 200 of user B detects that the signal level of its microphone 250 is below a threshold value, the audio device 200 of user B can send a first status signal of user B to the cloud network 20, which indicates that the microphone 250 is muted, and the audio device 200 of user A can receive the status signal from the cloud network 20. Thus, the video conference application running on the video device 100 of user A can display the microphone pattern on the block 513 overlaid with a red X mark. Meanwhile, the audio device 200 of user B can determine that its speaker 240 is working properly, and send a second status signal of user B to the cloud network 20, which indicates that the speaker 240 is working properly. Thus, the audio device 200 of user A can receive the second status signal from the cloud network 20, and the video conference application running on the video device 100 of user A can display the green speaker pattern.

[0056] If the audio device 200 of user C detects that its speaker 240 is not working, using Figure 4As described in the process, user C's audio device 200 can send a second status signal to the cloud network 20, indicating that the speaker 240 is not working. User A's audio device 200 can receive user C's second status signal from the cloud network 20, and the video conferencing application running on user A's video device 100 can display a speaker icon covered with a red X mark. Simultaneously, if user C's audio device 200 detects that the signal level of its microphone 250 is higher than a threshold, user C's audio device 200 can determine that its microphone 250 is working normally and send a first status signal indicating that the microphone 250 is working normally to the cloud network 20. Therefore, user A's audio device 200 can receive user C's first status signal from the cloud network 20, and the video conferencing application running on user A's video device 100 can display a speaker icon covered with a red X mark.

[0057] Specifically, when user A speaks during an audio conference, user A can view the diagrams in blocks 513-514 and 523-524 on the graphical user interface to see if users B and C can hear what he or she is saying. Since AEC processing is a recursive finite-impulse response (FIR) filter, if any problem occurs in the echo path or AEC loop at a specific time during the audio conference, the audio devices of the remote audio device 200's processing circuitry 210 (e.g., users B and C) can determine that their speakers 240 and / or microphones 250 are not working, and the local user's (e.g., user A's) video device 100 can know the device status of the remote audio device 200 by viewing the icons in the corresponding blocks of the graphical user interface. Therefore, during an audio or video conference, the local user (e.g., user A) does not need to ask the question, "Did you hear me?"

[0058] In view of this, a method for detecting device status and an audio device for audio and video conferencing are disclosed, capable of detecting whether the speakers or microphones of a local audio device are functioning properly, and then providing the detected speaker and microphone device status to other audio or video devices in the video conferencing system. Accordingly, remote users can know the device status of the speakers and microphones of the local audio device, and local users can also know the device status of the speakers and microphones of the remote audio device, thereby improving the user experience during audio or video conferencing.

[0059] While the invention has been described by way of example and preferred embodiments, it should be understood that the invention is not limited to the disclosed embodiments. Rather, it is intended to cover various modifications and similar arrangements that will be apparent to those skilled in the art. Therefore, the appended claims should be given the broadest interpretation to cover all such modifications and similar arrangements.

Claims

1. An audio device, comprising: a processing circuit connected to a speaker and a microphone, wherein the processing circuit is configured to play an echo reference signal from a remote on the speaker, and perform sound echo cancellation processing using the echo reference signal and a sound signal received by the microphone via a sound echo cancellation adaptive filter, wherein the processing circuit repeatedly determines a first state of the speaker according to a relationship between the played echo reference signal and the received sound signal, and transmits a first state signal indicative of the first state of the speaker to the remote via a cloud network, and the relationship between the played echo reference signal and the received sound signal is represented using a plurality of filter coefficients and a plurality of reference filter coefficients of the sound echo cancellation adaptive filter.

2. The audio device of claim 1, wherein, in response to the processing circuit determining that a signal level of the microphone is lower than or equal to a threshold value, the processing circuit determines that the microphone is muted, wherein in response to the processing circuit determining that the signal level of the microphone is higher than the threshold value, the processing circuit determines a second state of the microphone as the microphone working normally, transmits a second state signal indicative of the second state to the remote via the cloud network, obtains the plurality of filter coefficients from the sound echo cancellation adaptive filter, and calculates a similarity degree between the plurality of filter coefficients and the plurality of reference filter coefficients.

3. The audio device of claim 2, wherein, in response to the processing circuit determining that the calculated similarity degree is lower than a preset threshold value, the processing circuit determines that the first state of the speaker is that the speaker is not working, wherein in response to the processing circuit determining that the calculated similarity degree is higher than or equal to a preset threshold value, the processing circuit determines that the first state of the speaker is that the speaker is working normally.

4. The audio device of claim 1, wherein, the plurality of reference filter coefficients are calculated by the sound echo cancellation adaptive filter using playing white noise and sweep tone on the speaker for a first predetermined time period, and the calculated plurality of reference filter coefficients are pre-stored in a non-volatile memory of the audio device during a process of manufacturing the audio device in a factory.

5. The audio device of claim 1, wherein, the processing circuit initializes the plurality of filter coefficients of the sound echo cancellation adaptive filter to zero, and obtains the plurality of filter coefficients of the sound echo cancellation adaptive filter as the plurality of reference filter coefficients by calculating an average value of the plurality of filter coefficients of the sound echo cancellation adaptive filter running for a second predetermined time period.

6. The audio device of claim 2, wherein, the processing circuit calculates a cosine similarity degree between the plurality of filter coefficients and the plurality of reference filter coefficients as the similarity degree.

7. The audio device of claim 1, wherein, The processing circuit receives third and fourth status signals over the cloud network, the third and fourth status signals indicating third and fourth states of a speaker and a microphone of another audio device at the remote, respectively, and displays a plurality of icons corresponding to the third and fourth states on a graphical user interface of a video conferencing application running on a video device in which the audio device is disposed.

8. A method for an audio device connected to a speaker and a microphone, comprising: playing an echo reference signal from a remote on the speaker; performing sound echo cancellation processing on the echo reference signal and a sound signal received by the microphone using a sound echo cancellation adaptive filter; determining a first state of the speaker according to a relationship between the played echo reference signal and the received sound signal; and transmitting a first status signal indicating the first state of the speaker to the remote over a cloud network, wherein the relationship between the played echo reference signal and the received sound signal is represented using a plurality of filter coefficients and a plurality of reference filter coefficients of the sound echo cancellation adaptive filter.

9. The method for connecting an audio device to a speaker and a microphone according to claim 8, wherein, The method further comprises: in response to determining that a signal level of the microphone is lower than or equal to a threshold value, determining a second state of the microphone as the microphone being muted; and in response to determining that the signal level of the microphone is higher than the threshold value, performing the following steps: determining the second state of the microphone as the microphone being in normal operation; transmitting a second status signal indicating the second state of the microphone to the remote over the cloud network; obtaining the plurality of filter coefficients from the sound echo cancellation adaptive filter; and calculating a similarity between the obtained plurality of filter coefficients and the plurality of reference filter coefficients.

10. The method for connecting an audio device to a speaker and a microphone according to claim 9, wherein, The method further comprises: in response to determining that the calculated similarity is lower than a preset threshold value, determining the first state of the speaker as the speaker being inoperative; and in response to determining that the calculated similarity is higher than or equal to a preset threshold value, determining the first state of the speaker as the speaker being in normal operation.

11. The method for connecting an audio device to a speaker and a microphone according to claim 8, wherein, The method further comprises: calculating the plurality of reference filter coefficients using playing white noise and a sweep tone on the speaker for a first predetermined time period by the sound echo cancellation adaptive filter; and pre-storing the calculated plurality of reference filter coefficients in a non-volatile memory of the audio device during a manufacturing process of the audio device at a factory.

12. The method for connecting an audio device to a speaker and a microphone according to claim 8, wherein, The method further comprises: initializing the plurality of filter coefficients of the sound echo cancellation adaptive filter to zero; and obtaining the plurality of filter coefficients of the sound echo cancellation adaptive filter as the plurality of reference filter coefficients by calculating an average of the plurality of filter coefficients of the sound echo cancellation adaptive filter running for a second predetermined time period.

13. The method for connecting an audio device to a speaker and a microphone according to claim 8, wherein, The method further comprises: calculating a cosine similarity between the plurality of filter coefficients and the plurality of reference filter coefficients as the similarity.

14. The method for connecting an audio device to a speaker and a microphone according to claim 8, wherein, The method further includes: receiving, by the cloud network, a third status signal and a fourth status signal, the third status signal and the fourth status signal respectively indicating a third status of a speaker of another audio device remotely located and a fourth status of a microphone of the another audio device; and displaying, on a graphical user interface of a video conferencing application running on a video device in which the audio device is disposed, a plurality of icons corresponding to the third status and the fourth status.

Images