Power-on status detection and processing method, device, electronic equipment and product

Abstract

The present application relates to a method, device, electronic device and product for playback detection, power-on and echo processing, wherein the sound playback detection and processing method comprises: obtaining a first audio signal detected by a pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device; detecting whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference between the first audio signal and the second audio signal; if the audio and video playback device does not play the second audio signal, playing the second audio signal through a speaker of the audio and video source output device. The embodiment of the present invention realizes intelligent switching of audio playback devices, detection of power-on status and echo cancellation processing by analyzing and judging the correlation and phase difference between the first audio signal picked up by the pickup module of the audio and video source output device and the second audio signal sent to the audio and video playback device.

Description

Technical Field

[0001] The present application relates to a power-on status detection and processing method, device, electronic equipment and product, and belongs to the field of computer technology. Background Art

[0002] In the prior art, TV boxes are generally used as audio and video source output devices, and TV devices are generally used as audio and video playback devices. Compared with TV devices, the sound source provided by the TV box may be only one of the sound sources input by the TV device. When the TV device plays signals from other sound sources, the sound channel of the TV device is occupied, and the audio signal output by the TV box to the TV device may not be played. In addition, when there is a problem with the sound signal transmission line between the TV device and the TV box, the audio signal output by the TV box to the TV device may not be played. Such a situation will seriously affect the video playback effect, and users often need to solve it through manual adjustment or equipment, which causes great inconvenience to users. Summary of the invention

[0003] The embodiment of the present invention provides a playback detection, startup, and echo processing method, device, electronic device, and product, so as to intelligently select an audio and video playback device to play a sound source signal, thereby reducing the occurrence of situations where the sound source signal cannot be played.

[0004] In order to achieve the above object, an embodiment of the present invention provides a sound playback detection processing method, comprising:

[0005] Acquire a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device;

[0006] detecting whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference between the first audio signal and the second audio signal;

[0007] If the audio and video playback device does not play the second audio signal, the second audio signal is played through a speaker of the audio and video source output device.

[0008] The embodiment of the present invention also provides a method for detecting and processing the power-on status of an audio and video playback device, comprising:

[0009] Sending a detection audio signal to an audio and video playback device to obtain a first audio signal detected by a sound pickup module of the audio and video source output device;

[0010] Detecting whether the audio and video playback device plays the detection audio signal according to the audio signal correlation and phase difference between the first audio signal and the detection audio signal;

[0011] If the audio and video playback device plays the detection audio signal, an audio and / or video content signal is output to the audio and video playback device.

[0012] The embodiment of the present invention also provides an echo cancellation processing method, including:

[0013] Acquire a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device;

[0014] Comparing signal features of the first audio signal and the second audio signal to obtain a correlated audio signal portion whose audio signal correlation is greater than a preset correlation threshold;

[0015] determining a phase difference between a relevant audio signal portion of the first audio signal and a relevant audio signal portion of the second audio signal;

[0016] The second audio signal after the phase difference is eliminated is removed from the first audio signal.

[0017] The embodiment of the present invention further provides a sound playback detection processing device, comprising:

[0018] An audio signal acquisition module, used to acquire a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device;

[0019] a play detection module, configured to detect whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference between the first audio signal and the second audio signal;

[0020] The audio playback switching processing module is used to play the second audio signal through the speaker of the audio and video source output device when it is detected that the audio and video playback device does not play the second audio signal.

[0021] The embodiment of the present invention further provides a power-on status detection and processing device for an audio and video playback device, comprising:

[0022] A detection signal sending module is used to send a detection audio signal to an audio and video playback device;

[0023] An audio signal acquisition module, used to acquire a first audio signal detected by a sound pickup module of the audio and video source output device;

[0024] a play detection module, configured to detect whether the audio and video playback device plays the detection audio signal according to the audio signal correlation and phase difference between the first audio signal and the detection audio signal;

[0025] The playback processing module is used to output audio and / or video content signals to the audio and video playback device when it is detected that the audio and video playback device plays the detection audio signal.

[0026] The embodiment of the present invention further provides an echo cancellation processing device, comprising:

[0027] An audio signal acquisition module, used to acquire a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device;

[0028] a correlation processing module, configured to compare signal features of the first audio signal and the second audio signal, and obtain a correlated audio signal portion whose audio signal correlation is greater than a preset correlation threshold;

[0029] a phase difference determination module, configured to determine a phase difference between a relevant audio signal portion of the first audio signal and a relevant audio signal portion of the second audio signal;

[0030] The echo cancellation module is used to remove the second audio signal after the phase difference is eliminated from the first audio signal.

[0031] The embodiment of the present invention also provides a projection playback processing method, including:

[0032] Output audio and video signals to the projection device;

[0033] The sound pickup module is used to detect whether the projection device plays the audio signal in the audio and video signal. If the detection determines that the projection device does not play the audio signal, the audio signal is played through the speaker of the audio and video source output device.

[0034] The embodiment of the present invention also provides a remote conference collaborative playback processing method, including:

[0035] Obtain the same-screen image signal of the remote conference and the audio and video signal of the conference site;

[0036] Comparing the video signal in the audio and video signal with the on-screen image signal to determine the audio signal corresponding to the on-screen image signal;

[0037] The same-screen image signal and the corresponding audio signal are played synchronously.

[0038] The embodiment of the present invention further provides an audio and video source output device, comprising:

[0039] An audio and video signal output module, used to output audio and video signals to an audio and video playback device;

[0040] An audio signal playing module, used to play the audio signal output to the audio and video playing device according to the instruction of the audio playing detection module;

[0041] The audio playback detection module is used to detect whether the audio playback device plays the audio signal, and if it is determined that the audio playback device does not play the audio signal, trigger the audio signal playback module to play the audio signal.

[0042] An embodiment of the present invention further provides an electronic device, including:

[0043] Memory, used to store programs;

[0044] A processor is used to run the program stored in the memory to execute any one or more of the aforementioned sound playback detection processing method, the power-on status detection processing method of the audio and video playback device, the echo cancellation processing method, the projection playback processing method, and the remote conference collaborative playback processing method.

[0045] An embodiment of the present invention also provides a computer program product, including a computer program or instructions, characterized in that when the computer program or instructions are executed by a processor, the processor is caused to implement any one or more of the aforementioned sound playback detection processing method, audio and video playback device power-on status detection processing method, echo elimination processing method, projection playback processing method, and remote conference collaborative playback processing method.

[0046] The playback detection, power-on, and echo processing methods, devices, electronic devices, and products of the embodiments of the present invention analyze and determine the correlation and phase difference between the first audio signal picked up by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device, thereby realizing intelligent switching of the audio playback device and power-on status detection and echo cancellation processing of the audio and video playback device, thereby improving the user experience of audio and video playback.

[0047] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of an application scenario of the sound playback detection and processing method according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of an application scenario of a method for turning on a television according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of an application scenario of the echo cancellation processing method according to an embodiment of the present invention;

[0051] Figure 4 A flowchart of a sound playback detection and processing method according to an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of a process flow of a power-on status detection and processing method for an audio and video playback device according to an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of a flow chart of an echo cancellation processing method according to an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the structure of a sound playback detection and processing device according to an embodiment of the present invention;

[0055] Figure 8 It is a structural schematic diagram of a power-on status detection and processing device for an audio and video playback device according to an embodiment of the present invention;

[0056] Fig. 9 It is a structural schematic diagram of an echo cancellation processing device according to an embodiment of the present invention;

[0057] Fig.10 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0059] In some application scenarios, the audio and video source output device is connected to the audio and video playback device by wired or wireless means, and provides playback content to the audio and video playback device, which is then played through the speakers of the audio and video playback device. However, if the sound channel of the audio and video playback device is occupied by other playback sources, or there is a problem with the playback content transmission line, it will cause a situation where the sound cannot be played. In response to this situation, an embodiment of the present invention provides a sound playback detection and processing method, which collects and identifies the sound played by the audio and video playback device through a sound pickup module (such as a microphone) set on the audio and video source output device, and determines whether the audio and video playback device is playing the audio signal output by the audio and video source output device. If not, it is played through the speakers on the audio and video source output device.

[0060] like Figure 1 , which is a schematic diagram of an application scenario of the sound playback detection processing method according to an embodiment of the present invention, Figure 1 The scenario shown uses a television device as an example of an audio and video playback device, and a TV box as an example of an audio and video source output device. As shown in the figure, the TV box is connected to the audio and video content provider platform through the Internet, and obtains audio and video content data from the platform. In the following, audio and video content may include audio content, video content, and a combination of audio and video content. The TV box has a system for processing audio and video sources. The processing system will obtain audio and video content data from the platform, decode it, and then output it to the TV device through the HDMI (High Definition Multimedia Interface) signal output port. The TV device and the TV box can be connected by wire or wirelessly. As an example, Figure 1 The TV device and the TV box are connected via the HDMI transmission cable.

[0061] In the embodiment of the present invention, the processing of audio signals is mainly involved. The audio signals mentioned here can be independent audio content, such as music, broadcasting, etc., or audio content combined with video, such as film and television dramas, etc. For a television device, a TV box is an audio and video source output device. Under normal circumstances, the television device obtains the HDMI signal sent by the TV box through the HDMI signal input port, and plays it through the audio and video playback processing system, wherein the audio signal is played through the speaker 1 on the television device. In addition, the TV box can also send a pure audio signal to the television device through the HDMI channel, such as playing online music, etc. In this case, the television device is only used as a device for playing audio. However, in some cases, the channel for sending audio signals to the speaker may be occupied by other audio sources, and the television device may play the video screen but not the corresponding audio, or, in the case of playing music, although the television device receives the audio signal transmitted to it by the TV box, it cannot be played, for example, the television device is playing the audio content of the internal memory, or the channel is occupied by other HDMI signal input ports.

[0062] In response to this situation, in an embodiment of the present invention, the audio signal of the speaker 1 of the TV device can be detected by using the pickup module on the TV box. If the audio signal of the speaker 1 is detected to be consistent with the audio signal sent by the TV box to the TV device, it means that the TV device is playing the audio signal emitted by the TV box normally. Otherwise, it means that the TV device is playing the audio signal sent by the TV box to the TV device, and the speaker 2 on the TV box is turned on to play the audio signal.

[0063] Specifically, it is possible to detect whether the speaker 1 of the television device plays the audio signal sent by the TV box to the television device in the following manner. Figure 1 As shown, this part of the processing can be implemented by the audio playback detection module in the figure. The audio playback detection module can be implemented by hardware and / or software, and can exist in the TV box as a system for processing audio and video sources. On the one hand, the audio playback detection module can detect the audio signal in the environment through a microphone. In order to facilitate the distinction, it is called the first audio signal here. It should be noted that the first audio signal here is an audio signal generated by receiving the sound in the environment, which may include an audio signal emitted by the speaker 1, and may also include an audio signal emitted by other devices in the environment, and may also include an audio signal emitted by the speaker 2 of the TV box itself; on the other hand, the audio playback detection module can obtain the audio signal output to the TV device from the HDMI signal output port, which is called the second audio signal here. The second audio signal is a relatively certain audio signal, and the audio signal can be read from the hardware system or software system of the TV box, but in order to calculate the accuracy of the phase difference, it is better to obtain the signal from the HDMI signal output port, so that the signal delay inside the TV box of the second audio signal can be ignored.

[0064] After acquiring the first audio signal and the second audio signal, the audio playback detection module analyzes the audio signal correlation and phase difference between the first audio signal and the second audio signal, and determines whether the TV device is playing the second audio signal output by the TV box according to the analysis result, and further determines through which device's speaker the second audio signal is played. Specifically, there may be the following situations:

[0065] Scenario 1: If the audio signal correlation of the first audio signal and the second audio signal is greater than the preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than the preset phase difference threshold, it is determined that the TV device is playing the second audio signal, and no processing is required in this case. Among them, the audio signal correlation indicates whether the first audio signal contains the components of the second audio signal, and the audio phase difference indicates the lag between the correlated audio signal components. In the case where the TV device and the TV box are connected via an HDMI line, the transmission from the TV box to the device and the TV device receiving the HDMI signal will cause a certain delay, and the sound recovery through the microphone of the TV box and then converted into an audio signal also requires a certain processing delay. These delays are much larger than the delay caused by directly broadcasting the second audio signal through the speaker of the TV box. Generally speaking, the delay between the first audio signal after recovery and the correlated audio signal component in the second audio signal output by the TV box is about a few hundred milliseconds, while the delay caused by directly broadcasting the second audio signal through the speaker of the TV box is about tens of milliseconds. Therefore, the delay can be used to distinguish which speaker is playing the sound corresponding to the second audio signal.

[0066] Scenario 2: If the audio signal correlation between the first audio signal and the second audio signal is less than the preset correlation threshold, it means that the TV device does not play the second audio signal, and the second audio signal needs to be played through the speaker of the TV box.

[0067] Scenario 3: If the audio signal correlation between the first audio signal and the second audio signal is greater than the preset correlation threshold, but the phase difference between the first audio signal and the second audio signal is less than the preset phase difference threshold, it means that the speaker of the TV box is playing the second audio signal, and no processing is required in this case. Scenario 3 occurs in the subsequent situation of Scenario 2, that is, the TV device is still in a situation where the second audio signal cannot be played normally, and the speaker of the TV box continues to play it.

[0068] Scenario 4: If the audio signal correlation between the first audio signal and the second audio signal is greater than the preset correlation threshold, and the first audio signal includes a sub-audio signal and a sub-audio signal with different phase differences, and the phase difference between one sub-audio signal and the second audio signal is greater than the preset phase difference threshold, and the phase difference between the other sub-audio signal and the second audio signal is less than the phase difference threshold, then it is determined that the speakers of the TV device and the TV box are both playing the second audio signal. In this case, the speaker of the audio and video source output device can be stopped from playing the second audio signal. Scenario 4 generally occurs when the TV device cannot play the second audio signal normally before, and switches to playing the second audio signal through the speaker of the TV box, that is, the subsequent state of the above scenario 2. However, after a period of time, the TV device can play the second audio signal normally again, resulting in a situation where the speakers of both devices are playing. In this case, it is preferred to let the speaker of the TV device play the second audio signal, so the speaker of the TV box is stopped from playing the second audio signal.

[0069] The correlation between the above-mentioned audio signals can be determined by extracting the frequency domain features of the audio signals, and then performing frequency domain analysis and comparison. Each time a detection is required, the audio playback detection module can simultaneously collect the audio signals of the HDMI signal output port and the microphone, respectively, to form multiple frames of the second audio signal and the first audio signal. Since the audio signal collected back by the microphone will have a certain delay, multiple frames of audio signals within a certain time period can be compared. For example, the audio signal is collected as a frame every 200 milliseconds, and multiple frames of the second audio signal and 5 frames of the first audio signal can be continuously sampled. For a certain frame of audio signal, it can be compared with the second audio signal of the corresponding frame of the same time period and the second audio signal of the previous frames, so that it can be determined whether the first audio signal contains an audio signal component whose correlation with the second audio signal exceeds the preset correlation threshold, thereby determining whether the audio signal collected by the microphone contains the content of the second audio signal.

[0070] The phase difference between the above-mentioned audio signals can be determined on the basis of determining the component of the second audio signal contained in the first audio signal, finding the characteristic points of the parts with a higher correlation between the first audio signal and the second audio signal, and calculating the phase difference based on these characteristic points, such as the phase difference between the peak points of the audio signal. Determining the phase difference between the audio signals also determines the delay between the audio signals, so that it can be further determined whether the component of the second audio signal contained in the first audio signal is played by the speaker of the TV device or the speaker of the TV box.

[0071] The above describes the application of the embodiments of the present invention in the detection and switching of sound playback between a TV device and a TV box. Next, the application in the TV device startup scenario is described.

[0072] like Figure 2 As shown, it is a schematic diagram of the application scenario of the processing method for turning on a TV according to an embodiment of the present invention. In the figure, a TV box is used as an example of an audio and video source output device, and a TV device is used as an example of an audio and video playback device for explanation. In the following case, the TV box can detect the user's voice command through a microphone to trigger the audio and video playback process. For example, the user issues a "play XX video" command through voice. In this case, the TV box needs to confirm whether the TV device is in a powered-on state, and then send an audio and video signal, so as to avoid the TV device not being powered on or in the power-on process, resulting in the inability to play audio and video content for a period of time, thereby affecting the user experience. The power-on operation of the TV device can be performed independently of the TV box. For example, the user can perform the power-on operation through a remote control or a voice command. In addition, since the TV box is generally directly connected to the TV device, the power-on operation of the TV device can also be implemented through the TV box, that is, the TV box can control the power-on of the TV device through a control command. In this case, after receiving the user's audio and video playback voice command, the TV box can perform a power-on status detection on the TV device, and after confirming that the TV device is turned on, send the audio and video content signal to be played. If it is detected that the TV device has not been turned on, a power-on control command can be sent to the TV device to trigger the TV to start up. In addition, after receiving the user's audio and video playback voice command, the TV box can also directly send a power-on control command to the TV device, and continuously detect whether the TV device has completed startup.

[0073] It usually takes a while for a TV device to start up. During this process, the system of the TV device is starting up, so the audio or video content sent by the TV box cannot be played. In addition, some TV devices may also play startup videos or startup ads after the system starts up, making it impossible to play the audio and video content sent by the TV box. During the time period when the TV device cannot play audio and video content, if the TV box starts to send audio and video content to the TV device, the user will miss the content played during this period. In response to this situation, an embodiment of the present invention determines whether the TV device has completed startup by detecting an audio signal sent to the TV device and performing sound capture and comparison through the microphone of the TV box.

[0074] Specifically, Figure 2As shown, the TV box sends a detection audio signal to the TV device through the HDMI signal output port, and performs audio signal detection through the microphone to generate a first audio signal. Then, based on the audio signal correlation and phase difference between the first audio signal and the power-on detection audio signal, it is detected whether the TV device plays the power-on detection audio signal. The specific detection and judgment method is the same as the detection principle in the sound playback detection processing method introduced earlier, that is, if the audio signal correlation between the first audio signal and the detection audio signal is greater than the preset correlation threshold, and the phase difference between the first audio signal and the detection audio signal is greater than the preset phase difference threshold, it can be considered that the TV device has played the detection audio signal, so that the video content signal to be played can be transmitted to the TV device, that is, Figure 2 (b). In addition, if the TV device does not play the power-on detection audio signal, for example, it may be that the TV device has not completed startup, such as Figure 2 (a) The speaker of the TV device does not play any sound or plays the sound of the startup video or advertisement other than the detection audio. In this case, it is necessary to send the startup detection audio signal to the TV device again and perform a new round of detection processing to determine whether the TV device plays the startup detection audio signal. A certain time interval may be provided between each round of detection processing. Through such continuous repeated detection, the video playback can be automatically started after the TV device is turned on, and the user will not miss part of the audio or video content due to the startup process of the TV device.

[0075] Next, we will introduce another application scenario using the technical principle of the above sound playback detection. Figure 3As shown, it is a schematic diagram of the application scenario of the echo cancellation processing method of the embodiment of the present invention. Still taking the TV box as an example of the audio and video source output device, and the TV device as an example of the audio and video playback device. The echo cancellation processing method is applied to the situation of recognizing the user's voice command. As shown in the figure, the TV box transmits the audio and video content signal to the TV device through the HDMI signal output port. For the convenience of description, the audio part thereof is called the second audio signal. The second audio signal will be played through the speaker 1 of the TV device. In order to distinguish it from the second audio signal output by the original TV box (there is a phase difference), the audio played by the speaker 1 is called the content audio. On the other hand, the user may issue a voice command to the TV box, and the voice command is mixed with the content audio played by the speaker 1 in the environment to form a mixed audio. After the mixed audio is collected by the microphone of the TV box, a first audio signal is formed. That is, when the user issues a voice command, the first audio signal includes the second audio signal played after the delay and the voice command issued by the user. The echo cancellation processing method of the embodiment of the present invention is to eliminate the second audio signal from the first audio signal, so that a relatively pure user voice command can be obtained for precise control processing.

[0076] The echo cancellation method is still implemented by utilizing the characteristics of the audio signal correlation and phase difference between the first audio signal and the second audio signal. Specifically, the echo cancellation process can be implemented by the echo cancellation module in the figure. Its basic principle and Figure 1 Similar to the situation shown, the echo cancellation module can obtain the first audio signal from the microphone, and obtain the second audio signal sent by the TV box to the TV device from the HDMI signal output port. After that, the signal characteristics of the first audio signal and the second audio signal are compared, and the relevant audio signal part whose audio signal correlation is greater than the preset correlation threshold is obtained, and the phase difference between the relevant audio signal part of the first audio signal and the relevant audio signal part of the second audio signal is determined, and then according to the determined phase difference, the second audio signal after the phase difference is eliminated is removed from the first audio signal. The process of eliminating the phase difference mentioned here can be specifically to align the signals on the time axis according to the determined phase difference. For example, the determined phase difference is 100ms, that is, the relevant component in the first audio signal lags behind the second audio signal by 100ms, then the current first audio signal can be aligned with the second audio signal 100ms ago and then subtracted, so that the component corresponding to the second audio signal can be eliminated.

[0077] The above calculation process for determining the phase difference does not need to be performed each time the echo cancellation process is performed, and can be performed at a predetermined time interval, that is, after calculating the phase difference once, the phase difference can be used to perform echo cancellation within this time interval. Since the transmission delay of the HDMI signal may fluctuate, in practical applications, a reasonable time interval can be determined based on the delay fluctuation of the HDMI signal. In addition, in some cases, the above-mentioned second audio signal may be played through the speaker 2 of the TV box itself. The principle of echo processing for this case is the same as that described above, except that the determined phase difference will be relatively small.

[0078] The technical solution of the present invention is further illustrated by some specific embodiments below.

[0079] Embodiment 1

[0080] like Figure 4 As shown, it is a flow chart of the sound playback detection and processing method of an embodiment of the present invention. The method can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module (such as a microphone) and a speaker for collecting audio signals. The audio and video source output device is connected to an audio and video playback device such as a TV device through a wired or wireless method to transmit audio and video signals. Specifically, the method includes:

[0081] S101: Obtain a first audio signal detected by a sound pickup module of an audio and video source output device and a second audio signal sent by the audio and video source output device to an audio and video playback device. Among them, the first audio signal is an audio signal generated by collecting the sound in the environment through the sound pickup module of the audio and video source output device, which may include audio signals sent by the audio and video playback device, and may also include audio signals sent by other devices in the environment, and may also include audio signals sent by the speakers of the audio and video source output device itself. The second audio signal is the audio content sent by the audio and video source output device to the audio and video playback device, and is expected to be played by the audio and video playback device. Compared with the first audio signal, the second audio signal is a definite signal. In the case where the audio and video source output device is a TV box, the second audio signal can be obtained from the HDMI output port of the audio and video source output device.

[0082] S102: Detecting whether the audio and video playback device plays the second audio signal based on the audio signal correlation and phase difference between the first audio signal and the second audio signal. Specifically, the determination method may include: if the audio signal correlation between the first audio signal and the second audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than a preset phase difference threshold, then determining that the audio and video playback device plays the second audio signal; otherwise, determining that the audio and video playback device does not play the second audio signal.

[0083] Specifically, the correlation between the above-mentioned audio signals can be determined by extracting the frequency domain features of the audio signals and then performing frequency domain analysis and comparison. The phase difference between the audio signals can be determined based on determining the component of the first audio signal that contains the second audio signal, finding the feature points of the parts with higher correlation between the first audio signal and the second audio signal, and calculating the phase difference based on these feature points, such as the phase difference between the peak points of the audio signals.

[0084] Furthermore, when the audio / video playback device does not play the second audio signal, there may be the following specific situations: if the audio signal correlation between the first audio signal and the second audio signal is less than a preset correlation threshold, it means that the first audio signal does not contain a component related to the second audio signal, and of course it also means that the audio / video playback device does not play the second audio signal;

[0085] If the audio signal correlation between the first audio signal and the second audio signal is greater than the preset correlation threshold, but the phase difference between the first audio signal and the second audio signal is less than the preset phase difference threshold, this means that the first audio signal collected by the sound pickup module of the audio and video source output device contains the relevant components of the second audio signal, but the relevant components are not emitted by the audio and video playback device, but by the speaker of the audio and video source output device itself, so the phase difference is smaller.

[0086] If the audio signal correlation between the first audio signal and the second audio signal is greater than a preset correlation threshold, and the first audio signal includes a sub-audio signal and a sub-audio signal with different phase differences, wherein the phase difference between one sub-audio signal and the second audio signal is greater than a preset phase difference threshold, and the phase difference between the other sub-audio signal and the second audio signal is less than the phase difference threshold, it indicates that both the audio and video playback device and the audio and video source output device are playing the second audio signal.

[0087] S103: If the audio and video playback device does not play the second audio signal, the second audio signal is played through the speaker of the audio and video source output device. If the audio and video playback device plays the second audio signal, it indicates that the audio and video playback device is in a normal state and does not need to be switched.

[0088] In addition, if it is detected that the audio and video playback device is not playing the second audio signal, but the speaker of the audio and video source output device is playing the second audio signal, it means that the audio and video playback device has not yet returned to normal, and the speaker of the audio and video source output device continues to play the second audio signal. Therefore, in this case, no processing is required. If the speakers of the audio and video playback device and the audio and video source output device are both playing the second audio signal, the speaker of the audio and video source output device should stop playing the second audio signal.

[0089] The above-mentioned sound playback detection processing can be repeatedly executed in multiple rounds at certain time intervals, so as to continuously adjust the device playing the second audio signal according to the state of the audio and video playback device, thereby ensuring that the user can always hear the audio content output by the audio and video source output device without being affected by the state of the audio and video playback device.

[0090] Embodiment 2

[0091] like Figure 5 As shown, it is a flow chart of the power-on status detection and processing method of the audio and video playback device of an embodiment of the present invention, and the method can be applied to audio and video source output devices such as TV boxes. The audio and video source output device itself has a sound pickup module for collecting audio signals, and the audio and video source output device is connected to an audio and video playback device such as a TV device by a wired or wireless method to transmit audio and video signals. In the application scenario of this embodiment, when the audio and video source output device needs to transmit audio and video playback content to the audio and video playback device, it is hoped to first confirm whether the audio and video playback device is in a powered-on state, and then send the audio and video signal, so as to avoid the TV device not being powered on or in the process of powering on, resulting in the inability to play the audio and video content transmitted by the audio and video source output device for a period of time, thereby affecting the user experience. Specifically, the method includes:

[0092] S201: Send a detection audio signal to the audio and video playback device, and obtain the first audio signal detected by the sound pickup module of the audio and video source output device. The detection audio signal can be a simple audio content, such as "Wonderful content will be played for you soon". If the audio and video playback device is in a normal power-on state, this audio segment should be played quickly, and the audio and video source output device can collect the first audio signal containing this audio segment through the sound pickup module. In addition, the audio and video source output device can also control the audio and video playback device through a control protocol, such as the audio and video source output device can control the power on and off of the audio and video playback device through a control instruction. The audio and video source output device can also have the ability to detect the user's voice command through the sound pickup module and trigger the audio and video playback processing. Therefore, in some application scenarios, the user can trigger the audio and video source output device to initiate audio and video playback through a voice command, and the audio and video playback device is still in a power-off state at this time. For this situation, before sending the detection audio signal to the audio and video playback device, the audio and video source output device can also send a power-on instruction to the audio and video source output device.

[0093] S202: Detect whether the audio and video playback device plays the detection audio signal according to the audio signal correlation and phase difference between the first audio signal and the detection audio signal. The specific detection and determination method is the same as the detection principle in the sound playback detection processing method introduced above, that is, if the audio signal correlation between the first audio signal and the detection audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the detection audio signal is greater than a preset phase difference threshold, it is determined that the audio and video playback device plays the detection audio signal, otherwise it is determined that the audio and video playback device does not play the detection audio signal.

[0094] S203: If the audio and video playback device plays the detection audio signal, then the audio and / or video content signal is output to the audio and video playback device. If the television device does not play the detection audio signal, then the detection audio signal is sent to the television device again, and a new round of processing is performed to detect whether the television device plays the detection audio signal, until a predetermined round is detected or the audio and video playback device is detected to play the detection audio signal. In addition, the reason why the audio and video playback device does not play the detection audio signal may be because the audio and video playback device is not turned on. Therefore, when the audio and video playback device is not detected to play the detection audio signal, a power-on instruction can be sent to the audio and video playback device to trigger it to perform the power-on process.

[0095] The power-on status detection and processing method of the audio and video playback device of the embodiment of the present invention sends a detection audio signal to the audio and video playback device, then uses the sound pickup module to collect sound, and determines whether the collected audio signal contains the detection audio signal according to the audio signal correlation and phase difference, thereby realizing the detection of the power-on status of the audio and video playback device. In this way, it can be avoided that the user misses the audio and video playback content for a period of time due to the audio and video playback device not being turned on or being in the process of being turned on.

[0096] Embodiment 3

[0097] like Figure 6 As shown, it is a flow chart of the echo cancellation processing method of an embodiment of the present invention, and the method can be applied to audio and video source output devices such as TV boxes. The audio and video source output device itself has a sound pickup module for collecting audio signals, and can receive the user's voice commands through the sound pickup module. The audio and video source output device is connected to an audio and video playback device such as a TV device through a wired or wireless method to transmit audio and video signals. In some scenarios, when a user issues a voice command to the audio and video source output device, the audio and video playback device may be playing audio and video content, and these sounds will interfere with the recognition of the voice commands. The echo cancellation processing method provided in the embodiment of the present invention is to eliminate these interferences. Specifically, the method includes:

[0098] S301: Obtain the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device. The second audio signal corresponds to the audio content sent by the audio and video source output device to the audio and video playback device, that is, the audio content that the speaker of the audio and video playback device needs to play, and the first audio signal is an audio signal collected by the sound pickup module of the audio and video source output device, which includes the second audio signal played by the speaker of the audio and video playback device after a delay, and when the user issues a voice command, it may also include the voice command issued by the user. In addition, in some cases, the above-mentioned second audio signal may be played through the speaker of the audio and video source output device itself, so that the first audio signal includes the second audio signal played by the speaker of the audio and video source output device after a delay, and when the user issues a voice command, it may also include the voice command issued by the user. The difference between these two situations is that the delay of playing the second audio signal will be different.

[0099] S302: Compare the signal features of the first audio signal and the second audio signal to obtain a correlated audio signal portion whose audio signal correlation is greater than a preset correlation threshold. The fact that the audio signal correlation is greater than the preset correlation threshold indicates that the first audio signal contains the content of the second audio signal. The audio signal correlation between the above audio signals can be determined by extracting the frequency domain features of the audio signals and then performing frequency domain analysis and comparison.

[0100] S303: Determine the phase difference between the relevant audio signal part of the first audio signal and the relevant audio signal part of the second audio signal. The phase difference between the above audio signals can be based on determining the component of the first audio signal that contains the second audio signal, finding the feature points of the audio signal parts with higher correlation in the first audio signal and the second audio signal, and calculating the phase difference based on these feature points, for example, the phase difference between the peak points of the relevant audio signal parts.

[0101] S304: removing the second audio signal after eliminating the phase difference from the first audio signal. The process of eliminating the phase difference mentioned here can be specifically to align the signals on the time axis according to the determined phase difference. For example, if the determined phase difference is 100ms, that is, the relevant component in the first audio signal lags behind the second audio signal by 100ms, then the current first audio signal can be aligned with the second audio signal 100ms ago and then subtracted, so that the component corresponding to the second audio signal can be eliminated.

[0102] The above process of determining the phase difference may be performed at a preset time interval, that is, the processes of steps S302 and S303 are performed at a preset time interval. Accordingly, within the time interval, in step S304, the second audio signal after the phase difference is eliminated is removed from the first audio signal with the latest determined phase difference.

[0103] In addition, determining the phase difference between the audio signals also determines the delay between the audio signals, so that it can be further judged whether the component of the second audio signal contained in the first audio signal is played by the speaker of the audio and video playback device or the speaker of the audio and video source output device. However, in the scenario of the embodiment of the present invention, as long as the phase difference is determined, echo cancellation processing can be performed without distinguishing whether the audio is currently being played by the audio and video playback device or the audio and video source output device.

[0104] The echo cancellation processing method of the embodiment of the present invention calculates the phase difference between the first audio signal picked up by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device, and then uses the phase difference to perform echo cancellation, thereby shielding the influence of the second audio signal output by the audio and video source output device, so that relatively pure voice commands can be extracted, and accurate voice control processing can be performed.

[0105] Embodiment 4

[0106] like Figure 7 As shown, it is a structural schematic diagram of a sound playback detection and processing device according to an embodiment of the present invention. The device can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module and a speaker for collecting audio signals. The audio and video source output device is connected to an audio and video playback device such as a TV device by wired or wireless means to transmit audio and video signals. Specifically, the device includes:

[0107] The audio signal acquisition module 11 is used to acquire the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device. Among them, the first audio signal is an audio signal generated by collecting the sound in the environment through the sound pickup module of the audio and video source output device, which may include audio signals sent by the audio and video playback device, and may also include audio signals sent by other devices in the environment, and may also include audio signals sent by the speakers of the audio and video source output device itself. The second audio signal is the audio content sent by the audio and video source output device to the audio and video playback device, and is expected to be played by the audio and video playback device. Compared with the first audio signal, the second audio signal is a definite signal. In the case where the audio and video source output device is a TV box, the second audio signal can be obtained from the HDMI output port of the audio and video source output device.

[0108] The play detection module 12 is used to detect whether the audio and video playback device plays the second audio signal according to the audio signal correlation and phase difference between the first audio signal and the second audio signal. The specific determination method may include: if the audio signal correlation between the first audio signal and the second audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the second audio signal is greater than a preset phase difference threshold, then it is determined that the audio and video playback device plays the second audio signal; otherwise, it is determined that the audio and video playback device does not play the second audio signal.

[0109] Specifically, the correlation between the above-mentioned audio signals can be determined by extracting the frequency domain features of the audio signals and then performing frequency domain analysis and comparison. The phase difference between the audio signals can be determined based on determining the component of the first audio signal that contains the second audio signal, finding the feature points of the parts with higher correlation between the first audio signal and the second audio signal, and calculating the phase difference based on these feature points, such as the phase difference between the peak points of the audio signals.

[0110] Furthermore, when the audio / video playback device does not play the second audio signal, there may be the following specific situations: if the audio signal correlation between the first audio signal and the second audio signal is less than a preset correlation threshold, it means that the first audio signal does not contain a component related to the second audio signal, and of course it also means that the audio / video playback device does not play the second audio signal;

[0111] If the audio signal correlation between the first audio signal and the second audio signal is greater than the preset correlation threshold, but the phase difference between the first audio signal and the second audio signal is less than the preset phase difference threshold, this means that the first audio signal collected by the sound pickup module of the audio and video source output device contains the relevant components of the second audio signal, but the relevant components are not emitted by the audio and video playback device, but by the speaker of the audio and video source output device itself, so the phase difference is smaller.

[0112] If the audio signal correlation between the first audio signal and the second audio signal is greater than a preset correlation threshold, and the first audio signal includes a sub-audio signal and a sub-audio signal with different phase differences, wherein the phase difference between one sub-audio signal and the second audio signal is greater than a preset phase difference threshold, and the phase difference between the other sub-audio signal and the second audio signal is less than the phase difference threshold, it indicates that both the audio and video playback device and the audio and video source output device are playing the second audio signal.

[0113] The audio playback switching processing module 13 is used to play the second audio signal through the speaker of the audio and video source output device when it is detected that the audio and video playback device does not play the second audio signal. When it is detected that the audio and video playback device plays the second audio signal, it means that the audio and video playback device is in a normal state and does not need to be switched.

[0114] In addition, if it is detected that the audio and video playback device is not playing the second audio signal, but the speaker of the audio and video source output device is playing the second audio signal, it means that the audio and video playback device has not yet returned to normal, and the speaker of the audio and video source output device continues to play the second audio signal. Therefore, in this case, no processing is required. If the speakers of the audio and video playback device and the audio and video source output device are both playing the second audio signal, the speaker of the audio and video source output device should stop playing the second audio signal.

[0115] The above-mentioned sound playback detection processing can be repeatedly executed in multiple rounds at certain time intervals, so as to continuously adjust the device playing the second audio signal according to the state of the audio and video playback device, thereby ensuring that the user can always hear the audio content output by the audio and video source output device without being affected by the state of the audio and video playback device.

[0116] Embodiment 5

[0117] like Figure 8 As shown, it is a structural schematic diagram of the power-on status detection and processing device of the audio and video playback device of an embodiment of the present invention, and the device can be applied to audio and video source output devices such as TV boxes. The audio and video source output device itself has a pickup module for collecting audio signals, and the audio and video source output device is connected to an audio and video playback device such as a TV device by a wired or wireless method to transmit audio and video signals. In the application scenario of this embodiment, when the audio and video source output device needs to transmit audio and video playback content to the audio and video playback device, it is hoped to first confirm whether the audio and video playback device is in a powered-on state, and then send the audio and video signal, so as to avoid the TV device not being powered on or in the process of powering on, resulting in the inability to play the audio and video content transmitted by the audio and video source output device for a period of time, thereby affecting the user experience. Specifically, the device includes:

[0118] The detection signal sending module 21 is used to send a detection audio signal to the audio and video playback device. The detection audio signal can be a simple audio content, such as "Wonderful content will be played for you soon". If the audio and video playback device is in a normal power-on state, this audio should be played soon, and the audio and video source output device can collect the first audio signal containing this audio through the sound pickup module. In addition, the audio and video source output device can also control the audio and video playback device through a control protocol, such as the audio and video source output device can control the power on and off of the audio and video playback device through a control instruction. The audio and video source output device can also have a voice command of the user detected by the sound pickup module to trigger the audio and video playback processing capability. Therefore, in some application scenarios, it can be that the user triggers the audio and video source output device to initiate audio and video playback through a voice command, and the audio and video playback device is still in a power-off state. For this situation, the detection signal sending module 21 can also include the audio and video source output device sending a power-on instruction to the audio and video playback device before sending the detection audio signal to the audio and video playback device.

[0119] The audio signal acquisition module 22 is used to acquire a first audio signal detected by a sound pickup module of the audio and video source output device. The first audio signal is an audio signal generated by collecting sounds in the environment through the sound pickup module of the audio and video source output device, which may include an audio signal emitted by an audio and video playback device, may also include audio signals emitted by other devices in the environment, and may also include audio signals emitted by a speaker of the audio and video source output device itself.

[0120] The play detection module 23 is used to detect whether the audio and video playback device plays the detection audio signal according to the audio signal correlation and phase difference between the first audio signal and the detection audio signal. The specific detection and determination method is the same as the detection principle in the sound playback detection processing method and the device introduced above, that is, if the audio signal correlation between the first audio signal and the detection audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the detection audio signal is greater than a preset phase difference threshold, it is determined that the audio and video playback device plays the detection audio signal, otherwise it is determined that the audio and video playback device does not play the detection audio signal.

[0121] The playback processing module 24 is used to output audio and / or video content signals to the audio and video playback device when it is detected that the audio and video playback device plays the detection audio signal. In addition, if the television device does not play the detection audio signal, the detection audio signal is sent to the television device again, and the power-on status detection processing device is triggered to perform a new round of processing to detect whether the television device plays the detection audio signal, until a predetermined round is detected or the audio and video playback device is detected to play the detection audio signal. In addition, the reason why the audio and video playback device does not play the detection audio signal may be because the audio and video playback device is not turned on. Therefore, when it is not detected that the audio and video playback device plays the detection audio signal, a power-on instruction can be sent to the audio and video playback device to trigger it to perform the power-on processing.

[0122] The power-on status detection and processing device of the audio and video playback device of the embodiment of the present invention sends a detection audio signal to the audio and video playback device, then uses the sound pickup module to collect sound, and determines whether the collected audio signal contains the detection audio signal according to the audio signal correlation and phase difference, thereby realizing the detection of the power-on status of the audio and video playback device. In this way, it can be avoided that the user misses the audio and video playback content for a period of time due to the audio and video playback device not being turned on or being in the process of being turned on.

[0123] Embodiment 6

[0124] like Fig. 9 As shown, it is a schematic diagram of the structure of the echo cancellation processing device of an embodiment of the present invention, and the device can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module for collecting audio signals, and can receive the user's voice commands through the pickup module. The audio and video source output device is connected to an audio and video playback device such as a TV device through a wired or wireless method to transmit audio and video signals. In some scenarios, when a user issues a voice command to the audio and video source output device, the audio and video playback device may be playing audio and video content, and these sounds will interfere with the recognition of the voice commands. The echo cancellation processing device provided in an embodiment of the present invention is to eliminate these interferences. Specifically, the device includes:

[0125] The audio signal acquisition module 31 is used to acquire the first audio signal detected by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device. Among them, the second audio signal corresponds to the audio content sent by the audio and video source output device to the audio and video playback device, that is, the audio content that the speaker of the audio and video playback device needs to play, and the first audio signal is an audio signal collected by the sound pickup module of the audio and video source output device, which includes the second audio signal played by the speaker of the audio and video playback device after a delay, and when the user issues a voice command, it may also include the voice command issued by the user. In addition, in some cases, the above-mentioned second audio signal may be played through the speaker of the audio and video source output device itself, so that the first audio signal includes the second audio signal played by the speaker of the audio and video source output device after a delay, and when the user issues a voice command, it may also include the voice command issued by the user. The difference between these two situations is that the delay of playing the second audio signal will be different.

[0126] The correlation processing module 32 is used to compare the signal features of the first audio signal and the second audio signal, and obtain the correlated audio signal part whose audio signal correlation is greater than a preset correlation threshold. The fact that the audio signal correlation is greater than the preset correlation threshold indicates that the first audio signal contains the content of the second audio signal. The audio signal correlation between the above audio signals can be determined by extracting the frequency domain features of the audio signals and then performing frequency domain analysis and comparison.

[0127] The phase difference determination module 33 is used to determine the phase difference between the relevant audio signal part of the first audio signal and the relevant audio signal part of the second audio signal. The phase difference between the above audio signals can be calculated based on the determination of the component of the first audio signal containing the second audio signal, finding the feature points of the audio signal parts with higher correlation in the first audio signal and the second audio signal, and calculating the phase difference according to these feature points, for example, the phase difference between the peak points of the relevant audio signal parts.

[0128] The echo cancellation module 34 is used to remove the second audio signal after the phase difference is eliminated from the first audio signal. The process of eliminating the phase difference mentioned here can be specifically to align the signals on the time axis according to the determined phase difference. For example, if the determined phase difference is 100ms, that is, the relevant component in the first audio signal lags behind the second audio signal by 100ms, then the current first audio signal can be aligned with the second audio signal 100ms ago and then subtracted, so that the component corresponding to the second audio signal can be eliminated.

[0129] The above-mentioned process of determining the phase difference can be executed at a preset time interval, that is, the process performed by the correlation processing module 32 and the phase difference determination module 33 is executed at a preset time interval, and accordingly, within the time interval, the echo cancellation module 34 performs the process of removing the second audio signal after the phase difference is eliminated from the first audio signal with the latest determined phase difference.

[0130] In addition, determining the phase difference between the audio signals also determines the delay between the audio signals, so that it can be further judged whether the component of the second audio signal contained in the first audio signal is played by the speaker of the audio and video playback device or the speaker of the audio and video source output device. However, in the scenario of the embodiment of the present invention, as long as the phase difference is determined, echo cancellation processing can be performed without distinguishing whether the audio is currently being played by the audio and video playback device or the audio and video source output device.

[0131] The echo cancellation processing device of the embodiment of the present invention calculates the phase difference between the first audio signal picked up by the sound pickup module of the audio and video source output device and the second audio signal sent by the audio and video source output device to the audio and video playback device, and then uses the phase difference to perform echo cancellation, thereby shielding the influence of the second audio signal output by the audio and video source output device, thereby extracting relatively pure voice commands and performing accurate voice control processing.

[0132] Embodiment 7

[0133] This embodiment involves the application of the technical solution of the embodiment of the present invention in some scenarios to achieve the coordinated playback of audio and video. For example, in some multi-person meetings or education and training scenarios, video images are often played through projection devices. Some projection devices may have built-in audio playback functions, while some projection devices may only have image playback functions. In addition, for projection devices with audio playback functions, audio may not be played normally due to signal transmission failures or device settings.

[0134] In view of this situation, this embodiment provides a projection playback processing method, which can be applied to an audio and video source output device such as a TV box. The audio and video source output device itself has a pickup module (such as a microphone) for collecting audio signals. The audio and video source output device is connected to the projection device by wired or wireless means to transmit audio and video signals. Specifically, the method includes: the audio and video source output device outputs audio and video signals to the projection device; the projection device is detected by the pickup module whether the audio signal in the audio and video signal is played, if the detection determines that the projection device does not play the audio signal, the audio signal is played through the speaker of the audio and video source output device, if the detection determines that the projection device plays the audio signal normally, the projection device is used to play the audio signal first.

[0135] The method for detecting whether the projection device plays audio signals can adopt the method introduced in the above-mentioned embodiment, that is, the audio signals in the environment are collected by the sound pickup module, and then the correlation and phase difference between the signals are analyzed and judged to determine whether the projection device plays the audio output by the audio and video source output device.

[0136] In addition, in remote conference scenarios, on the one hand, the shared processing mechanism based on the cloud platform can realize remote same-screen image display, and on the other hand, the audio and video signals at the conference site can also be transmitted to the devices at the remote conference site through the network. The requirements of the remote conference site are: the video playback picture is highlighted on the same screen, the audio playback content is the audio signal transmitted at the conference site, and it is hoped that the two can be played in coordination as much as possible.

[0137] To this end, an embodiment of the present invention provides a remote conference collaborative playback processing method, which can be applied to an audio and video source output device such as a TV box, the audio and video source output device itself has a pickup module (such as a microphone) for collecting audio signals, and the audio and video source output device is connected to the projection device or large display device at the remote conference site by wired or wireless means, so as to transmit image signals and audio signals. It includes:

[0138] S401: Obtain the same-screen image signal of the remote conference and the audio and video signal of the conference site. In the remote conference venue, the same-screen image signal of the conference site and the audio and video signal of the conference site can be received through the network through audio and video source output devices such as TV boxes. Among them, the same-screen image signal can come from the cloud platform, that is, the conference host at the conference site shares the plane based on the office system of the cloud platform, thereby generating the same-screen image signal on the cloud platform and transmitting it to the audio and video source output device of the remote conference venue. On the other hand, recording and video recording equipment can be set up at the conference site, and audio and video signals can be synchronously transmitted to the audio and video source output device of the remote conference venue.

[0139] S402: Compare the video signal in the audio and video signal with the image signal on the same screen to determine the audio signal corresponding to the image signal on the same screen. In the audio and video source output device of the remote conference site, the synchronization between the video signal shot at the conference site and the image signal on the same screen can be determined by comparing and analyzing the two signals. Since the audio signal and video signal recorded at the conference site are corresponding, after determining the synchronization between the video signal and the image signal on the same screen, the synchronization state of the audio signal is also indirectly determined. That is, in this embodiment, the video signal shot at the site is mainly used to determine the corresponding relationship between the audio signal and the image signal on the same screen.

[0140] S403: Synchronously play the same-screen image signal and the corresponding audio signal. Based on the corresponding relationship between the same-screen image signal and the audio signal determined in the previous step, the audio signal and the same-screen image signal are collaboratively played.

[0141] In addition, an embodiment of the present invention further provides an audio and video source output device, which may be a TV box or other device. Specifically, the device includes:

[0142] The audio and video signal output module is used to output audio and video signals to the audio and video playback device. Specifically, the audio and video signal output module can transmit audio and video signals to audio and video playback devices such as smart TVs and projectors through wired (such as HDMI signal cables) or wireless (local area network) methods.

[0143] The audio signal playing module is used to play the audio signal output to the audio and video playing device according to the instruction of the audio playing detection module. The audio signal playing module can be a sound card and speaker module set in the audio and video source output device itself.

[0144] The audio playback detection module is used to detect whether the audio playback device plays the audio signal. If it is determined that the audio playback device does not play the audio signal, the audio signal playback module is triggered to play the audio signal. The specific detection method can adopt the method described in the above embodiment, where the audio signal in the environment is collected by the sound pickup module, and then the correlation and phase difference between the signals are analyzed and judged to determine whether the audio and video playback device plays the audio output by the audio and video source output device.

[0145] In addition, when both the audio and video playback device and the audio and video source output device can play audio, the user can make a selection through voice commands. Specifically, the above-mentioned device can also include: a switching control module, which is used to receive the user's voice command and switch the audio signal playback between the audio and video playback device and the audio signal playback module according to the voice command.

[0146] Embodiment 8

[0147] The above embodiments describe the process of playback detection, power-on, echo cancellation processing, projection playback processing, remote conference collaborative playback processing method and the corresponding device structure. The functions of the above methods and devices can be implemented with the help of an electronic device, such as Fig.10 As shown, it is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which specifically includes: a memory 110 and a processor 120.

[0148] The memory 110 is used to store programs.

[0149] In addition to the above-mentioned programs, the memory 110 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0150] The memory 110 may be implemented by any type of volatile or nonvolatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0151] The processor 120 is coupled to the memory 110 and is used to execute the program in the memory 110 to perform the operation steps of one or more of the playback detection, power-on, echo cancellation processing, projection playback processing, and remote conference collaborative playback processing methods described in the aforementioned embodiments.

[0152] In addition, the processor 120 may also include the various modules described in the aforementioned embodiments to perform one or more of the playback detection, power-on, and echo cancellation processing methods, and the memory 110 may be used, for example, to store the data required for these modules to perform operations and / or the output data.

[0153] The specific description of the above-mentioned processing process, the detailed description of the technical principle and the detailed analysis of the technical effect have been described in detail in the previous embodiments and will not be repeated here.

[0154] Furthermore, as shown in the figure, the electronic device may also include other components such as a communication component 130, a power component 140, an audio component 150, and a display 160. The figure only schematically shows some components, which does not mean that the electronic device only includes the components shown in the figure.

[0155] The communication component 130 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component 130 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 130 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0156] The power supply component 140 provides power to various components of the electronic device. The power supply component 140 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0157] The audio component 150 is configured to output and / or input audio signals. For example, the audio component 150 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may be further stored in the memory 110 or sent via the communication component 130. In some embodiments, the audio component 150 also includes a speaker for outputting audio signals.

[0158] The display 160 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0159] In addition, an embodiment of the present invention also provides a computer program product, including a computer program or instructions, characterized in that when the computer program or instructions are executed by a processor, the processor implements one or any multiple of the aforementioned sound playback detection processing method, the power-on status detection processing method of the audio and video playback device, the echo elimination processing method, the projection playback processing method, and the remote conference collaborative playback processing method.

[0160] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting and processing the power-on status of an audio and video playback device, comprising: Sending a detection audio signal to an audio and video playback device to obtain a first audio signal detected by a sound pickup module of an audio and video source output device; According to the audio signal correlation and phase difference between the first audio signal and the detection audio signal, detecting whether the audio and video playback device plays the detection audio signal; if the audio signal correlation between the first audio signal and the detection audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the detection audio signal is greater than a preset phase difference threshold, determining that the audio and video playback device plays the detection audio signal; otherwise, determining that the audio and video playback device does not play the detection audio signal; If the audio and video playback device plays the detection audio signal, an audio and / or video content signal is output to the audio and video playback device.

2. The method according to claim 1, wherein: If the audio and video playback device does not play the detection audio signal, the detection audio signal is sent to the audio and video playback device again, and a new round of processing is performed to detect whether the audio and video playback device plays the detection audio signal.

3. The method according to claim 2, wherein: Also includes: If the audio and video playback device does not play the detection audio signal, a power-on instruction is sent to the audio and video playback device.

4. A power-on status detection and processing device for an audio and video playback device, comprising: A detection signal sending module is used to send a detection audio signal to an audio and video playback device; An audio signal acquisition module, used to acquire a first audio signal detected by a sound pickup module of an audio and video source output device; a playback detection module, configured to detect whether the audio and video playback device plays the detection audio signal according to the audio signal correlation and phase difference between the first audio signal and the detection audio signal; if the audio signal correlation between the first audio signal and the detection audio signal is greater than a preset correlation threshold, and the phase difference between the first audio signal and the detection audio signal is greater than a preset phase difference threshold, then determining that the audio and video playback device plays the detection audio signal; otherwise, determining that the audio and video playback device does not play the detection audio signal; The playback processing module is used to output audio and / or video content signals to the audio and video playback device when it is detected that the audio and video playback device plays the detection audio signal.

5. A remote conference collaborative playback processing method, applied to the audio and video source output device according to claims 1-3, comprising: Obtain the same-screen image signal of the remote conference and the audio and video signal of the conference site; Comparing the video signal in the audio and video signal with the on-screen image signal, and after determining that the on-screen image signal is synchronized with the video signal at the conference site, determining the audio signal corresponding to the on-screen image signal; the on-screen image signal is the on-screen image signal transmitted to the audio and video source output device of the remote conference by the cloud platform or the like; The same-screen image signal and the corresponding audio signal are played synchronously.

6. An audio and video source output device applied to the method described in claims 1-3, comprising: An audio and video signal output module, used to output audio and video signals to an audio and video playback device; An audio signal playing module, used to play the audio signal output to the audio and video playing device according to the instruction of the audio playing detection module; The audio playback detection module is used to detect whether the audio playback device plays the audio signal, and if it is determined that the audio playback device does not play the audio signal, trigger the audio signal playback module to play the audio signal.

7. The audio and video source output device according to claim 6, further comprising: The switching control module is used to receive a user's voice command and perform a switching process of audio signal playback between the audio and video playback device and the audio signal playback module according to the voice command.

8. An electronic device, comprising: Memory, used to store programs; A processor is used to run the program stored in the memory to execute any one of the power-on status detection and processing methods of the audio and video playback device described in any one of claims 1 to 3, and any one or more of the remote conference collaborative playback processing methods of claim 5.

9. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by the processor, the processor is caused to implement any one or more of the power-on status detection and processing method of the audio and video playback device described in any one of claims 1 to 3 and the remote conference collaborative playback processing method of claim 5.

Images