Intelligent sensor, echo cancellation method, service device and echo cancellation system

Abstract

The present application discloses an intelligent sensor, an echo cancellation method, a service device, and an echo cancellation system, which are used to solve the problems in the related art. In the embodiments of the present application, in the case where the audio signal transmission link is long and complex, the echo cancellation solution based on the whole machine is no longer applicable. In the present application, a service device can divide one signal into two signals, the obtained first signal is transmitted to the intelligent sensor by means of wired transmission, and the obtained second signal is used to be output to the intelligent terminal for playing, and the played signal can be collected by the intelligent sensor. The intelligent sensor can perform echo cancellation processing on the two collected signals, thereby realizing echo cancellation processing for complex links.

Description

Technical Field

[0001] The present application relates to the technical field of signal processing, and in particular, to an intelligent sensor, an echo cancellation method, a service device, and an echo cancellation system. Background Art

[0002] Echo cancellation processing needs to be performed at both ends of a remote call to purify the call audio signal. The echoes to be cancelled can be divided into circuit echoes and acoustic echoes. Taking acoustic echoes as an example, the necessity of echo cancellation is described.

[0003] As Figure 1 shown, it is a typical scenario that requires echo cancellation processing. The voice of the remote speaker is collected by the remote microphone and transmitted into the communication device. After wireless or wired transmission, it reaches the proximal communication device and is played through the proximal speaker. The sound played by the speaker is picked up by the proximal microphone and forms an acoustic echo in its communication device. After transmission, it returns to the remote communication device and is played through the speaker of the remote communication device, so that the remote speaker will hear his own echo.

[0004] In the related art, echo cancellation is usually implemented inside the communication device. However, with the continuous update of intelligent devices, the path from audio collection to playback may be long and complex. Therefore, in the case where the audio signal transmission link is long and complex, the echo cancellation solution based on the whole machine is no longer applicable. Summary of the Invention

[0005] The purpose of the present application is to provide an intelligent sensor, an echo cancellation method, a service device, and an echo cancellation system, which are used to solve the problem that in the prior art, when the audio signal transmission link is long and complex, the echo cancellation solution based on the whole machine is no longer applicable.

[0006] In a first aspect, the present application provides an intelligent sensor, including: a first microphone, a second microphone, and a digital signal processor, where:

[0007] The first microphone is used to collect a first audio signal of the surrounding environment;

[0008] The second microphone is connected to the audio input interface of the service device through a signal line, and is used to obtain a second audio signal output by the audio input interface; wherein, the service device divides the remote signal into two signals, the first signal is the second audio signal, and the second signal is used to be output to the intelligent terminal for playback;

[0009] The digital signal processor is respectively connected to the first microphone and the second microphone, and is configured to:

[0010] Obtain the first audio signal input by the first microphone and obtain the second audio signal input by the second microphone;

[0011] Perform echo cancellation processing based on the first audio signal and the second audio signal to obtain an audio signal to be transmitted.

[0012] Optionally, the digital signal processor is further configured to: output the audio signal to be transmitted to the service device, so that the service device encodes the audio signal to be transmitted and then sends it to a remote device.

[0013] In a second aspect, the present application provides an echo cancellation method, and the method includes:

[0014] Obtain a remote signal sent by a remote device;

[0015] Decode the remote signal to obtain a decoded signal;

[0016] Preprocess the decoded signal to obtain a signal to be played;

[0017] Divide the signal to be played into two paths of signals. Among them, the first path of signal is transmitted to the intelligent sensor through an audio input interface, and the second path of signal is output to the intelligent terminal for playback;

[0018] Among them, the two paths of signals are used for the intelligent sensor to perform echo cancellation processing on the collected audio signal.

[0019] Optionally, the method further includes:

[0020] Adjust the time delay difference between the two paths of signals transmitted to the intelligent sensor by caching one of the two paths of signals.

[0021] Optionally, the adjusting the time delay difference between the two paths of signals transmitted to the intelligent sensor by caching one of the two paths of signals includes:

[0022] If it is necessary to increase the time delay of the first path of signal, based on the relationship that the cache space capacity is proportional to the time delay, cache the first path of signal with a first preset cache space capacity and then send it to the audio input interface; or,

[0023] If it is necessary to increase the time delay of the second path of signal, based on the relationship that the cache space capacity is proportional to the time delay, cache the second path of signal with a second preset cache space capacity and then send it to the intelligent terminal device.

[0024] Optionally, the method further includes:

[0025] If the time delay of the first signal reaching the intelligent sensor is greater than the time delay of the second signal reaching the intelligent sensor, and the time delay difference between the first signal and the second signal is greater than the upper limit of the time delay, it is determined that the time delay of the second signal needs to be increased;

[0026] If the time delay of the first signal reaching the intelligent sensor is less than the time delay of the second signal reaching the intelligent sensor, and the time delay difference between the first signal and the second signal is greater than the upper limit of the time delay, it is determined that the time delay of the first signal needs to be increased.

[0027] Optionally, the method further includes:

[0028] Receiving the signal to be transmitted that has undergone echo cancellation processing and is sent by the intelligent sensor;

[0029] Encoding the signal to be transmitted and then sending it to the remote device.

[0030] In a third aspect, the present application provides a service device, including an audio input interface, a processor, and a memory,

[0031] The audio input interface is connected to the intelligent sensor through a signal line;

[0032] The memory is used to store a computer program executable by the controller;

[0033] The controller is respectively connected to the speaker and the memory, and is configured to execute the method described in any one of the second aspects.

[0034] In a fourth aspect, the present application provides an echo cancellation system, the system includes an intelligent sensor, an intelligent terminal, and a service device, where:

[0035] The intelligent sensor includes a first microphone, a second microphone, and a digital signal processor, where:

[0036] The first microphone is used to collect the first audio signal of the surrounding environment;

[0037] The second microphone is connected to the audio input interface of the service device through a signal line, and is used to obtain the second audio signal output by the audio input interface;

[0038] The digital signal processor is respectively connected to the first microphone and the second microphone, and is configured to: obtain the first audio signal input by the first microphone, and obtain the second audio signal input by the second microphone; perform echo cancellation processing based on the first audio signal and the second audio signal to obtain the audio signal to be transmitted;

[0039] The service device is used to encode the audio signal to be transmitted and send it to the remote device; it is also used to receive the remote signal sent by the remote device and divide the remote signal into two signals, where the first signal is the second audio signal and the second signal is used to output to the smart terminal for playing.

[0040] Optionally, the service device is further used to adjust the time delay difference between the two signals transmitted to the smart sensor by caching one of the two signals.

[0041] In this application, one signal can be divided into two signals based on the service device. The first signal obtained is transmitted to the smart sensor through a wired transmission method, and the second signal obtained is used to output to the smart terminal for playing. The played signal can be collected by the smart sensor. The smart sensor can perform echo cancellation processing on the two collected signals, thereby realizing echo cancellation processing for complex links.

[0042] Other features and advantages of this application will be described in the subsequent specification, and part of them will become obvious from the specification, or be understood by implementing this application. The objectives and other advantages of this application can be realized and obtained through the structures specifically pointed out in the written specification, claims, and drawings. Description of the Drawings

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required to be used in the embodiments of this application. Obviously, the following introduced drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0044] Figure 1 It is a schematic diagram of the principle of acoustic echo generation provided by some embodiments of this application;

[0045] Figure 2A It is an application scenario diagram of an echo cancellation method provided by some embodiments of this application;

[0046] Figure 2B Provided by some embodiments of this application Figure 2A The hardware configuration block diagram of the smart sensor 100 in

[0047] Figure 2C Provided by some embodiments of this application Figure 2A The structure diagram of the service device 300 in

[0048] Figure 3 It is a schematic flowchart of the echo cancellation method provided by some embodiments of this application;

[0049] Figure 4 Schematic diagram of an echo cancellation system provided for some embodiments of the present application;

[0050] Figure 5 Schematic diagram of echo delay provided for some embodiments of the present application;

[0051] Figure 6 Another flowchart of an echo cancellation method provided for some embodiments of the present application;

[0052] Figure 7 Schematic diagram of an echo cancellation system provided for some embodiments of the present application. Detailed implementation manners

[0053] To make the objectives and implementation manners of the present application clearer, the following will clearly and completely describe the exemplary implementation manners of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application. Apparently, the described exemplary embodiments are only a part rather than all of the embodiments of the present application.

[0054] It should be noted that the brief descriptions of the terms in the present application are only for facilitating the understanding of the subsequent described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0055] The terms "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0056] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device including a series of components does not necessarily have to be limited to all the clearly listed components, but may include other components not clearly listed or inherent to these products or devices.

[0057] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to the element.

[0058] See Figure 2A , an application scenario diagram of echo cancellation provided for some embodiments of the present application. As Figure 2AAs shown, the intelligent sensor 100 is used to collect audio and multimedia data and transmit it to the service device 300. The service device 300 encodes the multimedia data collected by the intelligent sensor 100 and sends it to a remote device (not shown in the figure). The service device 300 processes the multimedia data sent by the remote device and sends it to the smart TV 200 for output.

[0059] Communication between the intelligent sensor 100 and the service device 300 can be carried out in a wired or wireless manner. The intelligent sensor can be, for example, a surveillance camera, an audio-video sensor, etc.

[0060] The structural block diagram of the intelligent sensor 100 is as Figure 2B shown, and it includes a digital signal processor 110, a memory 120, a communicator 130, a user input interface 140, and a power supply 150.

[0061] The digital signal processor 110 is used to perform echo cancellation based on a reference signal.

[0062] The memory 120 is used to store various operating programs, data, and applications for driving and controlling the intelligent sensor 100 under the control of the digital signal processor 110. The memory 120 can store various control signal instructions input by the user. For example, the monitoring range of a surveillance camera.

[0063] Under the control of the digital signal processor 110, the communicator 130 realizes the communication of control signals and data signals with the service device 300. For example: The intelligent sensor 100 sends a multimedia signal to the service device 300 via the communicator 130. The intelligent sensor 100 can collect the audio signal emitted by the speaker of the smart TV 200 via the communicator 130. The communicator 130 can include an infrared signal interface 131 and a radio frequency signal interface 132. For example: When it is an infrared signal interface, the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol and sent to the service device 300 through the infrared transmission module. Another example: When it is a radio frequency signal interface, the user input instruction needs to be converted into a digital signal, and then modulated according to the radio frequency control signal modulation protocol and sent to the service device 300 by the radio frequency transmission terminal.

[0064] The user input interface 140 can include a microphone 141 and can also include keys 142, etc.

[0065] The power supply 160 is used to provide operating power support for each component of the intelligent sensor 100. The form can be a battery and related control circuits.

[0066] The smart TV 200 and the service device 300 perform data communication through various communication methods. Here, the smart TV 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks. The service device 300 can provide various contents and interactions to the smart TV 200. Exemplarily, the smart TV 200 can send and receive information, such as: receiving electronic program guide (EPG) data, receiving software program updates, or accessing a remotely stored digital media library. The service device 300 can be implemented as a home brain server.

[0067] Figure 2C An exemplary hardware configuration block diagram of the service device 300 is shown. As Figure 2C shown, the components of the service device 300 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0068] The bus 33 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a processor, or a local area bus using any bus structure in a variety of bus structures.

[0069] The memory 32 may include a readable medium in the form of volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322, and may further include a read-only memory (ROM) 323.

[0070] The memory 32 may further include a program / utilities 325 having a set (at least one) of program modules 324. Such program modules 324 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0071] The service device 300 can also communicate with one or more external devices 34 (such as keyboards, pointing devices, etc.), and can also communicate with one or more devices that enable users to interact with the service device 300, and / or communicate with any device that enables the service device 300 to communicate with one or more other electronic devices (such as routers, modems, etc.). Such communication can be carried out through the input / output (I / O) interface 35. In addition, the service device 300 can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through the network adapter 36. As shown in the figure, the network adapter 36 communicates with other modules for the service device 300 through the bus 33. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the service device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0072] In some embodiments, the processor 31 may include two backplanes. One backplane is used for encoding and decoding audio and video data, and the other backplane can preprocess the decoded audio and video data to obtain playable images and audio.

[0073] In some embodiments, various aspects of an echo cancellation method provided in the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps in any of the echo cancellation methods of various exemplary embodiments of the present application.

[0074] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0075] A program product for echo cancellation according to an embodiment of the present application may be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on an electronic device. However, the program product of the present application is not limited to this. In this document, a readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0076] A readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0077] The program code contained on a readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0078] The following combines Figure 2A the scenarios shown to illustrate the echo cancellation method provided by the embodiments of the present application. It should be noted that in the embodiments of the present application, any intelligent terminal device can be used to replace Figure 2A the smart TV in to implement the playback of audio and video. Such intelligent terminals include, for example, a smartwatch, a mobile terminal, a desktop computer, etc.

[0079] As Figure 3 shown, it is a schematic flowchart of the echo cancellation method provided by the embodiments of the present application, including:

[0080] In step 301, the service device acquires the remote signal of the remote device.

[0081] In step 302, the service device first performs decoding processing on the remote information to obtain a decoded signal.

[0082] For example, if the remote signal includes audio and video signals, the service device can unpack the audio and video signals to obtain an audio stream and a video stream, and then obtain the decoded audio signal according to the decoding method of the audio signal, and decode the video stream according to the decoding method of the video signal to obtain the decoded video signal. During implementation, step 302 can be executed by one backplane, and then step 303 can be executed by another backplane to complete the preprocessing of the audio signal and the preprocessing of the video signal.

[0083] In step 303, the service device preprocesses the decoded signal to obtain a signal to be played.

[0084] The preprocessing may include, for example, the synchronization processing of audio and video signals. For example, when the timestamps of the first frames of audio frames and video frames are inconsistent, in order to achieve audio-video synchronization, the audio frames and video frames can be adjusted for synchronous output. For example, based on the timestamp of the audio frame, the playback time of the video frame is synchronously controlled to be consistent with the audio frame. In this solution, during the synchronization processing, if the playback time of the first frame of the video frame is later than the first frame of the audio frame, the playback of the first frame of the video frame can be delayed. If, during the synchronization processing, the playback time of the first frame of the video frame is earlier than the first frame of the audio frame, the video frames earlier than the first frame of the audio frame can be filtered out.

[0085] In step 304, the service device divides the signal to be played into two signals. Among them, the first signal is transmitted to the intelligent sensor through the audio input interface of the service device, and the second signal is output to the intelligent terminal for playback.

[0086] In step 305, the intelligent device plays the second signal.

[0087] For example, a smart TV plays an audio signal through a speaker.

[0088] In step 306, the intelligent sensor collects the first audio signal awakened around through the first microphone. Thus, both the audio signal played by the intelligent device through the speaker and the environmental signal can be collected by the first microphone. The intelligent sensor collects the second audio signal transmitted by the wired transmission method through the second microphone, that is, the first signal separated by the service device.

[0089] In step 307, the digital signal processor of the intelligent sensor performs echo cancellation processing based on the first audio signal and the second audio signal to obtain the audio signal to be transmitted. And the intelligent sensor sends the audio signal to be transmitted to the service device.

[0090] Of course, if the intelligent sensor also collects picture information, the picture information is synchronously transmitted to the service device.

[0091] In step 308, the service device encodes the audio signal to be transmitted transmitted by the intelligent sensor, and then sends it to the remote device for playback.

[0092] Among them, when the intelligent sensor also collects picture information, the service device encodes the audio signal and the picture information and sends them to the remote device together.

[0093] Thus, in the embodiments of the present application, the reference signal for echo cancellation includes two signals. One is the second audio signal transmitted by the service device to the second microphone of the intelligent sensor through the audio input interface, and the other is the first audio signal played by the intelligent terminal. Thus, the intelligent sensor can perform echo cancellation processing on the audio signal collected at the local end. Thus, in the embodiments of the present application, echo cancellation can be achieved in the case where the transmission link is long and complex.

[0094] See Figure 4 This is an example of the echo cancellation system framework provided in the embodiments of the present application using a visual intelligent sensor, a home brain server, and an intelligent TV.

[0095] As Figure 4 shown, the visual intelligent sensor in the embodiments of the present application includes a first microphone MIC1 and a second microphone MIC0. The first microphone can collect the remote signal played by the intelligent TV as the first audio signal. The second microphone is connected to the AUX port of the home brain server and can collect the remote signal output by the AUX port as the second audio signal.

[0096] After the digital signal processor DSP of the visual intelligent sensor performs echo cancellation processing based on the acquired first audio signal and second audio signal, the audio signal can be sent to the home brain server through the RI45 interface. The video signal collected by the visual intelligent sensor is sent to the home brain server through the HDMI port. The home brain server encodes the video signal and audio signal collected by the visual intelligent sensor and then sends them to the remote device. In Figure 4 it, the visual intelligent sensor transmits YUV (i.e., video stream) to the home brain server through the HDMI port, and the visual intelligent sensor transmits G711a / pcm (i.e., audio signal) to the home brain server through the RJ45 interface. The home brain server can perform the interaction of control signaling based on the RJ45 interface. The control signaling may include, for example, control messages, such as sending control messages to the visual intelligent sensor, and the visual intelligent sensor executes corresponding operations after parsing the messages.

[0097] For the remote signal sent by the corresponding remote device, the home brain server can perform a decoding operation on the remote signal to obtain a decoded signal. The decoded signal may include a video signal and an audio signal. After the decoded signal is further preprocessed by the home brain server, the audio signal can be disassembled into two signals, the video signal and the first audio signal among them, and transmitted to the smart TV through the HDMI port. The smart TV displays the video information and plays the audio signal through the speaker. The played audio signal will become an echo signal and be collected by the first microphone of the visual sensor as the first audio signal. The other audio signal is transmitted to the second microphone through the AUX interface of the service device via a transmission line. Thus, the visual sensor will collect the second audio signal, and then after echo cancellation processing, it is transmitted by the home brain server to the remote device for playback. Thus, the embodiment of the present application realizes an echo cancellation scheme for the multi-device path as shown in Figure 4 The echo cancellation scheme in the case of the multi-device path shown. It should be noted that the echo involved in the embodiment of the present application can be a circuit echo, an acoustic echo, and can also include these two types of echoes. The echo cancellation strategy of the DSP in the visual sensor can be configured according to the echo to be cancelled, and the present application does not limit this.

[0098] Continuing with the example of the framework shown in Figure 4 The included time delays are as shown in Figure 5 as follows:

[0099] 1) The time delay from the HDMI interface of the home brain server to the HDMI interface of the TV end is T1.

[0100] 2) The time delay for the TV end to process the audio and then send it to the speaker is T2.

[0101] 3) The time delay for the audio played by the TV end speaker to be collected by the mic1 of the visual sensor is T3.

[0102] 4) The time delay for the audio signal transmitted from the AUX interface of the home brain server to the visual sensor is T4.

[0103] Then the time delay difference ΔT between the first audio signal and the second audio signal = T1 + T2 + T3 - T4. Since the DSP in the visual sensor can handle a time delay difference with a time delay upper limit. That is, if the time delay difference between the first audio signal and the second audio signal exceeds this time delay upper limit, the DSP will not be able to cancel the echo well.

[0104] Since different smart terminals use different movement core schemes, the time delay for processing audio signals is different, and the time delays of different HDMI cables, etc. for transmission are also different. Then, the above echo cancellation scheme has very high requirements for the echo cancellation performance of the DSP module selected at the visual sensor end.

[0105] Therefore, in the embodiments of the present application, in order to ensure that the DSP can effectively eliminate echo, a signal caching mechanism is added between the home brain server (i.e., the service device) that transmits two signals to the smart TV and the AUX port respectively. The signal caching mechanism can adjust the time delay of the two signals reaching the smart sensor. As Figure 6 shown, the following is a flowchart of the service device adjusting the time delay of two signals through the caching mechanism in the embodiments of the present application, including the following steps:

[0106] In step 601, the service device divides the signal to be played into two signals.

[0107] As described above, the first signal in the two signals is used to be transmitted to the second microphone of the smart sensor, and the second signal is used to be played by the smart terminal.

[0108] If the time delay difference between the two signals reaching the smart sensor is greater than the time delay upper limit, the DSP of the smart sensor cannot effectively eliminate echo. Therefore, the service device can adjust the time delay difference of the two signals transmitted to the smart sensor by caching one of the two signals.

[0109] In this way, by caching one of the signals, the time delay of this signal reaching the smart sensor can be increased, so as to adjust the time delay difference of the two signals. This can enable the entire echo cancellation system to adapt to echo cancellation with any time delay difference.

[0110] For example, in step 602, if it is necessary to increase the time delay of the first signal, the service device caches the first signal with the first preset cache space capacity based on the relationship that the cache space capacity is proportional to the time delay, and then sends it to the audio input interface; or, if it is necessary to increase the time delay of the second signal, the service device caches the second signal with the second preset cache space capacity based on the relationship that the cache space capacity is proportional to the time delay, and then sends it to the smart terminal device.

[0111] Thus, the service device can adjust the time delay difference of the two signals reaching the smart sensor according to actual needs.

[0112] Of course, in implementation, due to different smart TVs, the differences in the AUX transmission lines will also bring differences in time delay. In the embodiments of the present application, the way of experimental measurement can be used to measure the delay duration of the service device for the first signal and / or the second signal when the echo cancellation system is composed of different models of devices. Then, when in use, the service device can adopt a reasonable delay scheme according to the differences of the smart terminal and the transmission line.

[0113] For example, the embodiments of the present application provide cache units. Each cache unit can cache data with a fixed capacity, which can be set according to actual situations, and the present application does not limit this. Thus, different cache units adapted to different smart TVs and transmission lines can be defined according to the experimental measurement results. For example, if a longer latency needs to be added, it can be achieved by increasing the cache unit; if a lower latency is required, a smaller number of cache units or no cache units can be used.

[0114] During implementation, if the latency of the first signal arriving at the intelligent sensor is greater than the latency of the second signal arriving at the intelligent sensor, and the latency difference between the first signal and the second signal is greater than the latency upper limit, the service device determines that it is necessary to increase the latency of the second signal;

[0115] If the latency of the first signal arriving at the intelligent sensor is less than the latency of the second signal arriving at the intelligent sensor, and the latency difference between the first signal and the second signal is greater than the latency upper limit, the service device determines that it is necessary to increase the latency of the first signal.

[0116] Thus, the service device can determine which signal to increase the latency according to the actual situation.

[0117] Continuing with Figure 4 the echo cancellation system framework shown as an example, the improved structural block diagram of the system through the cache mechanism is as shown in Figure 4 ... In the home brain server, the same audio signal is respectively pushed to the HDMI and AUX interfaces. Before being pushed to the HDMI interface, it passes through a latency cache unit B1. Similarly, before being pushed to the AUX interface, it passes through a latency cache unit B2. Figure 7 Both B1 and B2 are adjustable latency cache units, and the size of the cached audio data is cached in steps of Δt (for example, 25 ms).

[0118] The latency range of the improved ΔT can be adjusted according to actual situations and optimized to the most suitable latency range for echo cancellation. For example, if ΔT exceeds the maximum latency range of echo cancellation of the DSP, then the latency of part T4 can be increased by increasing the size of the cached data in B2, thereby reducing ΔT.

[0119]

[0120] ​Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application.

[0121] For the sake of explanation, the above description has been presented in connection with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Many modifications and variations are possible in light of the above teachings. The selection and description of the embodiments are intended to best explain the principles and practical applications, thereby enabling those skilled in the art to best utilize the embodiments and various embodiments suitable for specific uses considering the various different variations.

Claims

1. An intelligent sensor, characterized in that, Including: A first microphone, a second microphone, and a digital signal processor, where: The first microphone is configured to collect a first audio signal of the surrounding environment; The second microphone is connected to the audio input interface of the service device through a signal line and is configured to obtain a second audio signal output by the audio input interface; where the service device divides the remote signal into two signals, the first signal is the second audio signal, and the second signal is used to be output to the smart terminal for playback; if it is necessary to increase the delay of the first signal, the first signal after the increased delay is that the service device caches the first signal with a first preset cache space capacity based on the relationship that the cache space capacity is proportional to the delay, and then transmits it to the smart sensor through the audio input interface; if it is necessary to increase the delay of the second signal, the second signal after the increased delay is that the service device caches the second signal with a second preset cache space capacity based on the relationship that the cache space capacity is proportional to the delay, and then sends it to the smart terminal; The digital signal processor is respectively connected to the first microphone and the second microphone and is configured to: Obtain the first audio signal input by the first microphone and obtain the second audio signal input by the second microphone; Perform echo cancellation processing based on the first audio signal and the second audio signal to obtain an audio signal to be transmitted.

2. The intelligent sensor according to claim 1, wherein The digital signal processor is further configured to: output the audio signal to be transmitted to the service device so that the service device encodes the audio signal to be transmitted and then sends it to the remote device.

3. An echo cancellation method, characterized in that, The method includes: Obtain a remote signal sent by a remote device; Perform decoding processing on the remote signal to obtain a decoded signal; Perform preprocessing on the decoded signal to obtain a signal to be played; Divide the signal to be played into two signals, where the first signal is transmitted to the smart sensor through the audio input interface, and the second signal is output to the smart terminal for playback; the two signals are used for the smart sensor to perform echo cancellation processing on the collected audio signal; Wherein, the method further includes: If it is necessary to increase the delay of the first signal, then based on the relationship that the cache space capacity is proportional to the delay, cache the first signal with a first preset cache space capacity and then send it to the audio input interface; or, If it is necessary to increase the delay of the second signal, based on the relationship that the cache space capacity is proportional to the delay, cache the second signal with a second preset cache space capacity and then send it to the smart terminal.

4. The method according to claim 3, wherein The method further includes: If the delay of the first signal reaching the smart sensor is greater than the delay of the second signal reaching the smart sensor, and the delay difference between the first signal and the second signal is greater than the delay upper limit, then determine that it is necessary to increase the delay of the second signal; If the time delay of the first path signal arriving at the intelligent sensor is less than that of the second path signal arriving at the intelligent sensor, and the time delay difference between the first path signal and the second path signal is greater than the upper limit of the time delay, it is determined that the time delay of the first path signal needs to be increased.

5. The method according to any one of claims 3-4, characterized in that, The method further includes: Receiving the signal to be transmitted after echo cancellation processing sent by the intelligent sensor; Encoding the signal to be transmitted and sending it to the remote device.

6. A service device, characterized in that, Comprising an audio input interface, a processor and a memory, The audio input interface is connected to the intelligent sensor through a signal line; The memory is used to store the computer program executed by the controller; The controller is respectively connected to the speaker and the memory, and is configured to execute the method according to any one of claims 3-5.

7. An echo cancellation system, characterized in that, The system includes an intelligent sensor, an intelligent terminal and a service device, wherein: The intelligent sensor includes a first microphone, a second microphone and a digital signal processor, wherein: The first microphone is used to collect the first audio signal of the surrounding environment; The second microphone is connected to the audio input interface of the service device through a signal line and is used to obtain the second audio signal output by the audio input interface; The digital signal processor is respectively connected to the first microphone and the second microphone, and is configured to: obtain the first audio signal input by the first microphone and obtain the second audio signal input by the second microphone; perform echo cancellation processing based on the first audio signal and the second audio signal to obtain the audio signal to be transmitted; The service device is used to encode the audio signal to be transmitted and send it to the remote device; it is also used to receive the remote signal sent by the remote device and divide the remote signal into two path signals, wherein the first path signal is the second audio signal, and the second path signal is used to output to the intelligent terminal for playing; Wherein, the service device is further used for: If it is necessary to increase the time delay of the first path signal, based on the relationship that the cache space capacity is proportional to the time delay, cache the first path signal with the first preset cache space capacity and then send it to the audio input interface; or, If it is necessary to increase the time delay of the second path signal, based on the relationship that the cache space capacity is proportional to the time delay, cache the second path signal with the second preset cache space capacity and then send it to the intelligent terminal.

Images