False Trigger Correction for Voice-Activated Smart Devices

Abstract

A method for confirming a trigger in an audio input detected by a voice activated smart device, the voice activated smart device listening for and detecting a trigger in the audio input, confirming whether the trigger in the audio input is intended to wake the device, and if confirmed, instructing the device to activate. If the trigger cannot be confirmed, instructing the device to ignore the trigger. The step of confirming whether the trigger is intended to activate the device may include determining whether the audio input is human-generated speech, which may also include detecting a fingerprint in the audio input.

Description

Technical Field

[0001] The present disclosure relates to a method for generating, detecting and correcting false triggering of a voice activated smart device. Background Art

[0002] Voice command smart devices are specifically designed to recognize wake-up words as the gateway to their operation. The device listens for the wake-up word in real time, and the user expects the device to respond quickly and accurately. However, false positive wake-up words may cause the device to activate unexpectedly. Typically, benign events such as the device answering questions that are not asked or recording background sounds may occur. However, more adverse events that can raise privacy and security concerns may also occur. Summary of the invention

[0003] A method for confirming a trigger in an audio input detected by a voice-activated smart device. The audio input may be a signal spoken by a human speaker or a signal played by an audio source such as a television or radio. According to the method, a voice-activated smart device and / or an audio source listens for and detects a trigger in the audio input. The method confirms whether the trigger in the audio input is intended to wake up the device, and if so, instructs the device to activate. If the trigger cannot be confirmed, the device is instructed to ignore the trigger. The step of confirming whether the trigger is intended to activate the device is accomplished by determining whether the audio input is human-generated speech, or by detecting a fingerprint embedded in the audio input. The fingerprint may be compared to a database of stored / known fingerprint triggers, and may be detected as an inherent distortion of the audio signal that occurs due to the audio source transmitting the audio input. Alternatively, the fingerprint may be known in the audio source, detected before the audio source plays the audio input, and when detected by the voice-activated device, instructs the voice-activated device to ignore the trigger. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is an example environment for voice-activated smart devices;

[0005] Figure 2A is a flow chart of a method of confirming a trigger in an audio signal detected by a voice-activated smart device;

[0006] Figure 2B is a flow chart of a method for confirming that audio input is human-generated speech in a method of confirming a trigger in an audio signal detected by a voice-activated smart device;

[0007] Figure 3 is a flow chart of a method for wake-up confirmation of a voice-activated smart device;

[0008] Figure 4A is a flow chart of a method for wake-up confirmation of a voice-activated smart device;

[0009] Figure 4B is a flow chart of a method for wake-up confirmation of a voice-activated smart device;

[0010] Figure 5 is a flow chart of a method for wake-up confirmation of a voice-activated smart device;

[0011] Figure 6 is a flow chart of a method for voice-activated wake-up confirmation of a smart device; and

[0012] Figure 7 is a flow chart of a method for improving any one or more of the methods of confirming triggering of a voice activated smart device.

[0013] The elements and steps in the figures are shown for simplicity and clarity, and are not necessarily presented according to any particular order. For example, the figures show steps that can be performed simultaneously or in different orders to help increase understanding of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] Although various aspects of the present disclosure are described with reference to specific illustrative embodiments, the present disclosure is not limited to these embodiments, and other modifications, applications, and embodiments may be implemented without departing from the present disclosure. In the accompanying drawings, the same reference numerals will be used to illustrate the same components. Those skilled in the art will recognize that the various components set forth herein may be modified without departing from the scope of the present disclosure.

[0015] Any one or more of the servers, receivers or devices described herein include computer executable instructions that can be compiled or interpreted according to a computer program created using various programming languages ​​and / or techniques. Typically, a processor (such as a microprocessor) receives instructions from a memory, a computer-readable medium, etc. and executes these instructions. The processing unit includes a non-temporary computer-readable storage medium capable of executing instructions of a software program. The computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. Any one or more of the devices herein may rely on firmware, which may need to be updated from time to time to ensure compatibility with the operating system, improved and additional functionality, security updates, etc. The connection and networking server, receiver or device may include, but is not limited to, SATA, Wi-Fi, lightning, Ethernet, UFS, 5G, etc. One or more servers, receivers or devices may operate using a dedicated operating system, multiple software programs and / or interface platforms such as graphics, audio, wireless networks, enabling applications, integrated hardware of vehicle components, systems and external devices such as smartphones, tablets and other systems, etc. Any one or more of the methods described herein are performed by a processing unit comprising a non-transitory computer-readable storage medium capable of executing instructions of a software program as described above.

[0016] Figure 1 1 is a block diagram 100 of a voice-activated smart device 102 in an example setting for which the smart device 102 may be mistakenly activated by detecting a trigger word or phrase played by an audio source 104, such as a television. In practice, a voice-activated smart device is activated based on a trigger, voice command, from a user. In the case of an audio source 104, such as a television that receives playing content, an audio input or audio input signal 106 is played back through a speaker on the television. The smart device 102 may detect the audio played back through the television, and because smart devices are designed to "listen" for a trigger word or phrase in order to activate and act on the user's command, input from the audio source 104 that is not issued by a human may inadvertently activate the device. In practice, the device 102 may also communicate with a cloud-based environment, such as a cloud service 108. The cloud service 108 includes an automatic speech recognition (ASR) engine 110 that accesses a database 112. The database may include known triggers, fingerprints, false positive word combinations, known commercials that include trigger words and phrases, known false positive situations, etc. that the device 102 references when determining whether to operate based on the trigger word 114 or phrase 116.

[0017] Still reference Figure 1, audio source 104 may play a television commercial that includes a phrase 116 that may be detected as audio input at device 102. The phrase may include a trigger word 114 or a trigger phrase 116 that may wake up device 102, even though the trigger word 114 or phrase 116 actually comes from the playback and not from a user associated with device 102. For example, a television commercial in which a narrator says, "Cortana is your virtual assistant in life. It helps you get through your day and even helps you have fun." The narration script played by audio source 104 includes the trigger word 114 "Cortana." Device 102 may hear the trigger word by accident. Similarly, as part of the television commercial being played, an actor says, "Hey, Cortana, play the Beatles." The script played by audio source 104 includes the trigger phrase 116, i.e., "Cortana, play the Beatles." However, the trigger word 114 or phrase 116 detected at device 102 is not intended for device 102 to wake up device 102. However, because it contains a trigger that would normally wake up the device 102, it may cause the device 102 to be falsely triggered even though the content being played was not intended for the device 102. The device 102 may inadvertently detect the trigger word 114 or phrase 116 by accidentally hearing a commercial being played on television, and cause the device 102 to activate when the user did not in fact intend to trigger the device 102, thereby causing a false trigger.

[0018] To prevent accidental activation of the device, use Figure 2A and Figure 2B 200 is shown. The method 200 begins with an audio input being played 202. The audio input may be spoken by a human, or played through the speakers of an audio source such as a television or radio. The audio input may include a trigger word or phrase, which may be intentionally or unintentionally targeted to wake up the device. An algorithm in the device or in the audio source is designed to listen 204 for the trigger word or phrase.

[0019] according to Figure 2A In the method described in, upon hearing or detecting 206 a trigger word or phrase, a confirmation 208 is made as to whether the trigger word or phrase should activate 210 the device or be ignored 212 by the device. In the event that the audio source or device detects 206 the trigger word or phrase, a confirmation 208 is made as to whether the audio input is an intentional trigger spoken to wake up the device. In the event that the audio input is confirmed 214 to be an actual trigger intended to wake up the device, the device activates 210. In the event that the audio input is not confirmed 216 to be intended to wake up the device, the device is instructed to ignore 212 the trigger word or phrase, and the audio source and device continue to listen 204.

[0020] according to Figure 2BIn the method described in, audio processing 203 is performed, during which a distortion analysis is performed to determine 205 whether the audio input has a source of human-generated speech. In the event that the audio input has a source other than human-generated speech, the device is instructed to ignore 212 the audio input, and the audio source and / or audio device continues to listen 204 for future audio input 202. In the event that the audio input is determined to be human-generated speech 207, when a trigger word or phrase is heard or detected 206 in the audio input, a confirmation 208 is made as to whether the trigger word or phrase should activate 210 the device or be ignored 216 by the device. In the event that the audio input is confirmed 214 to be a real trigger intended to wake up the device, the device activates 210.

[0021] In either approach, confirmation that the trigger word or phrase was intentionally spoken to wake the device is typically determined by determining whether the audio input is a request spoken by a human being and is not actually audio input being played from an audio source other than human-generated speech, such as a television or radio audio source. Figure 2B In the method described in , when it has been determined that the audio input is human-generated speech, the steps of detecting a trigger and confirming its source 208 when the trigger is detected are simplified. However, there are several methods that can be used to complete the wake-up confirmation step 208, and they will be described in detail below.

[0022] Each of the methods described below involves determining whether a trigger is intentional or unintentional based on monitoring a trigger in an audio signal and detecting a fingerprint in the audio signal. The fingerprint may be a fingerprint that will be referred to below. Figure 3 The fingerprint may be a characteristic of an audio signal. Figure 4A and Figure 4B The embedded fingerprint described (before or after production). The fingerprint can be referred to below Figure 5 An inaudible tone or infrared blast described as intentionally added to an audio signal. Fingerprints may also be detected from a cloud-based source, predetermined and / or learned based on data stored in the cloud and actions collected from user actions in response to intentional or accidental activation of the audio input of the device, which will be referred to below Figure 6 describe.

[0023] refer to Figure 3, a method 300 of confirming that a trigger present in an audio input is intended to wake up a device is shown. The audio input contains a trigger word or phrase detected at an audio source or device by applying a listening algorithm. When a trigger in the audio input is heard or detected, a distortion analysis is performed 302. The audio played by the speaker includes inherent distortion of the audio signal that occurs naturally within the speaker. This inherent distortion leaves a fingerprint on the audio signal that can be detected by performing a distortion analysis at the audio source or device. Confirmation of the presence of the fingerprint is performed 304. The presence of the fingerprint 306 is an identification that the source of the audio signal is not human speech, thereby confirming that the audio signal is not intended to wake up the device and instructing the device to ignore the audio input 308. In the event that the distortion analysis does not identify a fingerprint 310, it is determined that the audio input is spoken by a human, and the device is activated 312.

[0024] refer to Figure 4A , shows a method 400 of confirming that a trigger present in a crafted audio input is not intended to wake up a device. Before the device is activated, an audio source listens 402 to the audio input. The audio source hears or detects 404 the trigger present in the audio input and processes the signal to embed a fingerprint 406 in the audio input. The device detects 408 the fingerprint and ignores 410 the trigger. Figure 4B , a method 450 of confirming the presence of a trigger in a pre-production audio input is shown. The fingerprint has been embedded into the audio before the audio input is received and played by the audio source. In a pre-production capability, advertisers will know when a trigger word or phrase appears in the audio of the audio stream. By placing the fingerprint in the signal before playback, the audio source and device are prepared to detect the fingerprint. The audio source and / or device listens 452 to the audio input. The audio source and / or device detects 454 the trigger in the audio input. The audio source and device detect the fingerprint embedded 456 in the audio input. The fingerprint has been embedded in the audio input during the pre-production process and is already present in the audio input, so the audio input is passed to the device, which also detects 458 the fingerprint and ignores 460 the trigger.

[0025] refer to Figure 5 , a method 500 is shown to confirm that a trigger present in an audio input is not intended to wake up a device. Figure 5 In the method shown, the fingerprint is an ultrasonic signal. Figure 4A and Figure 4BIn the method described in , the fingerprint can be inserted into the audio signal at the source before the audio signal is produced, and only passes through the audio source. Alternatively, the fingerprint is an ultrasonic signal inserted at the audio source. In one or more embodiments, a text-to-speech (TTS) technology can be implemented that recognizes trigger words in the audio signal and sends an encoded algorithm output when a duplicate voice is recognized. When the audio source issues a trigger, the trigger includes the fingerprint, and the device will ignore any commands or dialogues related to the trigger. Play 502 an audio input containing a trigger and an ultrasonic overlay. The algorithm (in the audio source or device) listens 504 for the trigger. When a trigger is detected 506, an ultrasonic signal analysis is performed 508. In the absence of a trigger 510, the algorithm continues to listen 504. If the ultrasonic signal analysis recognizes that the trigger 512 includes an ultrasonic overlay, the device is instructed to ignore the trigger 514. In the case where the ultrasonic signal analysis does not recognize an ultrasonic overlay 516, the device activates 518. In one or more embodiments, the ultrasonic overlay can be a simple tone, or it can be a digital overlay that allows false triggers to be detected and ignored.

[0026] refer to Figure 6 , a method 600 for wake-up confirmation when it is known that the advertisement to be played includes a false trigger. In one or more embodiments, trigger data is added to the audio source and notifies the device that there is a false trigger. The trigger data can be added to the audio source in several ways. For example, through a communication network, an infrared signal, a sub-audio tone, a visible light signal. The trigger data embedded in the audio signal played by the audio source is detected by the device and instructs to ignore the trigger.

[0027] Still reference Figure 6 , a method 600 for wake-up confirmation using a fingerprint embedded at an audio source. Before the device is activated, the audio input is screened 602 as it is passed to the audio source. The audio source screens 602 the audio input by listening for a trigger. When the trigger is detected, the fingerprint is embedded 604 in the audio input by the audio source to identify the trigger of the audio input as a false trigger. The audio input including the fingerprint is played by the audio source 606. Therefore, when the fingerprint is detected at the device, the device is instructed to ignore the trigger 608 and not activate.

[0028] Figure 7 It shows that Figure 1-6Method 700 for improving the accuracy and efficiency of wake-up confirmation using fingerprints described in one or more embodiments referenced in . Method 700 uses information and data collected from multiple audio inputs (1, 2 ... n) played by an audio source 702 and received by an audio device 704, which causes a user 706 to cancel (C1, C2 ... Cn) the activation (E1, E2 ... En) of the device 704. Each cancellation (C1, C2 ... C3) results in the generation 708 of a fingerprint associated with the audio input (1, 2 ... n). The information collected about the cancellation (C1, C2 ... Cn) is collected in a cloud-based service 108 and statistically analyzed 710. Based on the results of the statistical analysis 710, a final fingerprint is generated and synchronized 712 with the device 704. The identification, tracking and correction of false triggers will reduce or eliminate accidental activation of the device. In the case where a statistically feasible number of generated fingerprints is not met, the method 700 continues to monitor 714 newly generated fingerprints that occur based on the user's command to cancel the activation of the device.

[0029] In the foregoing description, the present disclosure has been described with reference to specific exemplary embodiments. However, various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the claims. The description and drawings are illustrative rather than restrictive, and these modifications are intended to be included within the scope of the present disclosure. Therefore, the scope of the present disclosure should be determined by the claims and their legal equivalents, rather than by the examples described alone.

[0030] For example, the steps recited in any method or process claim may be performed in any order and are not limited to the specific order presented in the claims. In addition, the components and / or elements recited in any device claim may be assembled or otherwise operably configured into various permutations and are therefore not limited to the specific configuration recited in the claims.

[0031] Benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments; however, any benefit, advantage, solution to problems, or any element that may cause any particular benefit, advantage, or solution to occur or become more apparent is not to be construed as a critical, required, or essential feature or component of any or all the claims.

[0032] The terms "comprise / comprises / comprising," "having," "including / includes," or any variations thereof, are intended to refer to a non-exclusive inclusion, such that a process, method, article, composition, or apparatus that includes a list of elements includes not only those stated elements, but may also include other elements not expressly listed into or inherent to such process, method, article, composition, or apparatus. Other combinations and / or modifications of the above-described structures, arrangements, applications, proportions, elements, materials, or components used in the practice of the present disclosure, in addition to those not specifically stated, may also be varied or otherwise specifically adapted to a particular environment, manufacturing specification, design parameters, or other operating requirements without departing from the general principles of the present disclosure.

Claims

1. A method for confirming the triggering of a voice-activated smart device, the method being performed on a device having a processing unit including a non-transitory computer-readable storage medium capable of executing instructions of a software program, the method comprising the steps of: receiving an audio input signal at an audio source; Listen for triggers in the audio input; Upon hearing the trigger, performing wake-up confirmation by detecting a fingerprint in the audio input, wherein performing wake-up confirmation further comprises the following steps: monitoring, in a cloud-based environment, an occurrence of a user cancelling a command at the voice-activated smart device as a result of a false trigger; generating a fingerprint to associate the trigger in the audio input with a false trigger; Performing statistical analysis on the generated fingerprints; Based on the results of the statistical analysis, generating a final fingerprint defining the trigger as a false trigger; and synchronizing the final fingerprint with the voice-activated intelligent device to detect and identify the trigger as false in subsequent audio input containing the trigger; Upon confirming that the audio input contains a trigger and a fingerprint, instructing the voice-activated smart device to ignore the audio input.

2. The method according to claim 1, wherein the step of performing a wake-up confirmation further comprises the following steps: In a cloud-based environment, the trigger heard in the audio input is compared to a database of triggers known to be false.

3. The method according to claim 1, wherein the step of performing a wake-up confirmation further comprises the following steps: In a cloud-based environment, the fingerprint detected in the audio input is compared to a database of known fingerprints.

4. The method according to claim 1, wherein the step of performing a wake-up confirmation further comprises the following steps: A fingerprint embedded in the audio input is detected to indicate that the audio input includes a false trigger.

5. The method of claim 1, wherein the step of detecting a fingerprint embedded in the audio input further comprises: A fingerprint embedded in the audio input at its source is detected.

6. The method of claim 5, wherein the step of detecting a fingerprint embedded in the audio input is performed at the audio source.

7. The method of claim 5, wherein the step of detecting a fingerprint embedded in the audio input is performed at the voice-activated smart device.

8. The method of claim 1, wherein the step of detecting a fingerprint embedded in the audio input further comprises: A fingerprint embedded by the audio source is detected at the voice-activated intelligent device.

9. The method of claim 8, wherein the fingerprint embedded by the audio source is selected from the group consisting of: an ultrasonic signal, a sub-audio tone, an infrared signal, an identification of inherent distortion in the audio input that occurs as a result of an audio source transmitting the audio input.

10. A method for detecting false triggers in an audio signal detected by a voice activated smart device, the method being performed on a device having a processing unit including a non-transitory computer readable storage medium capable of executing instructions of a software program, the method comprising the steps of: receiving an audio input signal at an audio source; Before propagating the audio input signal, performing wake-up confirmation in a processing unit associated with the audio source by detecting a false trigger in the audio input, wherein performing wake-up confirmation further comprises the following steps: monitoring, in a cloud-based environment, an occurrence of a user cancelling a command at the voice-activated smart device as a result of a false trigger; generating a fingerprint to associate the trigger in the audio input with a false trigger; Performing statistical analysis on the generated fingerprints; Based on the results of the statistical analysis, generating a final fingerprint defining the trigger as a false trigger; and synchronizing the final fingerprint with the voice-activated intelligent device to detect and identify the trigger as false in subsequent audio input containing the trigger; Upon detecting a false trigger in the audio input, the audio source transmits a fingerprint to the voice-activated smart device; detecting the fingerprint at the voice-activated smart device; and The audio input is ignored at the voice activated intelligent device.

11. The method according to claim 10, wherein the step of detecting false triggering further comprises: A fingerprint embedded in the audio input signal during pre-production and known to the audio source is identified.

12. The method according to claim 10, wherein the step of detecting false triggering further comprises: A distortion analysis is performed that identifies inherent distortion caused by the replicated speech.

13. The method of claim 10, wherein the step of transmitting a fingerprint to the voice-activated smart device further comprises: A fingerprint is embedded in the audio signal to be detected at the voice activated smart device.

14. The method according to claim 13, wherein the step of embedding a fingerprint in the audio signal further comprises: An ultrasonic overlay is applied to the audio signal.

15. The method according to claim 13, wherein the step of embedding a fingerprint in the audio signal further comprises: Sub-audio tones are inserted into the audio signal.

16. The method of claim 13, wherein the step of embedding a fingerprint in the audio signal further comprises: An infrared blast is inserted into the audio signal.

17. The method according to claim 10, further comprising the steps of: processing the audio input to determine whether the audio input is human-generated speech; Upon determining that the audio input is human-generated speech, monitoring the audio input for a trigger; upon determining that the audio input is human-generated speech and contains the trigger, activating the voice-activated smart device; as well as Upon determining that the audio input is not human-generated speech and includes the trigger, the trigger is ignored at the voice-activated intelligent device.

18. The method of claim 17, wherein the step of processing the audio input to determine whether the audio input is human-generated speech further comprises: A fingerprint in the audio input is detected that indicates the audio input is not human-generated speech.

19. The method of claim 18, wherein the step of detecting a fingerprint in the audio input confirming that the audio input is not human-generated speech further comprises the step of performing a distortion analysis to detect the fingerprint.

Images