Enabling speech recognition

JP7645230B2Active Publication Date: 2025-03-13QUALCOMM INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2022504699
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-07-30
Filing Date
2020-07-30
Publication Date
2025-03-13
Estimated Expiration
2040-07-30

Smart Images

  • Figure 0007645230000001
    Figure 0007645230000001
  • Figure 0007645230000002
    Figure 0007645230000002
  • Figure 0007645230000003
    Figure 0007645230000003
Patent Text Reader

Abstract

The device for processing an audio signal representing an input sound includes a hand detector configured to generate a first instruction in response to detecting at least a portion of a hand covering at least a portion of the device, and also includes an automatic speech recognition system configured to be enabled to process the audio signal in response to the first instruction.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Claiming priority under 35 U.S.C. § 119

[0001] This patent application claims priority to non-provisional application Ser. No. 16 / 526,608, entitled "ACTIVATING SPEECH RECOGNITION," filed on July 30, 2019, assigned to the assignee of this application, and expressly incorporated by reference herein.

[0002] FIELD OF THE DISCLOSURE

[0002] The present disclosure relates generally to speech recognition, and more particularly, to activating a speech enablement system. [Background technology]

[0003]

[0003] Speech recognition is often used to enable an electronic device to interpret verbal questions or commands from a user. Such verbal questions or commands may be identified by analyzing an audio signal, such as a microphone input, in an automatic speech recognition (ASR) engine that generates a text output of the verbal questions or commands. An "always-on" ASR system enables an electronic device to continuously examine the audio input to detect user commands or questions in the audio input. However, the continuous operation of an ASR system results in a relatively large power consumption, which shortens the battery life when implemented in a mobile device.

[0004]

[0004] In some devices, spoken voice commands are not recognized unless preceded by a spoken activation keyword. Recognition of the activation keyword allows such devices to activate an ASR engine to process the voice command. However, speaking the activation keyword before every command takes extra time and requires the speaker to use correct pronunciation and proper intonation. In other devices, a dedicated button is provided that the user presses to initiate speech recognition. However, in some situations, such as when driving a vehicle, finding and accurately pressing the button may distract the user from other tasks. Summary of the Invention

[0005] According to one implementation of the present disclosure, a device for processing an audio signal representative of an input sound includes a hand detector configured to generate a first indication in response to detection of at least a portion of a hand covering at least a portion of the device. The device also includes an automatic speech recognition system configured to be enabled to process the audio signal in response to the first indication.

[0006] According to another aspect of the present disclosure, a method for processing an audio signal representative of an input sound includes detecting, at a device, at least a portion of a hand covering at least a portion of the device. The method also includes, in response to detecting the portion of the hand covering the portion of the device, enabling an automatic speech recognition system to process the audio signal.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable medium includes instructions that, when executed by one or more processors of a device, cause the one or more processors to perform operations for processing an audio signal representative of an input sound, the operations including detecting at least a portion of a hand covering at least a portion of the device, and in response to detecting the portion of the hand covering the portion of the device, enabling an automatic speech recognition system to process the audio signal.

[0008] According to another aspect of the present disclosure, an apparatus for processing an audio signal representative of an input sound includes means for detecting at least a portion of a hand covering at least a portion of a device and means for processing the audio signal, the means for processing configured to be enabled in response to detecting the portion of the hand covering the portion of the device. [Brief description of the drawings]

[0009] [Figure 1]

[0009] FIG. 1 illustrates a particular exemplary implementation of a system including a device operable to enable speech recognition. [Diagram 2]

[0010] 2 includes a diagram of certain examples of components that may be implemented in the device of FIG. 1. [Diagram 3]

[0011] FIG. 2 is a diagram of another specific implementation of the device of FIG. 1. [Figure 4]

[0012] FIG. 1 is a diagram of another particular illustrative implementation of a device operable to enable speech recognition. [Diagram 5]

[0013] 2 is a diagram of a particular implementation of a method for enabling speech recognition that may be performed by the device of FIG. 1; [Figure 6]

[0014] 2 is a diagram of another implementation of a method for enabling speech recognition that may be performed by the device of FIG. 1 . [Figure 7]

[0015] 1 is a diagram of a vehicle operable to enable speech recognition. [Figure 8]

[0016] FIG. 1 is a diagram of a virtual reality or augmented reality headset operable to enable speech recognition. [Figure 9]

[0017] 1 is a block diagram of a particular illustrative example of a device operable to enable speech recognition; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010]

[0018] A device and method for enabling a speech recognition system are disclosed. An always-on ASR system that continuously examines an audio input to detect a user's command or question therein results in a relatively large power consumption, resulting in a short battery life when the ASR engine is implemented in a mobile device. In an attempt to reduce power consumption, some systems may use a low-capacity speech recognition processor that consumes less power than a full-power ASR engine to perform keyword detection on the audio input. Once an enabling keyword is detected, a full-power ASR engine can be enabled to process the voice command that follows the enabling keyword. However, requiring a user to speak the enabling keyword before every command is time-consuming and requires the speaker to use correct pronunciation and proper intonation. A device that requires a user to press a dedicated button to start speech recognition may result in a dangerous distraction of the user's attention, such as when driving a car.

[0011]

[0019] As described herein, speech recognition is enabled in response to detecting a hand covering a portion of the device, such as a user's hand hovering over the screen of the device. A user can enable speech recognition for voice commands by placing a user's hand over the device, without having to speak an enablement keyword or precisely locate and press a dedicated button. Removal of the user's hand over the device can signal that the user has finished speaking a voice command. As a result, speech recognition can be conveniently and safely enabled, such as when the user is driving a car. Furthermore, since placing a user's hand over the device can signal the device to begin speech recognition, and removing the user's hand over the device signals the end of the user's voice command, both inappropriate activation of speech recognition and inaccurate detection of the end of a voice command can be reduced.

[0012]

[0020] Unless expressly limited by context, the term "producing" is used to indicate any of its ordinary meanings, such as computing, generating, and / or providing. Unless expressly limited by context, the term "providing" is used to indicate any of its ordinary meanings, such as computing, generating, and / or producing. Unless expressly limited by context, the term "coupled" is used to indicate a direct or indirect electrical or physical connection. If the connection is indirect, there may be other blocks or components between the "coupled" structures. For example, a loudspeaker may be acoustically coupled to a nearby wall through an intervening medium (e.g., air) that allows for the propagation of waves (e.g., sound) from the loudspeaker to the wall (or vice versa).

[0013]

[0021] The term "configuration" may be used in reference to a method, an apparatus, a device, a system, or any combination thereof, as indicated by its particular context. The term "comprising," when used in this specification and claims, does not exclude other elements or operations. The term "based on" (such as in "A is based on B") is used to indicate any of its ordinary meanings, including (i) "based at least on" (e.g., "A is at least based on B"), and, where appropriate in the particular context, (ii) "equal to" (e.g., "A is equal to B"). In the case (i) where "A is based on B" includes "based at least on," this may include a configuration in which A is coupled to B. Similarly, the term "in response to" is used to indicate any of its ordinary meanings, including "at least in response to." The term "at least one" is used to indicate any of its ordinary meanings, including "one or more." The term "at least two" is used to indicate any of its ordinary meanings, including "two or more."

[0014]

[0022] The terms "apparatus" and "device" are used generically and interchangeably unless otherwise indicated by the specific context. Unless otherwise indicated, any disclosure of the operation of an apparatus having a particular feature is also expressly intended to disclose a method having similar features (and vice versa), and any disclosure of the operation of an apparatus with a particular configuration is also expressly intended to disclose a method with a similar configuration (and vice versa). The terms "method," "process," "procedure," and "technique" are used generically and interchangeably unless otherwise indicated by the specific context. The terms "element" and "module" may be used to indicate a portion of a larger configuration. The term "packet" may correspond to a unit of data that includes a header portion and a payload portion. Any incorporation by reference of a portion of a document shall also be understood to incorporate definitions of terms or variables referenced in that portion, if such definitions appear elsewhere in the document, as well as in any figures referenced in the incorporated portion.

[0015]

[0023] As used herein, the term "communication device" refers to an electronic device that can be used for voice and / or data communication over a wireless communication network. Examples of communication devices include smart speakers, speaker bars, mobile phones, personal digital assistants (PDAs), handheld devices, headsets, wireless modems, laptop computers, personal computers, etc.

[0016]

[0024] FIG. 1 illustrates a system 100 including a device 102 configured to enable an ASR system 140 to process input sounds 106, such as voice commands, when at least a portion of a hand 190 is placed over the device 102. The device 102 includes one or more microphones represented as microphone 112, a screen 110, one or more sensors 120, a hand detector 130, and an ASR system 140. A perspective view 180 illustrates the hand 190 placed over the device 102, and a block diagram 182 illustrates components of the device 102. In some implementations, as illustrative and non-limiting examples, the device 102 may include a portable communication device (e.g., a “smartphone”), a vehicle system (e.g., a speech interface for an automobile entertainment system, a navigation system, or an automated driving control system), a virtual reality or augmented reality headset, or a wireless speaker and voice command device with an integrated assistant application (e.g., a “smart speaker” device).

[0017]

[0025] The microphone 112 is configured to generate an audio signal 114 in response to the input sound 106. In some implementations, the microphone 112 is configured to be enabled to generate the audio signal 114 in response to an indication 132, as will be further described with reference to FIG.

[0018]

[0026] The one or more sensors 120 are coupled to the hand detector 130 and configured to provide sensor data 122 to the hand detector 130. For example, the sensors 120 may include one or more cameras, such as a low-power ambient light sensor or a main camera, an infrared sensor, an ultrasonic sensor, one or more other sensors, or any combination thereof, such as those further described with reference to FIG.

[0019]

[0027] The hand detector 130 is configured to generate an indication 132 in response to detecting at least a portion of a hand covering at least a portion of the device 102, such as the screen 110. As used herein, as an illustrative, non-limiting example, "at least a portion of a hand" may correspond to any portion of a hand (e.g., one or more fingers other than the thumb, the thumb, the palm or the back of the hand, or any portion thereof, or any combination thereof) or may correspond to the entire hand. As used herein, "detecting a hand" is equivalent to "detecting at least a portion of a hand" and may include, as illustrative, non-limiting examples, detecting two or more fingers other than the thumb, detecting at least one finger other than the thumb connected to a portion of the palm, detecting a thumb and at least one finger other than the thumb, detecting a thumb connected to at least a portion of the palm, or detecting the entire hand (e.g., four fingers other than the thumb, the thumb, and the palm).

[0020]

[0028] The hand 190 is described as being detected "over" the device 102, where "covering" the device 102 refers to being located at a specified relative location (or within a specified range of locations) relative to the location or orientation of the one or more sensors 120. In an example where the device 102 is oriented such that the sensors 120 point up, such as that shown in FIG. 1, detecting the hand 190 covering the device 102 indicates that the hand 190 is above the device 102. In an example where the device 102 is oriented such that the sensors 120 point down, detecting the hand 190 covering the device 102 indicates that the hand 190 is below the device 102.

[0021]

[0029] The hand detector 130 is configured to process the sensor data 122 to determine whether a hand 190 covering the device 102 is detected. For example, as further described with reference to FIG. 2, in some implementations, the hand detector 130 processes image data to determine whether a hand shape is captured by a camera, processes infrared data to determine whether a detected temperature of the hand 190 corresponds to a hand temperature, processes ultrasound data to determine whether a distance between the hand 190 and the device 102 is within a specified range, or a combination thereof.

[0022]

[0030] In some implementations, the device 102 may be configured to generate a notification for a user of the device 102 to indicate that speech recognition has been enabled in response to detecting a hand 190 covering the device 102, and may be further configured to generate a second notification to indicate that voice input for speech recognition is disabled in response to no longer detecting a hand 190 covering the device 102. For example, the device 102 may be configured to generate an audio signal, such as a chime or a voice message, such as "ready," a visual signal, such as a solid or flashing light, a digital signal to be played by another device, such as a car entertainment system, in communication with the device, or any combination thereof. Generating a notification may allow a user to confirm that the device 102 is ready to receive voice commands and may further allow the user to detect and prevent false activations (e.g., caused by another object that may be mistakenly identified as a hand 190) and missed activations due to improper placement of the hand 190. Because each activation of the ASR system 140 consumes power and uses processing resources, reducing false activations results in reduced power consumption and processing resource usage.

[0023]

[0031] ASR system 140 is configured to be enabled in response to instruction 132 to process audio signal 114. In an illustrative example, a particular bit in a control register represents the presence or absence of instruction 132, and control circuitry within or coupled to ASR system 140 is configured to read that particular bit. A value of "1" in that bit corresponds to instruction 132 and causes ASR system 140 to be enabled. In other implementations, as illustrative, non-limiting examples, instruction 132 is instead implemented as a digital or analog signal on a bus or control line, an interrupt flag in an interrupt controller, or an optical or mechanical signal.

[0024]

[0032] When enabled, the ASR system 140 is configured to process one or more portions (e.g., frames) of the audio signal 114 that includes the input sound 106. For example, the device 102 may buffer a series of frames of the audio signal 114 as sensor data 122 being processed by the hand detector 130 such that, upon generation of the instruction 132, the ASR system 140 may process the buffered series of frames of the audio signal 114 to generate an output indicative of the user's speech. The ASR system 140 may provide the recognized speech 142 as a text output of the speech content of the input sound 106 to another component of the device 102, such as a "virtual assistant" application as described with reference to FIG. 3 or other application, to initiate an activity based on the speech content.

[0025]

[0033] When disabled, ASR system 140 does not process audio signal 114 and consumes less power than when enabled. For example, to reduce dynamic power consumption, deactivation of ASR system 140 may include gating input circuitry of ASR system 140 to prevent audio signal 114 from being input to ASR system 140, gating clock signals to prevent switching of circuits within ASR system 140, or both. As another example, deactivation of ASR system 140 may include reducing the power supply to ASR system 140 to reduce static power consumption without losing state of circuit elements, removing power from at least some portion of ASR system 140, or a combination thereof.

[0026]

[0034] In some implementations, the hand detector 130, the ASR system 140, or any combination thereof are implemented using dedicated circuitry or hardware. In some implementations, the hand detector 130, the ASR system 140, or any combination thereof are implemented via firmware or software execution. By way of example, the device 102 may include a memory configured to store instructions and one or more processors configured to execute instructions to implement the hand detector 130 and the ASR system 140, such as those further described with reference to FIG. 9.

[0027]

[0035] During operation, a user may place a user's hand 190 over the device 102 before speaking a voice command. The hand detector 130 processes the sensor data 122 to determine that the hand 190 is covering the device 102. In response to detecting the hand 190 covering the device 102, the hand detector 130 generates an indication 132, which triggers activation of the ASR system 140. After receiving the voice command at the microphone 112, the ASR system 140 processes a corresponding portion of the audio signal 114 to generate a recognized utterance 142 indicative of the voice command.

[0028]

[0036] Activation of the ASR system 140 when a hand covering the device 102 is detected allows a user of the device 102 to enable speech recognition for voice commands by placing the user's hand 190 over the device without having to speak an activation keyword or precisely locate and press a dedicated button. As a result, speech recognition may be conveniently and safely enabled, such as when the user is driving a car. Furthermore, because placing the user's hand over the device signals the device to begin speech recognition, inappropriate activation of speech recognition may both be reduced when compared to systems that instead use keyword detection to enable speech recognition.

[0029]

[0037] 2 illustrates an example 200 illustrating further aspects of components that may be implemented in device 102 of FIG. 1. As illustrated in FIG. 2, sensor 120 includes one or more cameras 202 configured to provide image data 212 to hand detector 130, an infrared (IR) sensor 208 configured to provide infrared sensor data 218 to hand detector 130, and an ultrasonic sensor 210 configured to provide ultrasonic sensor data 220 to hand detector 130. Image data 212, infrared sensor data 218, and ultrasonic sensor data 220 are included in sensor data 122. Camera 202 includes a low power ambient light sensor 204 configured to generate at least a portion of image data 212, a main camera 206 configured to generate at least a portion of image data 212, or both. Although the main camera 206 can capture image data having a higher resolution than the ambient light sensor 204, the ambient light sensor 204 can generate image data having sufficient resolution to perform hand detection and operates using less power than the main camera 206.

[0030]

[0038] The hand detector 130 includes a hand pattern detector 230, a hand temperature detector 234, a hand distance detector 236, and an activation signal unit 240. The hand pattern detector 230 is configured to process the image data 212 to determine whether the image data 212 includes a hand pattern 232. In one example implementation, the hand pattern detector 230 processes the image data 212 using a neural network that is trained to recognize the hand pattern 232. In another example implementation, the hand pattern detector 230 applies one or more filters to the image data 212 to identify the hand pattern 232. The hand pattern detector 230 is configured to transmit a first signal 231 to the activation signal unit 240 indicating whether the hand pattern 232 is detected. Although a single hand pattern 232 is shown, other implementations may include multiple hand patterns representing different aspects of the hand, such as a fingers-together pattern, a fingers-spread pattern, a partial hand pattern, etc.

[0031]

[0039] The hand temperature detector 234 is configured to process the infrared sensor data 218 from the infrared sensor 208 and send a second signal 235 to the enabling signal unit 240 indicating whether the infrared sensor data 218 indicates a temperature source having a temperature indicative of a human hand. In some implementations, the hand temperature detector 234 is configured to determine whether at least a portion of the field of view of the infrared sensor 208 has a temperature source that is in a temperature range indicative of a human hand. In some implementations, the hand temperature detector 234 is configured to receive data indicative of a hand location from the hand pattern detector 230 to determine whether a temperature source at the hand location matches the temperature range of a human hand.

[0032]

[0040] The hand distance detector 236 is configured to determine a distance 250 between the hand 190 and at least a portion of the device 102. In one example, the hand distance detector 236 processes the ultrasonic sensor data 220 and generates a third signal 237 indicating whether the hand 190 is within a specified distance range 238. In some implementations, the hand distance detector 236 receives data indicative of the position of the hand 190 from the hand pattern detector 230, from the hand temperature detector 234, or from both, and uses the hand position data to determine an area in the field of view of the ultrasonic sensor 210 that corresponds to the hand 190. In other implementations, the hand distance detector 236 locates the hand 190 by finding the closest object to the screen 110 that is more than a specified portion (e.g., 25%) of the field of view of the ultrasonic sensor 210.

[0033]

[0041] In certain implementations, the range 238 has a lower boundary of 10 centimeters (cm) and an upper boundary of 30 cm (i.e., the range 238 includes distances equal to or greater than 10 cm and equal to or less than 30 cm). In other implementations, the range 238 is adjustable. For example, the device 102 may be configured to perform an update operation in which the user positions the hand 190 in a preferred location relative to the device 102 such that the distance 250 may be detected and used to generate the range 238 (e.g., by applying a lower offset from the detected distance 250 to set a lower boundary and applying an upper offset from the detected distance 250 to set an upper boundary).

[0034]

[0042] The enabling signal unit 240 is configured to generate the indication 132 in response to a first signal 231 indicating detection of a hand pattern 232 in the image data 212, a second signal 235 indicating detection of a hand temperature within a human hand temperature range, and a third signal 237 indicating detection of a hand 190 within a range 238 (e.g., the hand 190 is at a distance 250 of 10 to 30 centimeters from the screen 110). For example, in an implementation in which the signals 231, 235, and 237 each have a binary "1" value indicating detection and a binary "0" value indicating no detection, the enabling signal unit 240 may generate the indication 132 as a logical AND of the signals 231, 235, and 237 (e.g., the indication 132 has a value of 1 in response to all three signals 231, 235, 237 having a value of 1). In another example, the enable signal unit 240 is also configured to generate the indication 132 having a value of one in response to any two of the signals 231, 235, 237 having a value of one.

[0035]

[0043] In other implementations, one or more of the signals 231, 235, and 237 have a multi-bit value indicating a probability that a corresponding hand detection criterion is satisfied. For example, the first signal 231 may have a multi-bit value indicating a confidence that a hand pattern is detected, the second signal 235 may have a multi-bit value indicating a confidence that a hand temperature is detected, and the third signal 237 may have a multi-bit value indicating a confidence that a distance of the hand 190 from the device 102 is within a range 238. The enabling signal unit 240 may combine the signals 231, 235, and 237 and compare the combined result to a threshold to generate the indication 132. For example, the enabling signal unit 240 may apply a set of weights to determine a weighted sum of the signals 231, 235, and 237. In response to the weighted sum exceeding the threshold, the enabling signal unit 240 may output the indication 132 having a value indicating hand detection. As described further below, the weights and threshold values ​​may be hard-coded, or alternatively, may be dynamically or periodically adjusted based on user feedback regarding false positives and false negatives.

[0036]

[0044] In some implementations, the hand detector 130 is further configured to generate a second indication 242 in response to detecting that the hand 190 is no longer covering the device 102. For example, the hand detector may output the second indication 242 as having a value of 0 (indicating that hand removal is not detected) in response to detecting the hand 190, and may update the second indication 242 to have a value of 1 in response to determining that the hand is no longer detected (e.g., to indicate a transition from a "hand detected" state to a "hand not detected" state). As further described with reference to FIG. 3, the second indication 242 may correspond to an end-of-utterance signal for the ASR system 140.

[0037]

[0045] 2 shows multiple sensors including an ambient light sensor 204, a main camera 206, an infrared sensor 208, and an ultrasonic sensor 210, but in other implementations, one or more of the ambient light sensor 204, the main camera 206, the infrared sensor 208, or the ultrasonic sensor 210 are omitted. For example, the ambient light sensor 204 enables generation of at least a portion of the image data 212 for detecting a hand shape using less power compared to using the main camera 206, but in some implementations, the ambient light sensor 204 is omitted and the main camera 206 is used to generate the image data. For power reduction, the main camera 206 may be operated according to an on / off duty cycle, such as at quarter second intervals, for hand detection. As another example, the main camera 206 enables generation of at least a portion of the image data 212 to detect the shape of the hand with higher resolution and therefore more accuracy compared to using the ambient light sensor 204, although in some implementations the main camera 206 is omitted and the ambient light sensor 204 is used to generate the image data 212.

[0038]

[0046] As another example, the infrared sensor 208 enables generation of infrared sensor data 218 to detect whether an object has a temperature consistent with that of a person's hand, although in other implementations the infrared sensor 208 is omitted and the device 102 performs hand detection regardless of temperature. As another example, the ultrasonic sensor 210 enables generation of ultrasonic sensor data 220 to detect whether a distance to an object is within range 238, although in other implementations the ultrasonic sensor 210 is omitted and the device 102 performs hand detection regardless of distance from the device 102. Alternatively, one or more other mechanisms may be implemented for detecting distance, such as, by way of illustrative, non-limiting examples, comparing object positions in image data from multiple cameras of device 102 (e.g., parallax) or multiple cameras of a different device (e.g., an automobile in which device 102 is located) to estimate distance 250, using detected hand size in image data 212 or infrared sensor data 218 to estimate distance 250, or by projecting a structured light or other electromagnetic signal to estimate object distance.

[0039]

[0047] Increasing the number of sensors and variety of sensor types generally increases the accuracy of hand detection, but in some implementations, two sensors or a single sensor provides sufficient accuracy for hand detection. As a non-limiting example, in some implementations, the only sensor data used for hand detection is image data 212 from the ambient light sensor 204. In some implementations, the sensors 120 are continuously enabled, while in other implementations, one or more of the sensors 120 are controlled according to a "cascade" operation in which power is conserved by keeping one or more of the sensors 120 disabled until a hand detection criterion is met based on sensor data from another of the sensors 120. By way of example, the main camera 206, the infrared sensor 208, and the ultrasonic sensor 210 may remain disabled until the hand pattern detector 230 detects a hand pattern 232 in the image data 212 generated by the ambient light sensor 204, in response to which one or more of the main camera 206, the infrared sensor 208, and the ultrasonic sensor 210 are enabled to provide additional sensor data for the purpose of improving the accuracy of hand detection.

[0040]

[0048] 3 shows an example 300 illustrating further aspects of components that may be implemented in device 102. As shown in FIG. 3, activation circuitry 302 is coupled to hand detector 130 and ASR system 140, which includes a buffer 320 accessible to ASR engine 330. Device 102 also includes a virtual assistant application 340 and a speaker 350 (e.g., device 102 implemented as a wireless speaker and voice command device).

[0041]

[0049] The enablement circuit 302 is configured to enable the automatic speech recognition system 140 in response to receiving the instruction 132. For example, the enablement circuit 302 is configured to generate an activation signal 310 in response to the instruction 132 transitioning to a state indicating hand detection (e.g., the instruction 132 transitioning from a value of 0 indicating no hand detection to a value of 1 indicating hand detection). The enablement signal 310 is provided to the ASR system 140 via a signal 306 to enable the ASR system 140. Enabling the ASR system 140 includes commencing buffering of the audio signal 114 in a buffer 320 to generate buffered audio data 322. The enablement signal 310 is also provided to the microphone 112 via an enable microphone 112 signal 304 to enable the microphone to generate the audio signal 114.

[0042]

[0050] The enable circuit 302 is also configured to generate an end of speech signal 312. For example, the enable circuit 302 is configured to generate the end of speech signal 312 in response to the second indication 242 transitioning to a state indicating an end of hand detection (e.g., the second indication 242 transitioning from a value of 0 (indicating no change in hand detection) to a value of 1 (indicating that the detected hand is no longer detected)). The end of speech signal 312 is provided to the ASR system 140 via signal 308 to cause the ASR engine 330 to begin processing the buffered audio data 332.

[0043]

[0051] The enable circuit 302 is configured to selectively enable one or more components of the ASR system 140. For example, the enable circuit 302 may include or be coupled to a power management circuit, a clock circuit, a head switch or foot switch circuit, a buffer control circuit, or any combination thereof. The enable circuit 302 may be configured to initiate power-on of the buffer 320, the ASR engine 330, or both, such as by selectively applying or raising the voltage of the power supplies of the buffer 320, the ASR engine 330, or both. As another example, the enable circuit 302 may be configured to selectively gate or un-gate a clock signal to the buffer 320, the ASR engine 330, or both, such as to prevent circuit operation without removing a power supply.

[0044]

[0052] The recognized speech 142 output by the ASR system 140 is provided to a virtual assistant application 340. For example, the virtual assistant application 340 may be implemented by one or more processors executing instructions, such as those described in more detail with reference to FIG. 9. The virtual assistant application 340 may be configured to execute one or more search queries, such as via a wireless connection to an Internet gateway, search server, or other resource, a search of the device 102's local storage, or a combination thereof.

[0045]

[0053] To illustrate, the audio signal 114 may represent the spoken question "What's the weather today?" The virtual assistant application 340 may generate a query to access an Internet-based weather service to obtain a weather forecast for the geographic region in which the device 102 is located. The virtual assistant application 340 is configured to generate an output, such as an output audio signal 342, which causes a speaker 350 to generate an audible output, such as in a voice interface implementation. In other implementations, the virtual assistant application 340 generates another mode of output, such as a visual output signal that may be displayed by a screen or display integrated with or coupled to the device 102.

[0046]

[0054] In some implementations, values ​​of parameters such as weights and thresholds used by device 102 (e.g., in hand detector 130) may be set by a manufacturer or provider of device 102. In some implementations, device 102 is configured to adjust one or more such values ​​during the lifetime of device 102 based on detected false negatives, false activations, or combinations thereof associated with ASR system 140. A history of false activations may be maintained by device 102 such that characteristics of sensor data 122 that caused false activations may be periodically used to automatically adjust one or more weights or thresholds, such as to highlight the relative reliability of one sensor over another in using in hand detection to reduce the probability of future false activations.

[0047]

[0055] 1-3, it will be understood that such values ​​are provided for illustrative purposes only and not for limitation. To illustrate, in some implementations, the indication 132 is indicated by a value of "0." As another example, in some implementations, a value of "1" for the first signal 231 indicates a high probability that the hand pattern 232 is in the image data 212, while in other implementations, a value of "0" for the first signal 231 indicates a low probability that the hand pattern 232 is in the image data 212. Similarly, in some implementations, a value of "1" for the second signal 235, the third signal 237, or both indicates a high probability that the hand detection criteria is met, while in other implementations, a value of "1" for the second signal 235, the third signal 237, or both indicates a high probability that the hand detection criteria is not met.

[0048]

[0056] FIG. 4 illustrates an implementation 400 of a device 402 including a hand detector 130 and an ASR system 140 integrated into separate components such as a semiconductor chip or package, as will be further described with reference to FIG. 9. The device 402 includes an audio signal input 410, such as a first bus interface, to allow an audio signal 114 to be received from a microphone external to the device 402. The device 402 also includes a sensor data input 412, such as a second bus interface, to allow sensor data 122 to be received from one or more sensors external to the device 402. The device 402 may further include one or more outputs to provide processing results (e.g., the recognized speech 142 or the output audio signal 342) to one or more external components (e.g., a speaker 350). The device 402 enables implementation of hand detection and voice recognition enablement as a component in a system including microphones and other sensors, such as in a vehicle as shown in FIG. 7, a virtual reality or augmented reality headset as shown in FIG. 8, or a wireless communication device as shown in FIG. 9.

[0049]

[0057] 5, a particular implementation of a method 500 for processing an audio signal representing an input sound that may be performed by device 102 or device 402 is illustrated. The method begins at 502 and includes determining whether a hand is covering the screen of the device at 504, such as by hand detector 130 processing sensor data 122. In response to detecting the hand covering the screen, a microphone and buffer are enabled at 506. For example, microphone 112 and buffer 320 of FIG. 3 are enabled by enable circuit 302 via signals 304 and 306.

[0050]

[0058] In response to determining that the hand covering the screen has been removed, at 508, the method 500 includes enabling the ASR engine to process the buffered data, at 510. For example, the ASR engine 330 is enabled by a signal 308 generated by the enablement circuit 302 to process the buffered audio data 322.

[0051]

[0059] Enabling the ASR when a hand covering the screen is detected allows a user to enable speech recognition for voice commands by the placement of the user's hand without having to speak an enablement keyword or locate and press a dedicated button. As a result, speech recognition may be conveniently and safely enabled, such as when a user is driving a car. Furthermore, because placing a user's hand over the screen initiates enabling of components to receive voice commands for speech recognition, and removing the user's hand from covering the screen initiates processing of the received voice commands, improper enabling, disabling, or both of speech recognition may be reduced when compared to systems that instead use keyword detection to enable speech recognition.

[0052]

[0060] Referring to FIG. 6, as an illustrative and non-limiting example, a particular implementation of a method 600 for processing an audio signal representing an input sound that may be performed by device 102 or device 402 is illustrated.

[0053]

[0061] The method 600 starts at 602 and includes detecting at least a portion of a hand covering at least a portion of a device at 604. For example, the hand detector 130 detects the hand 190 via processing sensor data 122 received from one or more sensors 120. In some implementations, detecting the portion of the hand covering the portion of the device includes processing image data (e.g., image data 212) to determine whether the image data includes a hand pattern (e.g., hand pattern 232). In an example, the image data is generated in a low power ambient light sensor of the device, such as ambient light sensor 204. Detecting the portion of the hand covering the portion of the device may further include processing infrared sensor data from an infrared sensor of the device, such as infrared sensor data 218. Detecting the portion of the hand covering the portion of the device may also include processing ultrasonic sensor data from an ultrasonic sensor of the device, such as ultrasonic sensor data 220.

[0054]

[0062] The method 600 also includes, at 606, enabling the automatic speech recognition system to process the audio signal in response to detecting the portion of the hand covering the portion of the device. For example, the device 102 enables the ASR system 140 in response to the instruction 132. In some implementations, enabling the automatic speech recognition system includes the device 102 (e.g., the enablement circuit 302) initiating buffering of the audio signal, such as by enabling the buffer 320 via signal 306. In some examples, in response to detecting the portion of the hand covering the portion of the device, such as the screen of the device, the method 500 further includes enabling a microphone to generate an audio signal based on the input sound, such as by the device 102 (e.g., the enablement circuit 302) enabling the microphone 112 via signal 304.

[0055]

[0063] In some implementations, the method 600 includes detecting 608 that the portion of the hand is no longer covering the portion of the device, and providing an end-of-speech signal to the automatic speech recognition system in response to detecting 610 that the portion of the hand is no longer covering the portion of the device. In one example, the hand detector 130 detects that the hand is no longer covering the portion of the device, and the enablement circuit 302 provides the end-of-speech signal 312 to the ASR engine 330 in response to the second instruction 242.

[0056]

[0064] By enabling the ASR system in response to detecting a hand covering a portion of the device, method 600 allows a user to enable speech recognition for voice commands without having to speak an enablement keyword or find and press a dedicated button. As a result, speech recognition may be conveniently and safely enabled, such as when a user is driving a car. Additionally, false enablement of the ASR system may both be reduced when compared to systems that instead use keyword detection to enable speech recognition.

[0057]

[0065] The method 500 of Figure 5, the method 600 of Figure 6, or both, may be implemented by a field programmable gate array (FPGA) device, an application specific integrated circuit (ASIC), a processing device such as a central processing unit (CPU), a DSP, a controller, another hardware device, a firmware device, or any combination thereof. As an example, the method 500 of Figure 5, the method 600 of Figure 6, or both, may be performed by a processor executing instructions such as those described with reference to Figure 9.

[0058]

[0066] 7 illustrates an example of an implementation 700 of the hand detector 130 and the ASR system 140 integrated into an automobile dashboard device, such as a car dashboard device 702. A visual interface device, such as a screen 110 (e.g., a touch screen display), is mounted within the car dashboard device 702 so as to be visible to the driver of the car. The microphone 112 and one or more sensors 120 are also mounted on the car dashboard device 702, although in other implementations, one or more of the microphone 112 and the sensors 120 may be located elsewhere in the automobile, such as the microphone 112 in the steering wheel or near the driver's head. The hand detector 130 and the ASR system 140 as shown with a dashed border to indicate that the hand detector 130 and the ASR system 140 are not visible to the automobile occupants. The hand detector 130 and the ASR system 140 may be implemented in a device that also includes the microphone 112 and the sensor 120, such as device 102 of FIGS. 1-3, or may be separate from and coupled to the microphone 112 and the sensor 120, such as device 402 of FIG. 4.

[0059]

[0067] In some implementations, multiple sets of microphones 112 and sensors 120 are integrated into the automobile. For example, a set of microphones and sensors may be placed at each passenger's seat, such as on an armrest control panel or a seatback display device, to allow each passenger to input voice commands using detection of hands covering the device. In some implementations, each passenger's voice commands may be communicated to a common ASR system 140, and in other implementations, the automobile includes multiple ASR systems 140 to allow simultaneous processing of voice commands from multiple occupants of the automobile.

[0060]

[0068] FIG. 8 illustrates an example of an implementation 800 of a hand detector 130 and an ASR system 140 integrated into a headset 802, such as a virtual reality or augmented reality headset. A screen 110 is placed in front of the user to enable display of an augmented reality or virtual reality image or scene to the user while the headset 802 is being worn, and a sensor 120 is positioned to detect when the user's hand covers (e.g., is in front of) the screen 110 to initiate ASR recognition. A microphone 112 is positioned to receive the user's voice while the headset 802 is being worn. While wearing the headset 802, the user can raise their hand in front of the screen 110 to indicate to the headset 802 that the user is about to speak a voice command to enable ASR, and can lower their hand to indicate that the user has finished speaking the voice command.

[0061]

[0069] 9 shows a block diagram of a particular example implementation of a device 900 including a hand detector 130 and an ASR engine 330, such as in a wireless communication device implementation (e.g., a smartphone). In various implementations, the device 900 may have more or fewer components than those shown in FIG. 9. In an example implementation, the device 900 may correspond to the device 102. In an example implementation, the device 900 may perform one or more operations described with reference to FIGS. 1-8.

[0062]

[0070] In particular implementations, device 900 includes a processor 906 (e.g., a central processing unit (CPU)). Device 900 may include one or more additional processors 910 (e.g., one or more DSPs). Processor 910 may include a speech and music coder-decoder (codec) 908 and a hand detector 130. Speech and music codec 908 may include a voice coder ("vocoder") encoder 936, a vocoder decoder 938, or both.

[0063]

[0071] The device 900 may include a memory 986 and a codec 934. The memory 986 may include instructions 956 that are executable by one or more additional processors 910 (or processor 906) to implement functions described with reference to the hand detector 130, the ASR engine 330, the ASR system 140 of FIG. 1, the enablement circuit 302, or any combination thereof. The device 900 may include a wireless controller 940 coupled to an antenna 952 via a transceiver 950.

[0064]

[0072] The device 900 may include a display 928 (e.g., screen 110) coupled to a display controller 926. The speaker 350 and microphone 112 may be coupled to a codec 934. The codec 934 may include a digital-to-analog converter 902 and an analog-to-digital converter 904. In certain implementations, the codec 934 may receive an analog signal from the microphone 112, convert the analog signal to a digital signal using the analog-to-digital converter 904, and provide the digital signal to a speech and music codec 908. The speech and music codec 908 may process the digital signal, which may be further processed by the ASR engine 330. In certain implementations, the speech and music codec 908 may provide the digital signal to the codec 934. The codec 934 may convert the digital signal to an analog signal using the digital-to-analog converter 902, and provide the analog signal to the speaker 350.

[0065]

[0073] In certain implementations, the device 900 may be included in a system-in-package or system-on-chip device 922. In certain implementations, the memory 986, the processor 906, the processor 910, the display controller 926, the codec 934, and the wireless controller 940 are included in the system-in-package or system-on-chip device 922. In certain implementations, the input device 930 (e.g., one or more of the sensors 120) and the power source 944 are coupled to the system-on-chip device 922. Further, in certain implementations, the display 928, the input device 930, the speaker 350, the microphone 112, the antenna 992, and the power source 944 are external to the system-on-chip device 922, as shown in FIG. 9. In certain implementations, each of the display 928, the input device 930, the speaker 350, the microphone 112, the antenna 992, and the power source 944 may be coupled to a component of the system-on-chip device 922, such as an interface or a controller.

[0066]

[0074] The device 900 may include a smart speaker (e.g., the processor 906 may execute instructions 956 to run a voice-controlled digital assistant application 340), a speaker bar, a mobile communications device, a smartphone, a mobile phone, a laptop computer, a computer, a tablet, a personal digital assistant, a display device, a television, a gaming console, a music player, a radio, a digital video player, a digital video disc (DVD) or Blu-ray® Disc player, a tuner, a camera, a navigation device, a virtual reality augmented reality headset, an automobile console device, or any combination thereof.

[0067]

[0075] With the described implementations, an apparatus for processing an audio signal representative of an input sound includes means for detecting at least a portion of a hand covering at least a portion of the device. For example, the means for detecting the portion of the hand may correspond to the hand detector 130, the hand pattern detector 230, the hand temperature detector 234, the hand distance detector 236, one or more circuits or components configured to detect at least a portion of a hand covering at least a portion of the device, or any combination thereof.

[0068]

[0076] The apparatus also includes means for processing an audio signal. The means for processing is configured to be enabled in response to detection of the portion of the hand covering the portion of the device. For example, the means for processing an audio signal may correspond to ASR system 140, ASR engine 330, microphone 112, codec 934, speech and music codec 908, one or more other circuits or components configured to process audio signals and enabled in response to detection of the portion of the hand covering the portion of the device, or any combination thereof.

[0069]

[0077] In some implementations, the device includes a means for displaying information, and the means for detecting is configured to detect that portion of the hand covering the means for displaying information. For example, the means for displaying information may include the screen 110, the display 928, a display controller 926, one or more other circuits or components configured to display information, or any combination thereof.

[0070]

[0078] The device may also include means for generating an audio signal based on the input sound, and a generating means configured to be enabled in response to detecting the portion of the hand covering the means for displaying information. For example, the means for generating an audio signal may correspond to the microphone 112, the microphone array, the codec 934, the speech and music codec 908, one or more other circuits or components configured to generate an audio signal based on the input sound and to be enabled in response to the first instruction, or any combination thereof.

[0071]

[0079] In some implementations, the device includes a means for generating image data, and a means for detecting, such as hand pattern detector 230, configured to determine whether the image data includes a hand pattern. In some implementations, the device includes at least one of a means for detecting a temperature associated with the portion of the hand (e.g., hand temperature detector 234, infrared sensor 208, or a combination thereof) and a means for detecting a distance of the portion of the hand from the device (e.g., hand distance detector 236, ultrasonic sensor 210, a camera array, a structured light projector, one or more other mechanisms for detecting the distance from the device to the portion of the hand, or any combination thereof).

[0072]

[0080] In some implementations, the non-transitory computer-readable medium (e.g., memory 986) includes instructions (e.g., instructions 956) that, when executed by one or more processors (e.g., processor 906, processor 910, or any combination thereof) of the device, cause the one or more processors to perform operations to process an audio signal representative of an input sound. The operations include detecting (e.g., in hand detector 130) at least a portion of a hand covering at least a portion of the device. For example, detecting the portion of the hand covering the portion of the device may include receiving sensor data 122, processing the sensor data 122 using one or more detectors (e.g., hand pattern detector 230, hand temperature detector 234, or hand distance detector 236) to determine whether one or more detection criteria are met, and generating an indication 132 at least in part in response to detecting that the one or more criteria are met (e.g., as described with reference to enablement signal unit 240). For example, in some implementations, processing the sensor data 122 to determine whether the detection criteria are met includes applying a neural network classifier (e.g., as described with reference to the hand pattern detector 230) trained to recognize the hand pattern 232 to process the image data 212, or applying one or more filters to the image data 212 to detect the hand pattern 232.

[0073]

[0081] The operations also include enabling an automatic speech recognition system to process the audio signal in response to detecting the portion of the hand covering the portion of the device. For example, enabling automatic speech recognition may include detecting instructions 132 at an input to ASR system 140 and performing at least one of a power-up or clock enable for at least one component of ASR system 140 (e.g., buffer 320, ASR engine 330) in response to detecting instructions 132.

[0074]

[0082] Moreover, those skilled in the art will appreciate that the various exemplary logical blocks, configurations, modules, circuits, and algorithm steps described with respect to the implementations disclosed herein may be implemented as electronic hardware, computer software executed by a processor, or a combination of both. Various exemplary components, blocks, configurations, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or as processor-executable instructions depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0075]

[0083] The steps of a method or algorithm described with respect to the implementations disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a random access memory (RAM), a flash memory, a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a register, a hard disk, a removable disk, a compact disk read only memory (CD-ROM), or any other form of non-transient storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside in a computing device or a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a computing device or user terminal.

[0076]

[0084] The previous description of the disclosed implementations is provided to enable those skilled in the art to make or use the disclosed implementations. Various modifications to these implementations will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other implementations without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the implementations shown herein, but is to be accorded the widest possible scope consistent with the principles and novel features as defined by the following claims. The invention as described in the claims of the original application is set forth below. [C1] 1. A device for processing an audio signal representative of an input sound, comprising: a hand detector configured to generate a first indication in response to detecting at least a portion of a hand covering at least a portion of the device; and an automatic speech recognition system configured to be enabled to process the audio signal in response to the first instruction. [C2] a screen, wherein the hand detector is configured to generate the first indication in response to detecting at least a portion of the hand covering the screen. and a microphone configured to be enabled in response to the first instruction to generate the audio signal based on the input sound. [C3] The device of C2, wherein the hand detector is configured to generate the first indication in response to detecting that the portion of the hand is at a distance of 10 centimeters to 30 centimeters from the screen. [C4] The device of C1, further comprising one or more sensors coupled to the hand detector and configured to provide sensor data to the hand detector. [C5] The device of C4, wherein the one or more sensors include a camera configured to provide image data to the hand detector. [C6] The device of C5, wherein the camera includes a low power ambient light sensor configured to generate the image data. [C7] The device of C5, wherein the hand detector includes a hand pattern detector configured to process the image data to determine whether the image data includes a hand pattern. [C8] The device of C7, wherein the one or more sensors further include an infrared sensor. [C9] The device of C8, wherein the hand detector further includes a hand temperature detector configured to process infrared sensor data from the infrared sensor. [C10] The device of C1, further comprising an enablement circuit coupled to the hand detector and configured to enable the automatic speech recognition system in response to receiving the first instruction. [C11] The device of C1, wherein the automatic speech recognition system includes a buffer and an automatic speech recognition engine, and enabling the automatic speech recognition system includes starting buffering of the audio signal in the buffer. [C12] The device of claim 11, wherein the hand detector is further configured to generate a second indication in response to detecting that the portion of the hand is no longer covering the portion of the device, the second indication corresponding to an end of speech signal that causes the automatic speech recognition engine to start processing audio data from the buffer. [C13] The device of C1, wherein the hand detector and the automatic speech recognition system are integrated into an automobile. [C14] The device of C1, wherein the hand detector and the automatic speech recognition system are integrated into a portable communication device. [C15] The device of C1, wherein the hand detector and the automatic speech recognition system are integrated into a virtual reality or augmented reality headset. [C16] 1. A method for processing an audio signal representative of an input sound, comprising the steps of: Detecting, at a device, at least a portion of a hand covering at least a portion of the device; and in response to detecting the portion of the hand covering the portion of the device, enabling an automatic speech recognition system to process the audio signal. [C17] The method of C16, wherein the portion of the device includes a screen of the device, and further comprising, in response to detecting the portion of the hand covering the screen, enabling a microphone to generate the audio signal based on the input sound. [C18] detecting that the portion of the hand is no longer covering the portion of the device; and in response to detecting that the portion of the hand is no longer covering the portion of the device, providing an end-of-utterance signal to the automatic speech recognition system. [C19] The method of C16, wherein enabling the automatic speech recognition system includes starting buffering of the audio signal. [C20] The method of C16, wherein detecting the portion of the hand covering the portion of the device includes processing the image data to determine whether the image data includes a hand pattern. [C21] The method of C20, wherein the image data is generated in a low power ambient light sensor of the device. [C22] The method of C20, wherein detecting the portion of the hand covering the portion of the device further comprises processing infrared sensor data from an infrared sensor of the device. [C23] and instructions that, when executed by one or more processors of a device, cause the one or more processors to perform operations for processing an audio signal representative of an input sound, the operations comprising: detecting at least a portion of a hand covering at least a portion of the device; and in response to detecting the portion of the hand covering the portion of the device, enabling an automatic speech recognition system to process the audio signal. [C24] The non-transitory computer-readable medium of C23, wherein the portion of the device includes a screen of the device, and the operation further comprises, in response to detecting the portion of the hand covering the screen, enabling a microphone to generate the audio signal based on the input sound. [C25] The operation further comprises: detecting that the portion of the hand is no longer covering the portion of the device; and in response to detecting that the portion of the hand is no longer covering the portion of the device, providing an end-of-utterance signal to the automatic speech recognition system. [C26] The non-transitory computer-readable medium of C23, wherein detecting the portion of the hand covering the portion of the device includes processing sensor data to detect a shape of a hand. [C27] 1. An apparatus for processing an audio signal representative of an input sound, comprising: means for detecting at least some portion of a hand covering at least some portion of the device; means for processing the audio signal, the means for processing configured to be enabled in response to detection of the portion of a hand covering the portion of the device. An apparatus comprising: [C28] a means for displaying information and the means for detecting is configured to detect the portion of the hand covering the means for displaying information. means for generating the audio signal based on the input sound, the means for generating being configured to be enabled in response to detection of the portion of the hand covering the means for displaying. The apparatus of C27, further comprising: [C29] The apparatus of C27, further comprising means for generating image data, wherein the means for detecting is configured to determine whether the image data includes a hand pattern. [C30] means for detecting a temperature associated with said portion of said hand; and The apparatus of C29, further comprising at least one of means for detecting a distance of the portion of the hand from the device.

Claims

1. 1. A device for processing an audio signal representative of an input sound, comprising: one or more sensors coupled to a hand detector and configured to provide sensor data to the hand detector; A hand detector; an automatic speech recognition system including a buffer and an automatic speech recognition engine; The hand detector includes: generating a first indication in response to detecting, via the one or more sensors, that at least a portion of a hand is positioned within 10 cm to 30 cm of the one or more sensors; generating a second indication in response to detecting that the portion of the hand is no longer located within 10 cm to 30 cm of the one or more sensors, wherein the second indication corresponds to an end-of-speech signal that causes the automatic speech recognition engine to begin processing audio data from the buffer. [0023] the automatic speech recognition system is configured to initiate buffering of the audio signal in response to the first instruction and to process the audio signal, wherein the audio signal comprises the input sound received between the generation of the first instruction and the generation of the second instruction. A device comprising:

2. a screen, wherein the hand detector is configured to generate the first indication in response to detecting at least a portion of the hand covering the screen.

10. The device of claim 1, further comprising: a microphone configured to be enabled in response to the first indication to generate the audio signal based on the input sound.

3. 3. The device of claim 2, wherein the hand detector is configured to generate the first indication in response to detecting that the portion of the hand is at a distance of between 10 centimeters and 30 centimeters from the screen.

4. 2. The device of claim 1, wherein the one or more sensors include a camera configured to provide image data to the hand detector, the camera preferably including a low power ambient light sensor configured to generate the image data.

5. The device of claim 1 , wherein the hand detector comprises a hand pattern detector configured to process the image data to determine whether the image data includes a hand pattern.

6. The device of claim 5 , wherein the one or more sensors further comprise an infrared sensor.

7. The device of claim 6 , wherein the hand detector further comprises a hand temperature detector configured to process infrared sensor data from the infrared sensor.

8. 10. The device of claim 1, further comprising an enablement circuit coupled to the hand detector and configured to enable the automatic speech recognition system in response to receiving the first indication.

9. The device of claim 1 , wherein the hand detector and the automatic speech recognition system are integrated into an automobile.

10. The device of claim 1 , wherein the hand detector and the automatic speech recognition system are integrated into either a portable communication device or a virtual reality or augmented reality headset.

11. 1. A method for processing an audio signal representative of an input sound, comprising the steps of: generating a first indication in response to detecting, via one or more sensors of the device, that at least a portion of the hand is positioned within a range of 10 cm to 30 cm from the one or more sensors; enabling an automatic speech recognition system to begin buffering the audio signal in response to the generated first instruction; generating a second indication in response to detecting at the device that the portion of the hand is no longer located within 10 cm to 30 cm of the one or more sensors, wherein the generated second indication corresponds to an end of speech signal. enabling an automatic speech recognition engine to begin processing the buffered audio signal in response to the generated second instruction; Equipped with 13. The method of claim 12, wherein the buffered audio signal comprises the input sound received between the generating of the first instruction and the generating of a second instruction.

12. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a device comprising an automatic speech recognition system including a buffer and an automatic speech recognition engine, a hand detector, and one or more sensors coupled to the hand detector and configured to provide sensor data to the hand detector, cause the one or more processors to perform the method of claim 11.

Citation Information

Patent Citations

  • Mobile terminal and menu control method therefor

    JP2009253970A

  • Speech recognition device and speech recognition method

    JP2013080015A

  • Information processing apparatus, information processing method, and information processing program

    JP2014066848A

  • Control device and portable terminal

    JP2014119295A

  • Electronic device controlled by a motion and controlling method thereof

    US20120179472A1