Context-dependent conversion mode for digital assistants

Abstract

Procedure, comprehensive: Receiving a cue to initiate a context-dependent mode associated with a context on an information handling device; Receiving at least one context input during a duration of the context-sensitive mode on the information handling device; and In response to receiving at least one context input, the duration of the context-dependent mode is extended, whereby the extended duration does not require the receipt of subsequent hints; wherein the instruction includes a wake-up action and a context command, wherein the wake-up action includes at least a wake-up word or a wake-up gesture, and where the context of the context-dependent mode is determined based on the context command.

Description

BACKGROUND

[0001] Information handling devices (“devices”), such as smartphones, tablets, laptops, smart speakers, and the like, can use voice-activated digital assistants (“digital assistants”) that are capable of processing audible commands. In response to recognizing an audible command provided by a user, the digital assistant can perform a corresponding task.

[0002] Publication US 8,924,219 B1 relates to the continuous recognition of voice commands in mobile devices. In a first speech recognition mode, a computer device listens for speech that corresponds to one of several activation phrases or "hotwords," which prompt the computer device to recognize further voice input in a second speech recognition mode. Each activation phrase is associated with a specific application. During the first speech recognition mode, the computer device compares the recognized speech with the activation phrases to identify possible matches. If a matching activation phrase is identified with a sufficiently high degree of certainty, the computer device launches the application associated with the matching activation phrase and switches to the second speech recognition mode. In the second speech recognition mode, the computer device listens for voice input related to the launched application.

[0003] Publication EP 2 950 307 A1 discloses systems and methods for selectively processing and responding to spoken user input. An audio input containing spoken user input can be received by a user device. The spoken user input can be identified by determining its start and end points. Contextual information can be used to determine whether the spoken user input was intended for a virtual assistant. If it is determined that the spoken user input was intended for the virtual assistant, it can be processed and a corresponding response generated. Conversely, if it is determined that the spoken user input was not intended for the virtual assistant, it can be ignored and / or no response can be generated.

[0004] Publication US 2013 / 0 304 479 A1 describes methods and systems for determining intent in speech and gesture interfaces. BRIEF SUMMARY

[0005] The object of the present invention is to provide an improved context-dependent mode on an information handling device.

[0006] This problem is solved by the subject matter of main claim 1 and dependent claims 9 and 16, which define the present invention.

[0007] Preferred embodiments of the present invention are the subject of the dependent claims.

[0008] In summary, one aspect provides a method according to claim 1.

[0009] Another aspect provides an information handling device according to claim 9.

[0010] Another aspect is a product according to claim 16.

[0011] The foregoing is a summary and may therefore contain simplifications, generalizations and omissions of details; consequently, those familiar with the technique will recognize that the summary is merely descriptive and does not aim to be limiting in any way.

[0012] For a better understanding of the embodiments, along with other features and advantages thereof, reference is made to the following description in conjunction with the accompanying drawings. The scope of the invention is set out in the appended claims. BRIEF DESCRIPTION OF SOME VIEWS OF THE DRAWINGS Fig. Figure 1 shows an example of an information handling device circuit. Fig. Figure 2 shows another example of an information handling device circuit. Fig. Figure 3 shows an exemplary procedure for initiating a context-dependent input mode. DETAILED DESCRIPTION

[0013] It is readily apparent that the components of the embodiments, as generally described herein and shown in the figures, can be arranged and constructed in a wide variety of different configurations, in addition to the exemplary embodiments described. Therefore, the following detailed description of exemplary embodiments, as represented in the figures, is not suitable for limiting the scope of the embodiments as claimed, but only for illustrating exemplary embodiments.

[0014] A reference in this entire specification to a “single embodiment” or “an embodiment” (or similar) means that a particular feature, structure, or distinguishing characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of the phrases “in a single embodiment” or “in an embodiment” or the like at different points in this specification does not necessarily refer to the same embodiment.

[0015] Furthermore, the described features, structures, or characteristics can be combined in any suitable way in one or more embodiments. A multitude of specific details are provided in the following description to give a thorough understanding of the embodiments. However, a person skilled in the art will recognize that the different embodiments can be carried out without one or more of the specific details, or with or without methods, components, materials, etc. In other examples, known structures, materials, or methods are not shown or described in detail to avoid confusion.

[0016] Conventionally, digital assistant software on devices (e.g., Siri® for Apple®, Cortana® for Windows®, Alexa® for Amazon®, etc.) can identify a voice command provided by a user (e.g., by comparing it to an accessible command memory), associate that command with a corresponding task, and subsequently perform actions to complete the task. For example, a user might command a digital assistant to generate a shopping list. Upon recognizing the command, the digital assistant can enter a "conversation mode" in which it waits to receive subsequent voice input (e.g., voice input relating to items to be placed on the list) and then, upon receipt, store that input in an accessible memory location.

[0017] Digital assistants can be helpful for users who want hands-free voice control of a device. Common situations where digital assistants might be used include when a user is driving, when a user is physically occupied with another task, or when a user finds that a voice command can be a faster and more convenient way to complete a task.

[0018] Conventional digital assistants, however, have several aspects related to receiving and processing voice input. One such aspect is that a "wake-up word" must be provided to the digital assistant each time a user wishes to issue a voice command. This wake-up word signals to the digital assistant that it is ready to receive and process voice input. For example, if the wake-up word was "Assistant" when a shopping list was generated, a user would have to repeat this word every time they wanted to add a new item to the list (e.g., "Assistant, add milk to the list," "Assistant, add bread to the list," etc.). The required repetition of the wake-up word can become annoying and inconvenient for the user.If, in addition, a user forgets to communicate the wake word before issuing a voice command, the digital assistant cannot recognize the command as input to be processed. For example, when generating a shopping list, the digital assistant cannot recognize that milk should be added to the list if a user only says, "Add milk."

[0019] Accordingly, one embodiment provides a method for initiating a context-dependent input mode, wherein a digital assistant is able to receive and process speech context input during the duration of the context-dependent mode. In one embodiment, the digital assistant can receive a cue (e.g., a wake-up action followed by a context command) to initiate a context-dependent mode related to a specific context (e.g., news, email, sports, etc.). The duration of the context-dependent mode can be indefinite or until interrupted (e.g., a cue to end the current mode, a cue to switch to another mode, etc.). During the mode, speech commands associated with the context can be received and processed by the digital assistant.This method allows a user to communicate a variety of commands to the digital assistant without having to repeat a wake-up word or gesture before each command.

[0020] The exemplary embodiments shown are best understood by referring to the figures. The following description is intended only as an example and simply illustrates certain exemplary embodiments.

[0021] While various other circuits, circuits, or components are used in information handling devices in relation to a smartphone and / or tablet circuit 100, one in Fig. The example shown is a system-on-a-chip design, such as that found in tablets or other mobile computing platforms. Software and processor(s) are combined on a single chip 110. Processors include internal arithmetic units, registers, cache memory, buses, on / off ports, etc., as is well known in engineering. Internal buses and the like depend on different vendors, but essentially all of these peripheral devices (120) can be attached to a single chip 110. The circuit 100 combines the processor, memory control, and an on / off control node together on a single chip 110. Also, systems 100 of this type typically do not use SATA, PCI, or LPC. Common interfaces include, for example, SDIO and I2C.

[0022] There are power management chips 130, e.g., a battery management unit (BMU), which supplies power, for example, via a rechargeable battery 140 that can be recharged by connecting to a power source (not shown). In at least one design, a single chip 110 is used to provide BIOS-like functionality and DRAM memory.

[0023] System 100 typically includes one or more WWAN transceivers 150 and WLAN transceivers 160 for connecting to various networks, such as telecommunications networks and wireless internet devices, e.g., access points. It also usually includes devices 120, e.g., an image sensor such as a camera. System 100 often includes a touchscreen 170 for data input and display / playback. System 100 also typically includes various storage devices, e.g., flash memory 180 and SDRAM 190.

[0024] Fig. Figure 2 shows a block diagram of another example of information handling device circuits, circuits, or components. The example shown in Fig. Figure 2 shows that embodiments may correspond to computer systems such as those in the THINKPAD series of personal computers sold by Lenovo (US) Inc. of Morrisville, NC, or other devices. As can be seen from the description herein, embodiments may have other features or only some features of the example shown in Figure 2. Fig. 2 is shown, include.

[0025] The example of Fig. 2 comprises a so-called chipset 210 (a group of integrated circuits or chips that work together, chipsets) with an architecture that can vary depending on the manufacturer (for example, Intel, AMD, ARM, etc.). Intel is a registered trademark of Intel Corporation in the United States and other countries. AMD is a registered trademark of Advanced Micro Devices Inc. in the United States and other countries. ARM is an unregistered trademark of ARM Holdings plc in the United States and other countries. The chipset 210 architecture includes a core and memory control group 220 and an ON / OFF control node 250, which exchanges information (for example, data, signals, instructions, etc.) via a Direct Management Interface (DMI) 242 or a link control device 244. In Fig. 2. The DMI 242 is a chip-to-chip interface (sometimes also referred to as a connection between a "northbridge" and a "southbridge"). The core and memory control group 220 comprise one or more processors 222 (for example, single-core or multi-core) and a memory control node 226, which exchange information via a front-side bus (FSB) 224. It should be noted that the components of the group 220 can be integrated on a single chip, replacing the conventional "northbridge"-like structure. One or more processors 222 include internal arithmetic units, registers, cache memory, buses, ON / OFF ports, etc., as are well known in engineering.

[0026] In Fig. 2 The memory control node 226 forms interfaces with the memory 240 (for example, to provide support for a type of RAM memory that can be referred to as "system memory" or "memory"). The memory control node 226 also includes a low voltage differential signaling (LVDS) interface 232 for a display device 292 (for example, a CRT, a flat panel display, a touchscreen, etc.). A block 238 includes some technologies that can be supported via the LVDS interface 232 (for example, serial digital video, HDMI / DVI, display connector). The memory control node 226 also includes a PCI Express (PCI-E) interface 234, which can support discrete graphics 236.

[0027] In Fig. 2. The ON / OFF control node 250 includes a SATA interface 251 (for example, for HDDs, SSDs, etc. 280), a PCI-E interface 252 (for example, for wireless connections 282), a USB interface 253 (for example, for devices 284 such as a digitizer, keyboard, mouse, cameras, telephones, microphones, storage, other connected devices, etc.), a network interface 254 (for example, LAN), a GPIO interface 255, an LPC interface 270 (for ASICs 271, a TPM 272, a super ON / OFF control node 273, a firmware node 274, BIOS support 275, and various types of memory 276 such as ROM 277, Flash 278, and NVRAM 279), a power management interface 261, and a clock generator interface. 262, an audio interface 263 (for example for loudspeakers 294), a TCO interface 264, a system management bus interface 265 and an SPI flash 266, which can contain a BIOS 268 and a boot code 290.The ON / OFF control node 250 can feature Gigabit Ethernet support.

[0028] Once the system is powered on, it can be configured to execute a boot code 290 for the BIOS interface 268, as stored in the SPI flash 266, and can then process data under the control of one or more operating systems and application software (stored, for example, in system memory 240). An operating system can be stored in any number of different locations and accessed, for example, according to the instructions in the BIOS 268. As described herein, a device can have fewer or more features than those specified in the system. Fig. 2 are shown, included.

[0029] An information handling device circuit, such as those found in the Fig. 1 and Fig. The circuit shown in Figure 2 can be used in devices such as tablets, smartphones, smart speakers, personal computers in general, and / or electronic devices that allow users to communicate with a digital assistant. For example, the circuit shown in Figure 2 can be used in a computer for ... Fig. 1 is shown in a tablet or smartphone version, whereas the circuit shown in Fig. 2 is shown, and can be executed in a personal computer version.

[0030] Now, referring to Fig.3. An embodiment can initiate a context-dependent input mode while receiving context-dependent input and performing tasks associated with that input. In 301, an embodiment can receive a cue to initiate a context-dependent mode. The cue can be, for example, a wake-up action communicated by a user, such as a wake-up word or gesture (e.g., waving a hand), followed by a context command. The context command can trigger the digital assistant to initiate a context-dependent mode in which subsequent user input can relate to the context or subject of the context command. For example, if an embodiment receives a wake-up function followed by the context command "Read Emails," it can initiate a context-dependent mode related to the email browser.

[0031] In one embodiment, the context of a context command can be identified by comparing the context command with an accessible memory bank of recognizable contexts. In another embodiment, the context can be determined by a user's current activity. In one embodiment, a digital assistant can be connected to another device (e.g., television, radio, etc.) or synchronized with another application (e.g., video playback application, music playback application, etc.) and can determine the context of the active application or device. For example, if a user is watching television on a smart TV, the digital assistant can identify that a television-watching mode was active and accept a context input related to television (e.g., "List sports channels!").

[0032] In version 302, an embodiment can receive context input associated with the context of the context-aware mode. In one embodiment, the user does not need to repeat the wake-up action before communicating the context input during the duration of the context-aware mode. For example, for a context-aware mode related to email browsing, a user can provide email-related commands such as "Read this email for me," "Archive this email," "Delete this message," etc. In another example, for a context-aware mode related to creating a shopping list, a user can verbally list items to be added to the list, such as "Add milk," "Add bread," "Add cheese," etc. In yet another example, for a context-aware mode associated with music, a user can verbally list the songs or artists they wish to hear.

[0033] In one embodiment, eye-tracking software can be used to determine whether a user intends to transmit an input to the digital assistant. For example, if an embodiment determines (e.g., based on eye tracking) that a user is looking at the digital assistant or at a specific device with which a digital assistant has been associated during the transmission of a voice input, the digital assistant can determine that the voice communication should be perceived as a contextual input.

[0034] In one embodiment, context input can be received from multiple users. For example, during a shopping list context-dependent mode, the digital assistant can receive a command from one user to add milk to the list and another command from a different user to add bread. In another embodiment, multiple context-dependent modes can be active simultaneously, each relating to a separate context. For example, during the duration of a context-dependent mode associated with generating a shopping list, an embodiment can also receive context input for a context-dependent mode associated with browsing email. Each context-dependent mode can be initiated, for example, by a separate wake-up action associated with the mode's context.In one embodiment, multiple context-dependent modes can be activated simultaneously, with each context-dependent mode capable of receiving context input from a specific user. For example, two users can initiate separate context-dependent modes and provide context input to such modes, with the input being differentiated based on user identity. One embodiment can differentiate between inputs based, for example, on user speech recognition.

[0035] In one embodiment, a digital assistant can receive context-sensitive input until an interruption occurs in the context-sensitive mode. In one embodiment, the interruption can be a command to end the context-sensitive mode. For example, during a shopping list context-sensitive mode, a user can provide the voice command "End of shopping list," which signals the digital assistant to end the mode. In another embodiment, the interruption can be a command to start a different context-sensitive mode, temporarily pausing the initial context-sensitive mode and preventing the digital assistant from recognizing input associated with the initial mode while the other mode is active.For example, a user might initiate an email browsing context-sensitive mode while in a shopping list context-sensitive mode, during which contextual input associated with the shopping list (e.g., Add milk, Add bread, etc.) is ignored. In one embodiment, a notification can be provided to the user that the context-sensitive mode has ended.

[0036] In response to the identification of the received context input at 303, an embodiment at 304 can perform an action associated with the input. In one embodiment, a digital assistant can access instructions that specify the type of action to be performed in response to a particular input. For example, in a shopping list context-sensitive mode, in response to receiving the input "Add milk" or "Milk," an embodiment can add milk to the list of items on the shopping list. In one embodiment, instructions can be stored locally on the device or in an accessible remote storage location (e.g., the cloud).

[0037] In 305, an embodiment can disregard the context input in response to the fact that the received context input is not identified. An embodiment might fail to identify received context input for a variety of reasons. For example, the received input was not associated with the current context-dependent mode (e.g., an embodiment might receive a command associated with email browsing during a shopping list context-dependent mode). In another example, a user might generate unidentifiable speech output during a context-dependent mode that is not intended to be executed by the digital assistant (e.g., the user was speaking to another person). In 306, an embodiment can provide the user with a notification that the received input was not identified.In one embodiment, the notification can be audible, where a device can verbally inform a user that their input was not recognized. In another embodiment, the notification can be visual. For example, a text-based message can appear on a device's screen, notifying the user that their input was not recognized.

[0038] The various embodiments described herein thus represent a technical improvement for conventional communication with a digital assistant. Using the methods described herein, a user may be able to enter a context-dependent "conversation mode" with a digital assistant, which facilitates communication with the digital assistant. One embodiment initiates a context-dependent mode in which a user can transmit contextual input to the digital assistant without having to repeat the wake-up action before each transmission.

[0039] As is evident to a person skilled in the art, different aspects can be contained within a system, a process, or a device program product. Accordingly, aspects can take the form of an entire hardware implementation or an embodiment that includes software, all of which are referred to herein generally as a "circuit," "module," or "system." Furthermore, aspects can take the form of a device program product contained in one or more device-readable media that embody device-readable program code.

[0040] It should be noted that various functions described herein can be performed using instructions stored on a device-readable storage medium, such as a non-signal storage device, which are executed by the processor. A storage device may be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Larger, more specific examples of a storage medium include the following: a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.In the context of this document, a storage device is not a signal and "non-transitory" encompasses all media except signal media.

[0041] Program code that is present on your storage medium can be transferred using a suitable medium, which includes but is not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

[0042] Program code can execute operations written in any combination of one or more programming languages. The program code can run entirely on a single device, partially on a single device, as a standalone software package, partially on a single device and partially on another device, or entirely on another device. In some cases, the devices may be connected by some kind of connection or network, including a local area network (LAN) or a wide area network (WAN), or the connection may be established by other devices (for example, via the internet using an internet service provider), by wireless connection (e.g., near-field communication), or by a wired connection (e.g., via USB).

[0043] This document describes exemplary embodiments with reference to the figures, which represent, for example, methods, devices, and program products according to different exemplary embodiments. It is understood that the actions and functionality can be executed, at least in part, by program instructions. These program instructions can be provided to a processor of a device, a special-purpose information processing device, or other programmable data processing devices to create a machine such that the instructions, executed by a processor of the device, perform the specified functions / actions of the device.

[0044] It is important to note that while certain blocks are used in the figures and a specific arrangement of blocks is shown, these are not limiting examples. In certain contexts, two or more blocks may be combined, a block may be divided into two or more blocks, or certain blocks may be rearranged or appropriately reorganized, as the examples explicitly shown are used for descriptive purposes only and are not to be interpreted as limiting.

[0045] As used herein, the singular “ein” and “eine” can be interpreted as including a plural such as “one or more”, unless expressly stated otherwise.

[0046] This disclosure has been presented for illustrative and descriptive purposes; however, it is not intended to be exhaustive or limiting. Many modifications and variations will be obvious to the person skilled in the art. The exemplary embodiments have been selected to describe the principles and practical applications and to enable others with expert knowledge of the prior art to understand the disclosure of the different embodiments with different modifications, as suitable for the particular use under consideration.

[0047] It is therefore understandable that this description, although exemplary embodiments shown herein are described with reference to the accompanying drawings, is non-limiting and that various other changes and modifications to it can be made by a person skilled in the art without deviating from the scope of protection and the spirit of the disclosure.

Claims

[1] Method comprising: Receiving, on an information handling device, an indication to initiate a context-dependent mode associated with a context; Receiving at least one context input during a duration of the context-dependent mode on the information handling device; and in response to receiving at least one contextual input, extending the duration of the context-dependent mode, wherein the extended duration does not require receipt of subsequent cues; wherein the indication comprises a wake-up action and a context command, wherein the wake-up action comprises at least one wake-up word or one wake-up gesture, and where the context of the context-dependent mode is determined based on the context command. [2] The method of claim 1, wherein at least one context input is associated with the context. [3] The method of claim 1, wherein the duration is continuous until an interruption is detected. [4] The method of claim 3, wherein the interrupt is at least one interrupt selected from the group consisting of a command to end the context-dependent mode and a command to begin another context-dependent mode. [5] The method of claim 1, wherein receiving further comprises receiving at least one context input from a plurality of users. [6] The method of claim 1, further comprising determining an intent to transmit at least one contextual input based on eye tracking. [7] The method of claim 1, further comprising providing a notification in response to at least one context input. [8] The method of claim 1, further comprising performing an action associated with the context input using a processor. [9] Information handling device comprising: a processor; a storage device that stores instructions executable by the processor to: Receiving a cue to initiate a context-dependent mode associated with a context; Receiving at least one context input during a duration of the context-dependent mode; and in response to receiving at least one contextual input, extending the duration of the context-dependent mode, wherein the extended duration does not require receipt of subsequent cues; wherein the indication comprises a wake-up action and a context command, wherein the wake-up action comprises at least one wake-up word or one wake-up gesture, and where the context of the context-dependent mode is determined based on the context command. [10] The information handling device of claim 9, wherein at least one context input is associated with the context. [11] The information handling device of claim 9, wherein the duration is continuous until an interruption is detected. [12] The information handling device of claim 11, wherein the interrupt is at least one interrupt selected from the group consisting of a command to end the context-dependent mode and a command to begin another context-dependent mode. [13] The information handling device of claim 9, wherein at least one context input is received from multiple users. [14] The information handling device of claim 9, wherein the instructions are executable by the processor to determine an intent to transmit at least one context input based on eye tracking. [15] The information handling device of claim 9, wherein the instructions are executable by the processor to perform an action associated with the context input. [16] Product comprising: a storage device storing code, the code being executable by a processor to perform a method according to any one of claims 1 to 8.

Images