Method and device for enabling context-based voice responses to always-on display notifications

Abstract

A device comprising: a display configured to operate at least in a full-power mode or at least in an always-on display mode (“AoD” mode); a processor; memory storing code that can be executed by the processor to: analyze a notification received while the display is in AoD mode for the purpose of determining the notification type; determine whether the notification supports a voice response; in response to a determination that the notification supports a voice response, receive a voice response to the notification from a user and authenticate the user based on the voice response to the notification; and in response to user authorization, bypass a screen lock and execute a command based on the voice response to the notification.

Description

AREA OF INVENTION

[0001] The present invention relates to the interaction of a user with an electronic device. BACKGROUND DESCRIPTION OF THE STATE OF THE TECHNOLOGY

[0002] Electronic devices can provide various notifications for different events that a user may be interested in. Furthermore, some electronic devices are capable of displaying notifications in a high-performance or high-power mode (which utilizes, for example, all of the device's resources and screen capacity) and in a low-power mode (in which the device's application processor is shut down and a partial display technology is used for operation). Typically, the electronic device prompts the user with a notification in low-power mode, and the notification continues (e.g., via a flashing LED) until the user closes the notification or logs out (e.g., by entering a PIN code, fingerprint scan, password, pattern code, etc.).The device is authenticated to switch to high-performance mode and respond to notifications. The constant barrage of notifications and the need to authenticate are a waste of time and noticeably burdensome for the user. It would be beneficial to provide a better method for responding to these notifications while in power-saving mode and to develop improved electronic devices suitable for performing such procedures. OVERVIEW

[0003] A device for managing context-based voice responses in an always-on display (“AoD” environment) is proposed. A method and a software product perform the functions of the device.

[0004] The device comprises a display configured to operate at least in a full-power mode or at least in an always-on display mode (“AoD” mode), a processor, and memory that stores code executable by the processor while the display is operating in AoD mode. The executable code instructs the processor to parse a notification to determine its type and whether it supports voice responses. The executable code also instructs the processor to receive a voice command from a user, authorize the user based on the voice command, bypass a screen lock in response to the user authorization, and execute the voice command.

[0005] In one embodiment, the executable code also causes the processor to perform actions, including displaying a notification to the user on the screen and activating a voice input that receives a voice command from the user after the user acknowledges the notification. These actions may also include delaying the voice input from the moment the user acknowledges the notification. The delay may be based on a text amount corresponding to the notification.

[0006] In another embodiment, the actions can also include presenting voice command suggestions in response to the identified notification type, executing the voice command in full-power mode, and returning the display to AoD mode after executing the voice command.

[0007] A method for managing context-based voice responses in an AoD environment may include: displaying notifications at least in a full-performance mode or at least in an always-on display mode (“AoD” mode), analyzing a notification to determine its type, and, during AoD mode, determining whether the notification supports voice responses. The method also includes receiving a voice command from a user, authorizing the user based on the voice command, bypassing a screen lock in response to the user authorization, and executing the voice command.

[0008] In one embodiment, the method also includes activating a voice input that receives the voice command from the user after the user has acknowledged the notification, and enabling voice input after a delay from the time of the user's notification acknowledgment. The delay can be determined according to the text length of the associated notification.

[0009] In another embodiment, the method includes presenting voice command suggestions in response to a detected notification type, executing the voice command in full-power mode, and returning the display to AoD mode after executing the voice command.

[0010] A program product for managing context-based notifications in an AoD mode is also proposed. The program product code includes a computer-readable storage medium that stores instructions which, when executed by a processor of a computer device, cause the latter to perform the actions of the procedure. List of characters

[0011] A detailed description of the embodiments presented in the preceding overview follows, based on specific embodiments shown in the accompanying drawings. It is understood that these drawings show only some embodiments and therefore do not constitute a limitation of the invention. The drawings show: Fig. 1 a perspective view of an example electronic device according to the embodiments of the present invention; Fig. 2. A schematic block diagram showing example components (e.g., internal components) of the electronic device. Fig. 1 represents; Fig. 3, Fig. 4 and Fig. 5 Block diagrams illustrating embodiments of the display of the electronic device operating in AoD mode according to embodiments of the present invention; Fig. 6 a schematic block diagram illustrating an embodiment of a voice response module according to the embodiments of the present invention; and Fig. 7 A schematic flowchart illustrating an embodiment of a method for enabling voice command responses to notifications in an AoD environment according to embodiments of the present invention. DETAILED DESCRIPTION

[0012] Those skilled in the art will recognize that aspects of the present invention can be implemented as a system, a method, or a program product. Accordingly, the embodiments can be provided entirely as hardware, entirely as software (including firmware, resident software, microcode, etc.), or as a combination of hardware and software aspects, hereafter generally referred to as a "circuit," "module," or "system." Furthermore, embodiments can be provided as a program product implemented on one or more computer-readable storage media that store machine-readable code, computer-readable code, and / or program code, hereinafter referred to as "code." The storage devices can be physical, non-transitory, and / or non-transferable storage. The storage devices may not contain any signals.In a specific embodiment, the storage devices can use signals only for code access.

[0013] Many of the functional units described here are designated as modules, particularly to emphasize their implementation independence. For example, a module can be implemented as a hardware circuit, comprising specialized VLSI circuits or gate arrays, commercially available semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in programmable hardware devices, such as field-programmable gate arrays, programmable array logic, programmable logic devices, or the like.

[0014] Modules can also be implemented as code and / or software for execution by various types of processors. A detected code module can, for example, comprise one or more physical or logical blocks of executable code, which may be organized as an object, procedure, or function. However, the executable files of a detected module do not need to be physically located together; they can comprise distributed instructions stored in different locations. When logically linked, these instructions contain the module and fulfill its stated purpose.

[0015] A code module can be a single instruction or many instructions and can be distributed across multiple code segments between different programs and on multiple storage devices. Similarly, operational data can be identified and represented within modules and can be executed in any suitable form and organized within any suitable type of data structure. The operational data can be captured as a single record or distributed across various locations, including different computer-readable storage devices. If a module or sections thereof are executed in software, the software sections are stored on one or more computer-readable storage devices.

[0016] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable storage medium. The computer-readable storage medium can be a storage device that stores the code. The storage device can be, for example, an electronic, magnetic, optical, electromagnetic, infrared, holographic, micromechanical, or semiconductor system storage device, apparatus, or device, or a combination thereof, without limitation.

[0017] More specific examples (a non-exhaustive list) of storage devices include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or a suitable combination of the foregoing storage devices. In the context of this document, a computer-readable storage medium can be any physical medium capable of containing or storing a program for use by, or in conjunction with, a system, device, or apparatus for executing instructions.

[0018] The code for performing operations for implementations can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Python, Ruby, Java, Smalltalk, C++, or similar languages, and common procedural programming languages ​​such as C, and / or machine language such as assembly languages. The code can run entirely on the user's computer, partially on the user's computer, be a standalone software package, run partly on the user's computer and partly on a remote computer, or run entirely on the remote computer or on a server.In the latter case, the remote computer can be connected to the user's computer via any network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection can be to an external computer (for example, via the Internet by using an Internet service provider).

[0019] When this description refers to “an embodiment” (singular or indefinite article) or a similar phrase, this means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The phrases “in an embodiment” in the singular or with an indefinite article, or similar phrases in this description, may refer to the same embodiment, but need not; they mean “one or more, but not all, embodiments,” unless expressly stated otherwise. The terms “contain,” “comprise,” “have,” and their variants mean “contain, but not be limited to,” unless expressly stated otherwise.A numbered list of items does not imply that some or all of these items are mutually exclusive unless explicitly stated otherwise. The indefinite article "a" and the definite article "the" also refer to "one or more" unless explicitly stated otherwise.

[0020] Furthermore, the described features, structures, or characteristics of the embodiments can be combined in any suitable manner. The following description provides numerous specific details, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc., to illustrate the embodiments with sufficient clarity. However, those skilled in the art in the relevant field will recognize that embodiments can be implemented practically without one or more of the specific details or with other procedures, components, materials, etc. Conversely, sufficiently well-known structures, materials, or components are not shown or described in detail in order to avoid unnecessarily obscuring aspects of an embodiment.

[0021] Aspects of the embodiments are explained below with reference to the schematic flowcharts and / or schematic block diagrams of methods, devices, systems, and program products according to the embodiments. It is understood that each block of the schematic flowcharts and / or schematic block diagrams, and combinations of blocks in the schematic flowcharts and / or schematic block diagrams, may be implemented by code. This code may be provided to a processor of a general-purpose computer, a specialized computer, or another programmable data processing device to create a machine such that the instructions executed by the processor of the computer or other programmable data processing device provide the means for the practical implementation of the functions / operations in the schematic flowcharts and / or schematic block diagrams or blocks.

[0022] The code may also be stored in a storage device that can instruct a computer, other programmable data processing devices, or other equipment to operate in a particular manner, such that the instructions stored in the storage device produce a manufactured item with instructions or commands that practically execute the function / operation specified in the flowcharts and / or schematic block diagrams or blocks.

[0023] The code can also be loaded onto a computer, other programmable data processing device, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing device, or other equipment to create a computer-implemented procedure such that the code executed on the computer or other programmable data processing device provides procedures for performing the functions / operations specified in the flowchart and / or block diagram or in the blocks.

[0024] The schematic flowcharts and / or schematic block diagrams in the figures represent the architecture, functionality, and operation of possible implementations of devices, systems, processes, and program products according to the various embodiments. In this respect, each block in the schematic flowcharts and / or schematic block diagrams can represent a module, segment, or section of code containing one or more executable instructions from the code to perform the specified logical function(s).

[0025] It should also be noted that in some alternative implementations, the functions specified in the block may occur out of the order shown in the figures. For example, two blocks shown sequentially may be executed essentially simultaneously, or blocks may occasionally be executed in reverse order, depending on the functions involved. Further or different steps and procedures may be considered that are equivalent in function, logic, or effect to one or more blocks or sections thereof shown in the figures.

[0026] Although different types of arrows and lines may be used in the flowcharts and / or block diagrams, this should not be interpreted as limiting the scope of the corresponding embodiments. Some arrows or other connecting elements may be used simply to indicate the logical flow of the depicted embodiment. For example, an arrow may represent a waiting or monitoring period of unspecified duration between numbered steps of the depicted embodiment. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks within the block diagrams and / or flowcharts, may be implemented by specialized hardware-based systems that perform the functions or operations mentioned, or by combinations of specialized hardware and code.

[0027] The description of elements in each figure may refer to elements in subsequent figures. In all figures, the same reference numerals refer to the same elements, including alternative embodiments of the same or similar elements.

[0028] Fig. Figure 1 is a perspective view of an example electronic device 100 according to an embodiment of the present invention. In the present embodiment, the electronic device can 100 A device of any type suitable for providing interactive touchscreen functionality. Examples of electronic devices 100These include, but are not limited to: mobile devices, wireless devices, smartphones, tablet computers, personal digital assistants, personal navigation devices, touchscreen input devices, touch- or pen-based input devices, portable video or audio players, and any variety of other electronic devices. It is understood that the electronic device 100 It can be provided in a variety of form factors, for example as a rod, as a tablet, for unfolding / folding / collapsing, for pushing and for rotating, without limitation thereto.

[0029] In the present example implementation form, which is in Fig. As shown in 1, the electronic device 100 a case 101 with a front surface 103 , which has a touchscreen display 105It contains a user interface, which, together with certain other features described below, forms a user interface. In the present example, the touchscreen display is... 105 a touchscreen with a touch-sensitive surface that overlays a display area that is part of the front surface 103 (or is positioned directly below or inside the front surface). In one embodiment, the touchscreen display applies 105 (and in particular its display area) the technology of organic light-emitting diodes (OLED technology). The user interface of the depicted electronic device 100 It can also include one or more input keys 107 Examples of the Enter key or Enter keys 107These include, but are not limited to, the following: keys of an alpha or numeric (e.g., alphanumeric) keypad (or other keyboard), physical buttons, touch-sensitive surfaces, mechanical surfaces, multi-point directional keys, and side buttons or keys. Furthermore, the electronic device may 100 along its front surface (or on other external surfaces of the device) also a loudspeaker 109 and a microphone 111 for audio output and input. Fig. Figure 1 shows an example display and user interface features, but it is understood that the electronic device 100 Depending on the embodiment, it may have a variety of other combinations of display and user interface features.

[0030] As in Fig. As shown in 1, the electronic device contains 100 one or more sensors 113, a number of which are located on or within an outer boundary of the enclosure 101 is positioned (and on or in the case) 101 may be supported). As in Fig. As shown in 1, the sensor or sensors can 113 in the present embodiment, in particular on the front surface 103 , on another surface (for example, on one or more side surfaces) 115 ) the outer boundary of the housing 101 or be positioned on both. In the present embodiment, at least some of the sensors 113 (whether at the outer boundary or within the outer boundary, i.e., inside the enclosure) configured to detect one or more predefined environmental conditions associated with an external or internal environment of the enclosure. Further examples of sensors 113 are below with reference to Fig. 2 described.

[0031] Fig. Figure 2 is a schematic block diagram showing example components (e.g., internal components) 200 of the electronic device. 100 from Fig. 1 represents. In the illustrated embodiment, the components comprise 200 one or more wireless transceivers 201 , one or more processors 203 , one or more storage devices 205 , one or more output components 207 and one or more input components 209 As stated above, the electronic device 100 a user interface with a touchscreen display 105 , one or more of the output components 207 and has one or more of the input components 209. Furthermore, the electronic device has 100 Several of the sensors 113, some of which are described in more detail below. In one embodiment, the sensors 113in conjunction with a Sensor Hub 224 (to supply sensor signals to the hub or to receive them from the hub).

[0032] In one embodiment, the components comprise 200 a device interface 215 for providing a direct connection to auxiliary components or accessories for additional or extended functions. The internal components 200 They also have a power source or power supply 217 such as a portable battery that supplies power to the other internal components and enables the electronic device to 100 can be carried along. As shown, all components are 200 and especially the wireless transceivers 201 , the processors 203 , the storage 205 , the output components 207 , the input components 209 , the sensor hub 224 , the device interface 215 and the power supply217 through one or more internal communication links 218 (e.g. an internal communication bus) connected directly or indirectly.

[0033] In another embodiment, the wireless transmitters and receivers comprise 201 a cellular transceiver 211 and a Wi-Fi transceiver 213 In alternative embodiments, the transmitter-receivers are said to comprise numerous embodiments in which any number (e.g., more than two) of wireless transmitter-receivers are present, utilizing any number (e.g., two or more) of communication technologies.

[0034] An exemplary operation of the wireless transceivers 201 in conjunction with other components 200 of the electronic device 100This can take many forms, including, for example, the internal components detecting the communication signals upon receiving wireless signals (which are provided, for example, by one or more remote devices) and the transmitter / receiver 201 The communication signals are demodulated to restore incoming information such as voice or data transmitted via wireless signals. After receiving the incoming information from the transmitters / receivers... 201 format the processors 203 the incoming information for one or more output components 207 .

[0035] The processors format similarly. 203 Information transmitted via wireless signals through the input components 209They can be activated, but do not have to be, and transmit the outgoing information to one or more of the wireless transmitters / receivers. 201 for modulation, to provide modulated communication signals to be transmitted. The wireless transceiver(s). 201 It forwards the modulated communication signals via wireless (and possibly also wired) communication links to other devices (e.g., remote devices). The wireless transceivers 201 In one embodiment, this allows the electronic device 100 exchanges messages with remote devices, for example with a remote network unit 214 of a cellular network or a WLAN network. Examples of remote network units 214These include, among others, an application server, a network server, a database server, a content server, an SMS server, or other network units that are connected via one or more intermediate devices or networks (e.g., via a WLAN access point, the internet, an LTE network, or another network) through the wireless transmitters / receivers. 201 are accessible either directly or indirectly.

[0036] Depending on the design, the output and input components can 207 , 209 the components 200 a variety of visual, audio, or mechanical outputs. For example, the output device(s) may be... 207The input device(s) may include one or more visual output devices such as a cathode ray tube, liquid crystal display, plasma display, video screen, incandescent light, fluorescent light, front- or rear-projection display, and LED display; one or more audio output devices such as a loudspeaker, alarm, or buzzer; and one or more mechanical output devices such as a vibration mechanism or a motion-based mechanism. Similarly, the input device(s) may include one or more visual output devices such as a cathode ray tube, liquid crystal display, plasma display, video screen, incandescent light, fluorescent light, front- or rear-projection display, and LED display; one or more audio output devices such as a loudspeaker, alarm, or buzzer; and one or more mechanical output devices such as a vibration mechanism or a motion-based mechanism. 209 for example, one or more input devices such as an optical sensor (e.g., a camera lens or a photosensor), one or more audio input devices such as a microphone, and one or more mechanical input devices such as a flip sensor, a keypad, a keyboard, a selection button, a navigation cluster, a touchpad, a capacitive sensor, a motion sensor, and a switch.

[0037] As already mentioned, the user interface and especially the touchscreen display can 105 of the electronic device 100 of Fig. 1. They are considered to be one or more of the input components. 209 , in particular one in Fig. 2 shown touch-sensitive input components 219 form or contain, as well as one or more of the output components 207 form or include. Furthermore, it should be recognized that the measures by which one or more of the input devices 209 can be activated, not just by physically pressing / activating the touchscreen display. 105or the keys or other operating elements of the user interface or other things, but also, for example, opening the electronic device 100 (if it has an open and a closed position), unlocking the electronic device 100 , including moving the electronic device to initiate movement, moving the electronic device to activate a location positioning system, and operating the electronic device.

[0038] In one embodiment, one or more input components can be used. 209 such as one or more input components encompassed by the user interface, like those in Fig. 1. Touch-sensitive component shown 219 For example, it can generate an input signal in response to the recognition of a predefined gesture. The touch-sensitive component can be used for this purpose. 219It can be considered a gesture sensor and may, for example, be a touch-sensitive sensor or include one with a touch-sensitive surface parallel to the display. The touch-sensitive sensor may include at least a capacitive touch sensor, resistive touch sensor, acoustic sensor, ultrasonic sensor, proximity sensor, or optical sensor.

[0039] In one embodiment, the components can 200 also one or more sensors 113 comprise one or more species. Although the sensors 113 for the purposes of Fig. 2 separate from the input devices 209 As depicted, the sensors can be considered to be separated from the input devices. 209are included. In alternative embodiments, one or more of the input devices can be included by the sensors. One or more of the sensors can be considered to be separate from the input devices. One or more of the input devices can be considered to be separate from the sensors, or all sensors can be considered to be separate from all input devices, and vice versa.

[0040] What the sensors 113 As regards, these include in particular various sensors 225 up to and including 231, which are examples of sensors that are in the electronic device 100 are included or used by it. As already explained and as in Fig. As shown in 2, the various sensors can 225 - 231 in the present embodiment by the sensor hub 224be controlled, which reacts to the processor(s) 203 can work or independently of this (these). The various sensors 225 up to and including 231, one or more power sensors can be used. 225 , one or more temperature sensors 227 , one or more pressure sensors 228 , one or more humidity sensors 229 and one or more ambient noise sensors 231 include, without limitation thereto.

[0041] Although these different sensors 225 up to and including 231 in Fig. As shown in 2, the sensors can 113In other embodiments, the device may include one or more of numerous other sensor types, for example, one or more motion sensors, including, for example, one or more accelerometers or gyro sensors (not shown), one or more light sensors, one or more proximity sensors (e.g., a light detector, an ultrasonic transceiver, or an infrared transceiver), one or more touch sensors, one or more height sensors, one or more localization circuits / components, which may include, for example, a global positioning system (GPS) receiver, a triangulation receiver, an accelerometer, a tilt sensor, a gyroscope, or another information acquisition device that provides a current location or user device interface (wearing position) of the electronic device. 100 can recognize.

[0042] Regarding the processor(s) 203This may include one or more processing or control devices, such as a microprocessor, a microcomputer, an application-specific integrated circuit, etc. The 203 processors can generate instructions, for example, based on information from one or more input components. 209 be received. The processor(s) 203 The received information can be used alone or in combination with other data, such as in the storage devices. 205 process stored information. In this way, the memory of the components can 205 200 from the processors 203 They can be used to store and retrieve data.

[0043] Furthermore, the memory (or memory sections) 205 of the components can 200One or more storage devices can be of various forms (e.g., read-only storage, random access storage, static random access storage, dynamic random access storage, etc.) and can be accessed by the processors. 203 They are used for storing and retrieving data. In some embodiments, one or more of the storage devices can be used for this purpose. 205 with one or more of the processors 203 be combined into a single device (e.g., a processing device with a memory or a processor-in-memory (PIM)), although such a single device typically still has different areas / sections that perform the various processing and storage functions and that can be considered separate storage devices. The storage 205The stored data may include operating system, application and information data, but is not limited to these.

[0044] Every operating system contains executable code that performs basic functions of the electrical device. 100 controls, such as the interaction between the different components of the components 200 , communication with external devices or networks via the wireless transceivers 201 or the device interface 215 and the saving and retrieval of applications, modules, and data to and from memory. 205 Each application (or module) contains executable code that uses an operating system to provide more specific functions, such as file system services and the handling of protected and unprotected data stored in memory. 205are stored. Such operating system or application information may contain information for a software update (which is to be understood as potentially including updates from one or more operating systems or one or more applications, or both). As for the information data, this is typically non-executable code or information that an operating system or application can access or handle to perform functions of the electronic device. 100 to execute.

[0045] In one embodiment, the electronic device 100 configured in such a way that it can switch between different operating modes or states. These are mainly a "fully active" or an "on" operating mode, in which the processors 203There are three operating modes: an "off" mode and an "intermediate" mode in which "breathing" and "peak" processes take place or can take place. Although the processors 203 In the present embodiment, even though the electronic device 100 is switched off or shut down during the off mode, an intermediate operating mode is still possible (in contrast to a fully active operating mode in which the processors are switched off or shut down). 203 are powered on and running), so that the electronic device operates in off mode or in intermediate mode or in both modes in such a way that the touchscreen display 105Several display functions (including the potential output of information and the potential reception of input) are possible. Such operation of the 105 touchscreen display in intermediate mode or in off mode is achieved due to the control functions provided by the sensor hub. 224 be provided.

[0046] Furthermore, it should be noted that operation in fully active mode is generally associated with a normal performance mode of the 203 processors (application processors) or with a substantially active mode of such processors. The intermediate operating mode and the off operating mode, on the other hand, are associated with a power-saving operating mode (or even a completely shut-down mode) of the processors. 203(application processors) or an essentially inactive mode of such processors. The intermediate operating mode can also be referred to as "Always-on Display" mode (AoD mode) (permanently switched on mode at reduced brightness), "Doze" mode (power-saving mode), or "Sleep" mode (standby mode) (or the off mode can generally also be encompassed by "Sleep" mode), or as an energy-saving operating mode as opposed to the fully active mode, which can be referred to as "Non-Sleep" mode or a higher-performance operating mode.

[0047] As will be explained in detail below, one or more notifications can be sent from the electronic device, at least in some of the embodiments described here. 100Notifications may be displayed in a "non-sleep" mode, such as fully active mode, and one or more notifications may also be displayed by the electronic device in a "sleep" mode, such as intermediate or AoD mode (including the described submodes thereof). At least in some such embodiments, the notifications provided in fully active mode or in "non-sleep" mode are those received during operation in that mode and are displayed by activating all or substantially all of the display screen (e.g., full-screen information is displayed, with the full-screen information containing data that substantially corresponds to all pixels suitable for display by the screen).In contrast, in such embodiments, the notifications provided in intermediate mode or AoD mode are notifications received during operation in that mode and are displayed by activating only a part or parts of the display screen (in particular, a part or parts that are substantially smaller than the full screen or that display substantially less than the full screen information, i.e., information that corresponds to less or substantially less than all the pixels that can be displayed by the display).

[0048] The electronic device 100 It is generally suitable for receiving input while in intermediate or AoD mode. During operation in this mode, the electronic device displays 100As explained above, notifications are displayed to a user using significantly fewer pixels than the entire display. To exit Always-on Display (AoD) mode and respond to a notification, a user typically needs to enter authentication data to bypass a screen lock. This authentication data can include, but is not limited to, an unlock pattern, PIN, password, or fingerprint.

[0049] In a useful manner, the present description includes embodiments that allow a user to use their voice to respond to a message without using the usual authentication data. In one embodiment, the components comprise 200 of the electronic device 100 a voice response module 233 The voice response module 233In short, it is configured to enable context-based voice responses to AoD notifications without requiring prior user authentication. A detailed description of the voice response module is available. 233 The following is done with reference to Fig. 3. The voice response module 233 It can be a separate hardware circuit or, in an alternative embodiment, an application that runs within the operating system accessed by the memory. 224 is feasible.

[0050] It goes without saying that Fig. 2 only for illustrative purposes and to show components of an electronic device 100 This is a schematic representation of the various components required for an electronic device, as provided for in various embodiments of the present invention. 100It can therefore contain various other components that are in Fig. 2 are not shown, or may include a combination of two or more components or a division of a particular component into two or more separate components and is still within the scope of the embodiments described herein.

[0051] The Fig. 3, Fig. 4 and Fig. 5 are block diagrams, the embodiments of the display of the electronic device. 100 in the AoD operating mode according to the embodiments of the present invention. As already explained above, the electronic device 100 Suitable for operation in full-power mode, intermediate power mode, and energy-saving mode. In intermediate or energy-saving mode, a significant portion of the display is... 105 deactivated to save power. The electronic device 100is designed to display information while operating in a mode with less power than full power mode (i.e., in AoD mode), with this information including, among other things, the time. 302 , a date and notifications 304 are, without limitation. Examples of notifications 304 Notifications that can be displayed in AoD mode include, but are not limited to, text notifications (e.g., SMS), calendar notifications, email notifications, notifications from other messaging apps, etc. In other words, any application suitable for providing notifications while the electronic device is in full-power mode. 100 , can also present a notification in AoD mode.

[0052] The electronic device 100 It may be suitable for displaying multiple notifications. 304 at the same time, as in Fig. 3 shown while the device is in AoD mode. The notifications are indeed 304 They are essentially arranged in a line in the middle of the display, but can also be positioned according to the user's wishes.

[0053] The electronic device described above 100 It is configured to receive input from a user via their various input devices (i.e., the touchscreen, buttons, etc.). The user can touch an icon that indicates the type of notification 304 the user wishes to read. Fig. Figure 4 shows an embodiment in which a user can select from the available notifications of Fig. 3. Calendar notification 304 has selected. In one embodiment, the electronic device 100 configured to display information 402 , which are relevant for the notification. Shown only as an example. Fig. 4, that the calendar notification 304 Information 402 is contained within the notification. When selecting calendar notifications... 304 in Fig. 3 shows the electronic device 100 the information 402 on. Similarly, the electronic device 100 Display the information associated with a selected notification without bypassing a screen lock and requiring authentication to switch to full performance mode.

[0054] As explained in detail below, the voice response module 233 of the electronic device 100 Configured to allow replies to notifications 304 , which occur during the operation of the electronic device 100 presented in AoD mode, via voice or speech command. Fig. Figure 5 merely shows an example of a voice- or speech-activated response to the appointment notification. The voice response module 233 In one embodiment, a symbol may 502 Displays an indicator that shows the electronic device is capable of receiving a voice command from a selected notification. For example, the voice response module 233 a microphone symbol 504 present to point out that the electronic device 100 suitable for a voice command for calendar notifications 304 to receive. Voice command suggestions are also possible. 504 will be presented. The voice response module 233 It is configured to recognize the type of notification selected (i.e., SMS, calendar, messaging, etc.) and to present possible voice commands. In the illustrated embodiment, a suggestion is made. 504for “Navigate”. Alternatively, a suggestion may be displayed. 504 A calendar event may also include a request to discard / delete the calendar event, or to postpone or reschedule the calendar event. The suggestions given here 504 These are just examples, as the expert will recognize that there are many potential voting command suggestions at this point. 504 could be presented.

[0055] In one embodiment, the voice command suggestions 504 Context-based. In other words: the voice command suggestions. 504 based on the type of notification selected and the context of the notification. For example, if the selected notification were a text message, the voice command suggestion would 504The suggested voice commands are "reply," "dismiss," etc. If the notification also contains, for example, a URL or a phone number, the 504 voice command suggestion may include an option to open the URL in a browser or an option to dial the phone number. Other or additional voice command suggestions 504 are being considered.

[0056] Fig. Figure 6 is a schematic block diagram illustrating an embodiment of a voice response module. 233 according to embodiments of the present invention. In one embodiment, the voice response module includes 233 a notification manager 602 , a notification renderer 604 , a differentiator 606 , an audio input 608, a delay manager 610 , a speech recognition engine 612 , a voice biometrics engine 614 and an authentication module 616 .

[0057] In one embodiment, the notification manager 602 Configured to handle or manage received or self-generated notifications while the electronic device is in AoD mode. The notification manager 602 It can track whether a notification has been answered, read, and / or closed. Furthermore, the notification manager can 602 be configured to store potential voice command suggestions 504, which are included in the notifications 304 correspond. For example, the notification manager can 602 be configured to store a list of potential voice commands for calendar notifications, a list of potential commands for SMS notifications, a list of potential commands for email notifications, etc.

[0058] In one embodiment, the notification manager 602Configured to generate voice command suggestions based on the notification context. These voice command suggestions can include information from the notification and can be notification-specific. For example, a voice command suggestion for a text message might include "reply to sender," etc. The notification manager 602 It can also be configured to recognize which notification the user has selected from among several notifications and restrict the voice commands received from the user to the selected notification. In other words, if a user selects an instant message from other notifications, the notification manager can 602According to its configuration, the following voice command will be applied to the selected notification. This allows a user to usefully respond to a chat message without having to say, for example, "Reply Tom," since the notification manager 602 has already recognized or identified the sender of the selected notification.

[0059] In another embodiment, the notification manager can 602 Voice commands or suggestions can be made through a special application. A third-party application, for example, can be integrated with the notification manager. 602 Suggest voice commands to be sent to the notification manager 602 They would otherwise be unknown, as these voting command suggestions may be unfamiliar or unusual.

[0060] The notification renderer 604is configured in one embodiment for the presentation or display of notifications 304 on the display of the electronic device 100 , while the electronic device 100 is in AoD mode. The notification renderer 604 is configured to present notifications according to a predefined format, as described in Fig. 3 is shown. Alternatively, the notification renderer can be used. 604 The presentation of notifications can be configured according to the user's preference. For example, the user's preference might require that the notifications be displayed in a certain way. 304 They are positioned not in a horizontal, but instead in a vertical orientation. In another alternative embodiment, the notifications can 304 be distributed across the screen according to the user's preference.

[0061] In one embodiment, the differentiator 606 analyzes or decomposes the notifications. 304 , to identify a notification type. For example, the differentiator 606 Trained to determine whether a notification is an email notification, a calendar notification, an SMS notification, etc. The differentiator 606The system is also trained to analyze the information associated with the notification and to determine the notification's context. The suggested voice commands can be based on the identified notification type and the context. In other words, the voice commands presented to the user can be based on the notification type and the context of the related information. For example, a calendar event that includes a location might receive the suggested voice command "Navigate," whereas a calendar event that does not include a location might receive a different suggested voice command, even though both examples are calendar events.

[0062] The audio input 608 In one embodiment, a microphone sensor is connected to the sensor hub. 224 from Fig. 2 is connected. The audio input 608It is configured to capture and receive voice commands from the user. In one embodiment, the audio input listens. 608 constantly after a trigger phrase such as "OK, make a call". Alternatively, the audio input 608 only activated when it is through the voice response module 233 is instructed accordingly. This reduces electricity consumption and storage requirements in a beneficial way.

[0063] The delay manager 610 is configured to determine a delay time before activating the audio input. 608 The delay time allows the user to read the information related to the notification before the audio input. 608 is activated. This delay time can begin as soon as the user selects a notification. In one implementation, the delay manager determines 610A delay time is based on the length (i.e., the amount of text) of the associated notification. A delay time for a six-word text message is shorter than a delay time for a thirty-word email. The difference in these two cases can be one or two seconds.

[0064] The speech recognition engine 612 It is configured to extract text from the received voice command. In other words, the speech recognition engine converts spoken language into written text, which can then be used to execute the voice command. In another embodiment, the speech recognition engine can 612 It must be configured to load context-specific grammar recognition patterns. The speech recognition engine 612It is configured in such a way that, for example, if the notification is related to calendar events, it loads grammar recognition patterns related to those events, such as: "What's the traffic like?", "Navigate to...", "Book a taxi", etc. However, the expert will recognize that many different speech recognition engines... 612 are available and can be used in the embodiments of the present invention.

[0065] The voice biometrics engine 614 It is configured to authenticate a user based on their voice. In other words, the voice biometrics engine 614 It identifies a user based on their speech pattern. The voice biometrics engine 614It is configured to extract personal voice and speech patterns in order to identify the user solely by voice and speech. A user's vocal tract is shaped by numerous physical characteristics, including the form of the larynx, mouth, and nose. Furthermore, the voice biometrics engine can 614 They are trained to identify the user based on learned behavioral patterns (e.g., pitch, speech pattern, etc.). These physical characteristics and patterns become part of the user's voice waveform (i.e., a voiceprint or speech model), which is difficult to change or alter. The voice biometrics engine 614 is configured to match a speaker with an authenticated user of the electronic device 100 .

[0066] The authentication module 616It is configured to bypass the screen lock in response to a positive voiceprint match of the speaker. The screen lock can also be referred to as a "firewall," which protects the electronic device from unauthorized access. The authentication module 616 enables the execution of a voice command in response to a positive voiceprint match by the voice biometrics engine. 614 The authentication module 616 The screen lock can also be bypassed in response to the successful entry of a password, PIN, etc. by the user.

[0067] Fig. Figure 7 shows a schematic flowchart illustrating an embodiment of a process. 700 to enable voice command responses to notifications in an AoD environment according to embodiments of the present invention. The method 700This is carried out by processing logic that may include hardware (circuits, special logic, etc.), software (such as that running on a general-purpose or dedicated computer), or a combination of hardware and software. In one embodiment, the voice response module performs this function. 233 the procedure 700 carry out. Alternatively, other components of the electronic device carry out. 100 the entire process 700 or part of it.

[0068] In one embodiment, the processing logic starts the procedure. 700 and receives in block 702 A notification in AoD mode. The processing logic can present the notification to the user or play this new notification along with other notifications that have already been received and / or generated, but not yet read or dismissed.

[0069] In one embodiment, the processing logic in block 704 receives a user selection of a notification. In another embodiment, the processing logic receives a user selection by receiving an input indicating a selection, including, for example, a touch of the screen that selects a notification, optionally from a group of notifications, without limitation. In block 706 The processing logic analyzes the notification to determine the notification type and the context of the information associated with the notification.

[0070] In the decision block 708The processing logic determines whether the now-identified notification type supports a voice response. In an example, the processing logic stores a table of notification types and their corresponding enabled voice responses, and determines whether the notification matches a notification type in the table. If the decision is "Yes," the processing logic in block [block name missing] determines [missing information]. 710 Based on the length of the information associated with the notification, a delay time is determined, and the opening of the microphone (i.e., the audio input) is delayed according to this delay time.

[0071] In block 712 The processing logic detects the voice command and recognizes the text of the command. In the decision block 714 The processing logic determines whether the speaker of the voice command is an authorized user of the electronic device. 100If so, the processing logic in block [block name] bypasses it. 716 A screen lock causes the electronic device to switch to full power mode and execute the command, thus completing the procedure. 700 ends. In an alternative embodiment, the processing logic directs the electronic device. 100 back to AoD mode after the voice command has been executed. In yet another alternative embodiment, the processing logic keeps the electronic device in a full-power mode after the voice command has been executed. If the response is either in decision block 708 or in decision block 714 If the answer is "no", the procedure ends at 700.

[0072] Embodiments can also be implemented in other practical ways. The described embodiments are in every respect exemplary and do not constitute a limitation of the invention. For this reason, the scope of protection of the invention is not defined by the present description, but rather by the attached claims. All modifications within the scope of meaning and equivalence of the claims fall within their scope of protection.

Claims

[1] A device comprising: a display that is configured to operate at least in a full-power mode or at least in an always-on display mode (“Aod” mode); a processor; a memory that stores code which can be executed by the processor while the display is in AoD mode to: to analyze a notification for the purpose of determining a notification type; to determine whether the notification supports a voice response; to receive a voice command from a user and to authenticate the user based on the voice command; and in response to the user's authorization to bypass the screen lock and execute the voice command. [2] Device according to claim 1, wherein the code is executable by the processor to display a notification to the user via the display. [3] Device according to claim 1 or claim 2, wherein the code is executable by the processor to activate a voice input which receives the voice command from the user after the user has acknowledged the notification. [4] Device according to any one of claims 1 to 3, wherein the code is executable by the processor to enable voice input after a delay from the time of confirmation of the notification by the user. [5] Device according to claim 4, wherein the code is executable by the processor to determine the delay based on a set of text corresponding to the notification. [6] Device according to any one of claims 1 to 5, wherein the code is executable by the processor to make suggestions for a voice command in response to either the specific notification type or to content of the notification. [7] Device according to any one of claims 1 to 6, wherein the code is executable by the processor to execute the voice command in full-power mode. [8] Device according to any one of claims 1 to 7, wherein the code is executable by the processor to return the display to AoD mode after the execution of the voice command. [9] Procedures, comprehensive: Displaying notifications at least in a full-performance mode or at least in an always-on display mode (“AoD” mode); Analyzing a notification to determine its notification type; Determining during AoD mode whether the notification supports voice responses; Receiving a voice command from a user and authorizing the user based on the voice command; and Bypassing a screen lock in response to user authorization and execution of the voice command. [10] Method according to claim 9, further comprising activating a voice input which receives the voice command from the user after the user has acknowledged the notification. [11] Method according to claim 9 or claim 10, further comprising enabling voice input after a delay from a time at which the user acknowledges the notification. [12] Method according to claim 11, further comprising determining the delay based on the amount of text corresponding to the notification. [13] Method according to any one of claims 9 to 12, further comprising presenting suggestions for a voting command in response to either the identified notification type or the content of the notification. [14] Method according to any one of claims 9 to 13, further comprising executing the voting command in full-power mode. [15] Method according to any one of claims 9 to 14, further comprising returning the display to AoD mode after execution of the voice command. [16] Program product comprising a computer-readable storage medium that stores instructions which, when executed by a processor of a computer device, cause the computer device to perform actions, comprising: Displaying notifications at least in a full-performance mode or at least in an always-on display mode (“AoD” mode); Analyzing a notification to determine the notification type; Determining, during AoD mode, whether a notification supports voice responses; Receiving a voice command from a user and authorizing the user based on the voice command; and Bypassing a screen lock in response to user authorization and execution of the voice command. [17] Program product according to claim 16, wherein the actions further include activating a voice input which receives the voice command from the user after the user has received the notification. [18] Program product according to claim 16 or 17, wherein the actions further include enabling voice input after a delay time from the time of confirmation of the notification by the user. [19] Program product according to claim 18, wherein the actions further include determining the delay based on the amount of text corresponding to the notification. [20] Program product according to one of claims 16 to 19, wherein the actions further include presenting voice command suggestions in response to either the specified notification type or a content of the notification.

Images