Dynamic registration of user-defined wake-up key phrases for computer systems with voice functionality

Abstract

Techniques are provided for voice wake-up (WOV) key phrase registration. A method implementing the techniques according to an embodiment includes generating a WOV key phrase model based on an identification of a sequence of sub-phonetic units of a user-provided key phrase. The WOV key phrase model is used by a WOV processor to detect the key phrase spoken by the user and trigger operation of an automatic speech recognition (ASR) processor in response to the detection. The method also includes updating an ASR language model based on the user-provided key phrase. The updating includes one of: embedding the WOV key phrase model into the ASR language model; converting sub-phonetic units of the WOV key phrase model and embedding the converted WOV key phrase model into the ASR language model; or generating an ASR key phrase model by applying a phoneme-syllable based statistical language model to the user-provided key phrase and embedding the generated ASR key phrase model into the ASR language model.

Description

Background Technology

[0001] Some computer systems or platforms become active or are "wake up" in response to detecting keywords or key phrases spoken by a user. Once woken up, the computer continues to recognize and process the additional user speech following the keyword. Such systems employ speech recognition technology and typically require key phrases to be predefined (e.g., static) rather than user-selectable, which is undesirable. Some other systems require the user to pause between the wake-up keyword and the rest of the requested speech to switch processing modes (e.g., from keyword discovery mode to full speech recognition mode). Such pauses interrupt natural speech communication and negatively impact the quality of the user experience. Attached Figure Description

[0002] The features and advantages of the claimed subject matter will become apparent as the following detailed description is carried out and with reference to the accompanying drawings, wherein the same reference numerals denote the same parts.

[0003] Figure 1 This is a top-level block diagram of a voice-enabled computer system configured according to certain embodiments of the present disclosure, which has dynamic wake-on-voice (WOV) key phrase enrollment.

[0004] Figure 2 This is a block diagram of a dynamic WOV key phrase registration system configured according to certain embodiments of the present disclosure.

[0005] Figure 3 This is another block diagram of a dynamic WOV key phrase registration system configured according to certain embodiments of the present disclosure.

[0006] Figure 4 This is another block diagram of a dynamic WOV key phrase registration system configured according to certain embodiments of the present disclosure.

[0007] Figure 5 This is a flowchart illustrating a method for dynamic WOV key phrase registration according to certain embodiments of the present disclosure.

[0008] Figure 6 This is a schematic diagram illustrating a computing platform configured to perform dynamic WOV key phrase registration according to certain embodiments of the present disclosure.

[0009] Although the following detailed description will be made with reference to illustrative embodiments, many alternatives, modifications and variations will be apparent from the present disclosure. Detailed Implementation

[0010] In general, this disclosure provides techniques for dynamic speech wake-up (WOV) keyphrase registration in spoken language understanding systems that combine WOV keyphrase discovery with automatic speech recognition (ASR). Keyphrase discovery techniques are used to detect the user's utterance of a wake-up phrase, which can then trigger ASR to recognize the complete request spoken by the user, including the wake-up phrase. Although keyphrase detection and ASR systems typically use different language models, the disclosed techniques allow for a degree of sharing of runtime-generated keyphrase models between the two systems, as will be explained in more detail below. This technique allows the user to dynamically select or customize the system to detect any desired wake-up phrase; conversely, this would be impossible in systems using separate, dedicated models for keyphrase detection and speech recognition.

[0011] The disclosed techniques can be implemented in computing systems or software products, for example, that can be executed or otherwise controlled by such systems, though other embodiments will be readily apparent. The system or product is configured to provide dynamic WOV key phrase registration. According to embodiments, a method for implementing these techniques includes generating a WOV key phrase model based on the identification and determination of correct sequences of sub-phonetic units of user-provided key phrases. A WOV processor uses the WOV key phrase model to detect key phrases spoken by the user and triggers operation of an automatic speech recognition (ASR) processor in response to the detection. The method also includes updating an ASR language model based on user-provided key phrases, which will be described in more detail below. In some embodiments, the update may include embedding the WOV key phrase model into the ASR language model. In other embodiments, the update may include performing a transformation of the sub-phonetic units of the WOV key phrase model and embedding the transformed WOV key phrase model into the ASR language model. In other embodiments, the update may include generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to the user-provided key phrases, and embedding the generated ASR key phrase model into the ASR language model. In other embodiments, the user-provided key phrases may be provided as text entries, which can be directly provided to the WOV processor and ASR processor for registration.

[0012] It will be understood that the techniques described herein can improve the user experience for speech language understanding systems by allowing dynamic registration of user-selected wake-up key phrases in a more user-friendly and transparent manner compared to existing methods (which enforce static key phrases or require awkward pauses between the key phrase and subsequent speech). The disclosed techniques can be implemented on a wide range of platforms, including laptops, tablets, smartphones, workstations, and embedded systems or devices. These techniques can be further implemented in hardware or software, or a combination thereof.

[0013] Figure 1 This is a top-level block diagram of a voice-enabled computer system 100 configured with dynamic wake-up voice (WOV) key phrase registration according to certain embodiments of this disclosure. The voice-enabled computer system 100 is shown as including a WOV processor 110, an ASR processor 120, and a dynamic WOV key phrase registration system 140. In some embodiments, the voice-enabled computer system 100 may be hosted on a computing or communication platform, as described below. Figure 6 As described in detail.

[0014] An example of audio input 130 is shown as including a voice from a user, in this example: “Hello, Computer, turn on the kitchen light.” The audio input may be provided by a microphone, a microphone array (e.g., configured for beamforming), or any other suitable audio capture device, or from a stored recording. The WOV processor 110 is configured to detect the key phrase “Hello, Computer” from the audio input 130, which has been selected by the user through a dynamic registration process implemented by the dynamic WOV key phrase registration system 140. The operation of the dynamic WOV key phrase registration system 140 is described in more detail below. The WOV processor 110 is also configured to trigger the ASR processor 120 to recognize the entire spoken expression, including both the key phrase and the remaining user request to turn on the kitchen light. In some embodiments, the trigger may further be used to wake the computing platform or any part thereof from a sleep or idle state to an active recognition state.

[0015] In some embodiments, the WOV processor 110 employs key phrase detection techniques known from this disclosure. These key phrase detection techniques may further employ key phrase models including wake-up key phrases predefined or selected by the user, and acoustic models for performing lower-level analysis of phonemes and sub-speech units after applying the key phrase model.

[0016] In some embodiments, the ASR processor 120 employs speech recognition techniques known from this disclosure. These speech recognition techniques may further employ a language model separate from the key phrase model, which includes a wake-up key phrase in addition to the expected syntax of the user request. As a simplified example, the ASR language model may be a syntax that recognizes “[Start|Stop|Pause] Play”, where the key phrase is “Hello, Computer”. In this case, the combined syntax understood by the language model is “Hello, Computer, [Start|Stop|Pause] Play”.

[0017] In some embodiments, at least when the ASR processor is in a higher power identification state, the WOV processor or circuit 110 is configured to consume less power than the ASR processor or circuit 120, thereby allowing power optimization between the two functions. For example, in some embodiments, the WOV processor 110 may be implemented on a low-power CPU or digital signal processor (DSP), and the ASR processor 120 may be implemented on a hardware accelerator or a suitably optimized coprocessor.

[0018] Figure 2 This is a block diagram of a dynamic WOV key phrase registration system 140a configured according to certain embodiments of the present disclosure. The dynamic WOV key phrase registration system 140a is shown to include a WOV key phrase model generation circuit 214, a WOV key phrase model 216, an ASR model update circuit 217, and an ASR language model 218. Additionally, a WOV acoustic model 222 is shown to be used by a voice wake-up processor 110, and an ASR acoustic model 224 is shown to be used by an ASR processor 120.

[0019] WOV key phrase model generation circuit 214 is configured to generate WOV key phrase model 216 based on the identification and determination of the correct sequence of sub-speech units of user-provided WOV registered key phrase 212. WOV key phrase model 216 is used by WOV processor 110 to detect WOV key phrases from audio input 130 spoken by the user during operation 220.

[0020] The ASR model update circuit 217 is configured to update the ASR language model 218 based on the user-provided WOV registered key phrases 212. The ASR language model 218 is used by the ASR processor 120 to recognize speech expressions from the speech input 130 spoken by the user during operation 220. The update is achieved using techniques known in this disclosure by incorporating or embedding the generated WOV key phrase model 216 into the ASR language model 218. The WOV acoustic model 222 and the ASR acoustic model 224 use common phoneme and state sequences (e.g., they use at least some of the same phonemes and state sequences), and in this respect, the techniques herein assume they are relatively similar.

[0021] Figure 3 This is another block diagram of a dynamic WOV key phrase registration system 140b configured according to certain embodiments of the present disclosure. The dynamic WOV key phrase registration system 140b is shown as including a WOV key phrase model generation circuit 314, a WOV key phrase model 316, a model conversion circuit 317, an ASR model update circuit 217, and an ASR language model 318. Additionally, a WOV acoustic model 322 is shown as being used by a voice wake-up processor 110, and an ASR acoustic model 324 is shown as being used by an ASR processor 120.

[0022] WOV key phrase model generation circuit 314 is configured to generate WOV key phrase model 316 based on the identification and determination of the correct sequence of sub-speech units of user-provided WOV registered key phrase 212. WOV key phrase model 316 is used by WOV processor 110 to detect WOV key phrases from audio input 130 spoken by the user during operation 220. Model conversion circuit 317 is configured to perform conversion of sub-speech units of WOV key phrase model 316 using techniques known from this disclosure.

[0023] ASR model update circuit 217 is configured to update ASR language model 318 using techniques known from this disclosure by merging or embedding the converted WOV key phrase model generated by model conversion circuit 317 into ASR language model 318. This is generally possible even if the WOV acoustic model 322 may differ from the ASR acoustic model 324 (as is often the case in most applications), because the conversion of sub-speech units provides compatibility between the converted WOV key phrase model and the ASR language model. This technique assumes that sub-speech unit conversion is possible for a given model, which may not always be the case.

[0024] Figure 4This is another block diagram of a dynamic WOV key phrase registration system 140c configured according to certain embodiments of the present disclosure. The dynamic WOV key phrase registration system 140c is shown as including a WOV key phrase model generation circuit 414, a WOV key phrase model 416, an ASR key phrase model generation circuit 417, an ASR model update circuit 217, and an ASR language model 418. Furthermore, a WOV acoustic model 422 is shown as being used by a voice wake-up processor 110, and an ASR acoustic model 424 is shown as being used by an ASR processor 120.

[0025] The WOV key phrase model generation circuit 414 is configured to generate a WOV key phrase model 416 based on the identification and determination of the correct sequence of sub-speech units of the user-provided WOV registered key phrase 212. The WOV key phrase model 416 is used by the WOV processor 110 to detect WOV key phrases from the audio input 130 spoken by the user during operation 220.

[0026] The ASR key phrase model generation circuit 417 is configured to generate an ASR key phrase model by applying a phoneme-syllable-based statistical language model (SLM) to user-provided WOV registered key phrases 212. Given a history of previously identified words (e.g., based on a hidden Markov model or a maximum entropy model), the SLM calculates the probability of the words. Key phrases can be identified and extracted from the identification result lattice (which represents different assumptions about what was said), and then the lattice is directly embedded into the ASR statistical language model.

[0027] The ASR model update circuit 217 is configured to update the ASR language model 418 using techniques known from this disclosure by merging or embedding new ASR key phrase models generated by the ASR key phrase model generation circuit 417 into the ASR language model 418. This is generally feasible, requiring no other assumptions about the WOV acoustic model 422 and the ASR acoustic model 424 except that the ASR processor 120 and the ASR key phrase model generation circuit 417 use the same ASR acoustic model 424.

[0028] In some embodiments, the dynamic WOV key phrase registration system 140 may further include a character-to-phoneme conversion circuit (not shown) to allow user-defined key phrases 212 to be input as text. The character-to-phoneme conversion circuit is configured to convert text entries into phonemes to generate WOV key phrase models via WOV key phrase model generation circuits 214, 314, and 414. The character-to-phoneme mapping may be provided directly by the user or obtained using techniques known from this disclosure as a statistical model generated through a machine learning process.

[0029] method

[0030] Figure 5 This is a flowchart illustrating an example method 500 for dynamic WOV key phrase registration according to certain embodiments of the present disclosure. It can be seen that this example method includes multiple stages and sub-processes, the order of which may vary depending on the embodiment. However, when considered in general, these stages and sub-processes form a process for key phrase registration according to certain embodiments of the present disclosure. This can be achieved, for example, by using the methods described above. Figure 1-4 The system architecture shown in this disclosure is used to implement these embodiments. However, it will be apparent from this disclosure that other system architectures may be used in other embodiments. Therefore, Figure 5 The relevance of the various functions shown to the figures and the specific components shown in the other figures is not intended to imply any structural and / or usage limitations. Rather, other embodiments may include, for example, varying degrees of integration, where multiple functions are efficiently performed by a single system. For example, in an alternative embodiment, a single module with decoupled submodules may be used to perform all the functions of method 500. Thus, depending on the granularity of the implementation, other embodiments may have fewer or more modules and / or submodules. In other embodiments, the depicted method may be implemented as a computer program product comprising one or more non-transitory machine-readable media, which causes the method to be performed when executed by one or more processors. Many variations and alternative configurations will be apparent from this disclosure.

[0031] like Figure 5 As shown, in this embodiment, the method 500 for dynamic WOV key phrase registration begins at operation 510 by generating a WOV key phrase model based on the identification and determination of the correct sequence of sub-speech units of the user-provided WOV registered key phrase. The WOV key phrase model is used by the WOV processor to detect the WOV key phrase when the user speaks it, thereby triggering the operation of the ASR processor in response to this key phrase detection.

[0032] Next, at operation 520, the ASR language model is updated based on the user-provided WOV registered key phrases. The ASR language model is used by the ASR processor to recognize the user's spoken speech.

[0033] In some embodiments, updating the ASR language model further includes directly incorporating the WOV key phrase model into the ASR language model. In other embodiments, updating the ASR language model further includes performing sub-speech conversion of the WOV key phrase model and incorporating the converted WOV key phrase model into the ASR language model. In other embodiments, updating the ASR language model further includes generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to user-provided WOV registered key phrases, and incorporating the generated ASR key phrase model into the ASR language model.

[0034] Of course, in some embodiments, as previously described in conjunction with the system, additional operations can be performed. For example, user-provided WOV registration key phrases can be provided as text entries, in which case a character-to-phoneme conversion can be performed on the text entries to generate a WOV key phrase model. In some embodiments, triggering the ASR processor can further include waking the ASR processor from a lower-power idle state to a higher-power recognition state.

[0035] Example System

[0036] Figure 6 An example system 600 configured according to certain embodiments of the present disclosure for performing dynamic WOV key phrase registration is illustrated. In some embodiments, system 600 includes a computing platform 610, which may be hosted or otherwise incorporated into a personal computer, workstation, server system, smart home management system, laptop computer, ultra-laptop computer, tablet computer, touchpad, portable computer, handheld computer, PDA, personal digital assistant (PDA), cellular phone, combination of cellular phone and PDA, smart device (e.g., smartphone or smart tablet), mobile internet device (MID), messaging device, data communication device, imaging device, wearable device, embedded system, etc. Any combination of different devices may be used in some embodiments.

[0037] In some embodiments, platform 610 may include any combination of processor 620, memory 630, dynamic WOV key phrase registration system 140, WOV processor 110, ASR processor 120, network interface 640, input / output (I / O) system 650, user interface 660, audio capture device 662, and storage system 670. It can also be seen that a bus and / or interconnect 692 is provided to allow communication between the various components listed above and / or other components not shown. Platform 610 may be coupled to network 694 via network interface 640 to allow communication with other computing devices, platforms, or resources. As per this disclosure, not reflected in… Figure 6 Other components and functions in the block diagram will be obvious, and it will be understood that other embodiments are not limited to any particular hardware configuration.

[0038] Processor 620 can be any suitable processor and may include one or more coprocessors or controllers (e.g., an audio processor, a graphics processing unit, or a hardware accelerator) to assist in control and processing operations associated with system 600. In some examples, processor 620 may be implemented as any number of processor cores. A processor (or processor core) can be any type of processor, such as a microprocessor, embedded processor, digital signal processor (DSP), graphics processing unit (GPU), network processor, field-programmable gate array, or other device configured to execute code. Processors may be multi-threaded cores, as each core may include more than one hardware thread context (or “logical processor”). Processor 620 may be implemented as a Complex Instruction Set Computer (CISC) or Reduced Instruction Set Computer (RISC) processor. In some embodiments, processor 620 may be configured as an x86 instruction set compatible processor.

[0039] Memory 630 can be implemented using any suitable type of digital storage device, including, for example, flash memory and / or random access memory (RAM). In some embodiments, as known to those skilled in the art, memory 630 may include various layers of a memory hierarchy and / or memory cache. Memory 630 may be implemented as a volatile memory device, such as, but not limited to, RAM, dynamic RAM (DRAM), or static RAM (SRAM) devices. Storage system 670 may be implemented as a non-volatile storage device, such as, but not limited to, one or more of hard disk drive (HDD), solid-state drive (SSD), universal serial bus (USB) drive, optical disk drive, tape drive, internal storage device, attached storage device, flash memory, backup battery synchronous DRAM (SDRAM), and / or network-accessible storage device. In some embodiments, when multiple hard disk drives are included, storage device 670 may include techniques for enhancing storage performance protection for valuable digital media. In some embodiments, WOV key phrase models, ASR language models, and acoustic models may be stored in separate blocks or areas of memory.

[0040] Processor 620 can be configured to execute operating system (OS) 680, which may include any suitable operating system, such as Google Android (Google, Inc., Mountain View, California), Microsoft Windows (Microsoft, Inc., Redmond, Washington), Apple OS X (Apple, Inc., Cupertino, California), Linux, or a real-time operating system (RTOS). As will be understood from this disclosure, the techniques provided herein can be implemented without regard to a specific operating system provided in conjunction with system 600, and therefore can also be implemented using any suitable existing or subsequently developed platform.

[0041] Network interface circuitry 640 may be any suitable network chip or chipset that allows wired and / or wireless connectivity between network 694 and / or other components of computer system 600, enabling system 600 to communicate with other local and / or remote computing systems, servers, cloud-based servers, and / or other resources. Wired communication may conform to existing (or under development) standards such as Ethernet. Wireless communication may conform to existing (or under development) standards such as cellular communication including LTE (Long Term Evolution), Wi-Fi, Bluetooth, and / or Near Field Communication (NFC). Exemplary wireless networks include, but are not limited to, wireless LANs, wireless personal area networks, wireless metropolitan area networks, cellular networks, and satellite networks.

[0042] I / O system 650 can be configured to interface between various I / O devices and other components of computer system 600. I / O devices may include, but are not limited to, user interface 660 and audio capture device 662 (e.g., microphone). User interface 660 may include devices (not shown) such as display elements, touchpads, keyboards, mice, and speakers. I / O system 650 may include a graphics subsystem configured to perform image processing for presentation on a display element. The graphics subsystem may be, for example, a graphics processing unit or a visual processing unit (VPU). Analog or digital interfaces can be used to communicatively couple the graphics subsystem and the display element. For example, the interface may be any of the following: High Definition Multimedia Interface (HDMI), DisplayPort, Wireless HDMI, and / or any other suitable interface using technology compliant with Wireless High Definition. In some embodiments, the graphics subsystem may be integrated into any chipset of processor 620 or platform 610.

[0043] It will be understood that, in some embodiments, various components of system 600 may be combined or integrated within a system-on-a-chip (SoC) architecture. In some embodiments, a component may be a hardware component, firmware component, software component, or any suitable combination of hardware, firmware, or software.

[0044] As previously described, the dynamic WOV key phrase registration system 140 is configured to generate a WOV key phrase model and update the ASR language model at runtime, enabling users to define new WOV key phrases. As mentioned above, the dynamic WOV key phrase registration system 140 may include... Figure 1-4 Any or all of the circuits / components shown. These components may be implemented or otherwise used in conjunction with various suitable software and / or hardware coupled to or otherwise forming part of platform 610. These components may additionally or alternatively be implemented or otherwise used in conjunction with user I / O devices capable of providing information to and receiving information and commands from the user.

[0045] In some embodiments, these circuits may be installed locally on system 600, such as Figure 6As illustrated in the example embodiments, system 600 can alternatively be implemented in a client-server configuration, wherein at least some of the functionality associated with these circuits is provided to system 600 using applets (e.g., JavaScript applets) or other sets of downloadable modules or submodules. Such remotely accessible modules or submodules can be configured in real-time in response to a request from a client computing system to access a given server, wherein the given server has resources of interest to the user of the client computing system. In such embodiments, the server may be coupled locally to network 694 or remotely via one or more other networks and / or communication channels. In some cases, accessing resources on a given network or computing system may require credentials such as usernames and passwords and / or adherence to any other suitable security mechanisms.

[0046] In various embodiments, system 600 can be implemented as a wireless system, a wired system, or a combination of both. When implemented as a wireless system, system 600 may include components and interfaces suitable for communication via a wireless shared medium, such as one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, etc. Examples of a wireless shared medium may include a portion of the wireless spectrum, such as radio frequency spectrum. When implemented as a wired system, system 600 may include components and interfaces suitable for communication via a wired communication medium, such as input / output adapters, physical connectors for connecting input / output adapters to a corresponding wired communication medium, network interface cards (NICs), disk controllers, video controllers, audio controllers, etc. Examples of a wired communication medium may include wires, cable leads, printed circuit boards (PCBs), backplanes, switch fabrics, semiconductor materials, twisted pairs, coaxial cables, optical fibers, etc.

[0047] Various embodiments can be implemented using hardware components, software components, or a combination of both. Examples of hardware components may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, ASICs, programmable logic devices, digital signal processors, FPGAs, logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc. Examples of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application programming interfaces, instruction sets, computational code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. The determination of whether to use hardware components and / or software components to implement an embodiment can vary based on any number of factors, such as desired computational speed, power level, thermal tolerance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed, and other design or performance constraints.

[0048] Some embodiments may be described using the expressions “coupled” and “connected” and their derivatives. These terms are not intended to be synonymous with each other. For example, some embodiments may be described using the terms “connected” and / or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term “coupled” may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0049] The various embodiments disclosed herein can be implemented in various forms, including hardware, software, firmware, and / or dedicated processors. For example, in one embodiment, at least one non-transitory computer-readable storage medium has instructions encoded thereon that, when executed by one or more processors, enable one or more of the dynamic key phrase registration methods disclosed herein to be implemented. The instructions can be implemented using a suitable programming language (e.g., C, C++, object-oriented C, Java, JavaScript, Visual C++, etc.). Basic.NET, Beginner's General Symbolic Instruction Code (BASIC), or alternatively, custom or proprietary instruction sets may be used for encoding. Instructions may be provided as one or more computer software applications and / or applets tangibly embodied on a memory device and executable by a computer with any suitable architecture. In one embodiment, the system may be hosted on a given website and implemented using, for example, JavaScript or another suitable browser-based technology. For example, in some embodiments, the system may utilize the processing resources provided by a remote computer system accessible via Network 694. In other embodiments, the functionality disclosed herein may be incorporated into other voice-based software applications, such as car control / navigation, smart home management, entertainment, and robotics applications. The computer software applications disclosed herein may include any number of different modules, submodules, or other components with different functions and may provide information to or receive information from other components. These modules may be used, for example, to communicate with input and / or output devices (e.g., displays, touch-sensitive surfaces, printers, and / or any other suitable devices). Other component portions and functions not reflected in the illustrations will be apparent according to this disclosure, and it should be understood that other embodiments are not limited to any particular hardware or software configuration. Therefore, in other embodiments, with Figure 6 Compared to those included in the example embodiments, system 600 may include additional, fewer, or alternative sub-components.

[0050] The aforementioned non-transitory computer-readable medium can be any suitable medium for storing digital information, such as a hard disk drive, server, flash memory, and / or random access memory (RAM), or a combination of memories. In alternative embodiments, the components and / or modules disclosed herein can be implemented in hardware, including gate-level logic such as a field-programmable gate array (FPGA), or alternatively, application-specific semiconductors such as an application-specific integrated circuit (ASIC). Other embodiments can be implemented using a microcontroller having multiple input / output ports for receiving and outputting data and multiple embedded routines for performing the various functions disclosed herein. It will be apparent that any suitable combination of hardware, software, and firmware can be used, and other embodiments are not limited to any particular system architecture.

[0051] Some embodiments may be implemented, for example, by using a machine-readable medium or article that can store instructions or instruction sets that, if executed by a machine, can cause the machine to perform the methods and / or operations according to the embodiments. Such a machine may include, for example, any suitable processing platform, computing platform, computing device, processing device, computing system, processing system, computer, processing, etc., and may be implemented using any suitable combination of hardware and / or software. Machine-readable media or articles may include, for example, any suitable type of memory cell, memory device, memory article, memory medium, storage device, storage article, storage medium, and / or storage unit, such as memory, removable or non-removable media, erasable or non-erasable media, writable or rewritable media, digital or analog media, hard disk, floppy disk, optical disc read-only memory (CD-ROM), optical disc recordable (CD-R) memory, optical disc rewritable (CD-RW) memory, optical disc, magnetic media, magneto-optical media, removable memory cards or discs, various types of digital multifunction discs (DVDs), magnetic tape, cassette tape, etc. Instructions may include any appropriate type of code (e.g., source code, compiled code, interpreted code, executable code, static code, dynamic code, cryptographic code, etc.) implemented using any appropriate high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0052] Unless otherwise specified, it will be understood that terms such as “processing,” “computing,” “calculating,” “determining,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or convert data representing physical quantities (e.g., electrons) within registers and / or memory units of the computer system into other data representing physical quantities similarly represented within registers, memory units, or other such information storage, transmission, or display of the computer system. Embodiments are not limited to this context.

[0053] As used in any embodiment herein, the terms "circuit" or "circuit system" are functional and may, for example, individually or in any combination, include hardwired circuit systems, programmable circuit systems such as computer processors including one or more individual instruction processing cores, state machine circuit systems, and / or firmware storing instructions executed by the programmable circuit system. A circuit system may include a processor and / or controller configured to execute one or more instructions to perform one or more operations described herein. These instructions may be embodied, for example, as an application, software, firmware, etc., configured to cause the circuit system to perform any of the above operations. Software may be embodied as software packages, code, instructions, instruction sets, and / or data recorded on a computer-readable storage device. Software may be embodied or implemented as comprising any number of processes, and consequently, processes may be embodied or implemented as comprising any number of threads, etc., in a hierarchical manner. Firmware may be embodied as hard-coded (e.g., non-volatile) code, instructions, or instruction sets and / or data in a memory device. A circuit system can be embodied collectively or individually as a circuit system forming part of a larger system, such as an integrated circuit (IC), an application-specific integrated circuit (ASIC), a system-on-a-chip (SoC), a desktop computer, a laptop computer, a tablet computer, a server, a smartphone, etc. Other embodiments can be implemented as software executed by a programmable control device. In this context, the term "circuit" or "circuit system" is intended to include a combination of software and hardware, such as a processor or programmable control device capable of executing software. As described herein, various embodiments can be implemented using hardware elements, software elements, or any combination thereof. Examples of hardware elements may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chips, microchips, chipsets, etc.

[0054] This document has set forth numerous specific details to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that these embodiments can be practiced without these specific details. In other instances, well-known operations, components, and circuits have not been described in detail so as not to obscure the embodiments. It is understood that the specific structural and functional details disclosed herein may be representative but do not necessarily limit the scope of the embodiments. Furthermore, although the subject matter has been described in language specific to structural features and / or method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described herein. Rather, the specific features and actions described herein are disclosed as exemplary forms for implementing the claims.

[0055] Other example embodiments

[0056] The following examples relate to other embodiments, from which many permutations and configurations will be apparent.

[0057] Example 1 is a processor implementation method for registering Word Wake-up (WOV) key phrases, comprising: generating a WOV key phrase model by a processor-based system based on user-provided WOV registered key phrases, the WOV key phrase model being used by a WOV processor to detect the correct sequence of sub-speech units of the WOV key phrases spoken by the user and triggering an operation of an automatic speech recognition (ASR) processor in response to the WOV key phrase detection; and updating an ASR language model by the processor-based system based on user-provided WOV registered key phrases, the ASR language model being used by an ASR processor to recognize the speech expressions spoken by the user.

[0058] Example 2 includes the topics of Example 1, where the update to the ASR language model includes incorporating the WOV key phrase model into the ASR language model.

[0059] Example 3 includes the subject of Example 1 or 2, wherein the WOV processor uses a first acoustic model (AM), the ASR processor uses a second acoustic model (AM), and the first AM and the second AM use a common phoneme and state sequence.

[0060] Example 4 includes the subject of any of Examples 1-3, wherein updating the ASR language model includes performing a transformation of the sub-speech units of the WOV key phrase model and merging the transformed WOV key phrase model into the ASR language model.

[0061] Example 5 includes the subject of any of Examples 1-4, wherein updating the ASR language model includes generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to user-provided WOV registered key phrases, and merging the generated ASR key phrase model into the ASR language model.

[0062] Example 6 includes the topic of any of Examples 1-5, wherein user-provided WOV registration key phrases are provided as text entries, and the method further includes performing a grapheme-to-phoneme conversion on the text entries for generating a WOV key phrase model.

[0063] Example 7 includes the subject of any of Examples 1-6, wherein triggering the ASR processor includes waking the ASR processor from a lower power idle state to a higher power recognition state.

[0064] Example 8 includes the subject of any of Examples 1-7, wherein the WOV processor consumes less power than the ASR processor when the ASR processor is in a higher power identification state.

[0065] Example 9 is a system for registering voice wake-up (WOV) key phrases, comprising: a WOV key phrase model generation circuit for generating a WOV key phrase model based on user-provided WOV registered key phrases, the WOV key phrase model being used by a WOV processor to detect the correct sequence of sub-speech units of the WOV key phrases spoken by the user and to trigger an operation of an automatic speech recognition (ASR) processor in response to the WOV key phrase detection; and an ASR model updating circuit for updating an ASR language model based on user-provided WOV registered key phrases, the ASR language model being used by an ASR processor to recognize the speech expressions spoken by the user.

[0066] Example 10 includes the topics of Example 9, where the ASR model update circuit is also used to incorporate the WOV key phrase model into the ASR language model.

[0067] Example 11 includes the subject of Example 9 or 10, where the WOV processor uses a first acoustic model (AM), the ASR processor uses a second acoustic model (AM), and the first AM and the second AM use a common phoneme and state sequence.

[0068] Example 12 includes the subject of any of Examples 9-11, and also includes model conversion circuitry for performing the conversion of sub-speech units of the WOV key phrase model and merging the converted WOV key phrase model into the ASR language model.

[0069] Example 13 includes the subject of any of Examples 9-12, and also includes an ASR key phrase model generation circuit for generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to user-provided WOV registered key phrases, and merging the generated ASR key phrase model into an ASR language model.

[0070] Example 14 includes the subject of any of Examples 9-13, wherein user-provided WOV registration key phrases are provided as text entries, and the system also includes a character-to-phoneme conversion circuit for converting the text entries into phonemes for use in generating a WOV key phrase model.

[0071] Example 15 includes the subject of any of Examples 9-14, wherein triggering the ASR processor includes waking the ASR processor from a low-power idle state to a high-power recognition state.

[0072] Example 16 includes the subject of any of Examples 9-15, wherein the WOV processor consumes less power than the ASR processor when the ASR processor is in a higher power identification state.

[0073] Example 17 is at least one non-transitory computer-readable storage medium encoded with instructions that, when executed by one or more processors, produce the following operations for voice wake-up (WOV) key phrase registration, including: generating a WOV key phrase model based on user-provided WOV registered key phrases, the WOV key phrase model being used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrases spoken by the user; and triggering operations of an automatic speech recognition (ASR) processor in response to the WOV key phrase detection; and updating an ASR language model based on user-provided WOV registered key phrases, the ASR language model being used by the ASR processor to recognize the speech expressions spoken by the user.

[0074] Example 18 contains the subject of Example 17, where the update to the ASR language model includes the operation of merging the WOV key phrase model into the ASR language model.

[0075] Example 19 includes the subject of Example 17 or 18, where the WOV processor uses a first acoustic model (AM), the ASR processor uses a second acoustic model (AM), and the first AM and the second AM use shared phonemes and state sequences.

[0076] Example 20 includes the subject of any of Examples 17-19, wherein updating the ASR language model includes performing a transformation of the sub-speech units of the WOV key phrase model and merging the transformed WOV key phrase model into the ASR language model.

[0077] Example 21 includes the subject of any of Examples 17-20, wherein updating the ASR language model includes generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to user-provided WOV registered key phrases and merging the generated ASR key phrase model into the ASR language model.

[0078] Example 22 includes the subject of any of Examples 17-21, wherein user-provided WOV registration key phrases are provided as text entries, and also includes operations that perform a grapheme-to-phoneme conversion on the text entries for use in generating a WOV key phrase model.

[0079] Example 23 includes the subject of any of Examples 17-22, wherein triggering the ASR processor includes an operation that wakes the ASR processor from a lower power idle state to a higher power recognition state.

[0080] Example 24 includes the subject of any of Examples 17-23, wherein the WOV processor consumes less power than the ASR processor when the ASR processor is in a high-power recognition state.

[0081] Example 25 is a system for registering voice wake-up (WOV) key phrases, comprising: components for generating a WOV key phrase model based on user-provided WOV registered key phrases, the WOV key phrase model being used by a WOV processor to detect the correct sequence of sub-speech units of a user-spoken WOV key phrase and triggering operations of an automatic speech recognition (ASR) processor in response to WOV key phrase detection; and components for updating an ASR language model based on user-provided WOV registered key phrases, the ASR language model being used by an ASR processor to recognize user-spoken speech expressions.

[0082] Example 26 includes the subject of Example 25, where the update to the ASR language model includes components for incorporating the WOV key phrase model into the ASR language model.

[0083] Example 27 includes the subject of Example 25 or Example 26, wherein the WOV processor uses a first acoustic model (AM), the ASR processor uses a second acoustic model (AM), and the first AM and the second AM use a common phoneme and state sequence.

[0084] Example 28 includes the subject of any of Examples 25-27, wherein the update to the ASR language model includes components for performing the transformation of sub-speech units of the WOV key phrase model and components for incorporating the transformed WOV key phrase model into the ASR language model.

[0085] Example 29 includes the subject of any of Examples 25-28, wherein the update to the ASR language model includes a component for generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to user-provided WOV registered key phrases, and a component for incorporating the generated ASR key phrase model into the ASR language model.

[0086] Example 30 includes the subject of any of Examples 25-29, wherein user-provided WOV registration key phrases are provided as text entries, and the system also includes components for performing a grapheme-to-phoneme conversion on the text entries for generating a WOV key phrase model.

[0087] Example 31 includes the subject of any of Examples 25-30, wherein triggering the ASR processor includes components for waking the ASR processor from a lower power idle state to a higher power recognition state.

[0088] Example 32 includes the subject of any of Examples 25-31, wherein the WOV processor consumes less power than the ASR processor when the ASR processor is in a higher power identification state.

[0089] The terms and expressions used herein are descriptive and not limiting, and in using such terms and expressions, no equivalents of the features shown and described (or portions thereof) are intended to be excluded, and it should be understood that various modifications may be made within the scope of the claims. Therefore, the claims are intended to cover all such equivalents. Various features, aspects, and embodiments have been described herein. As will be understood by those skilled in the art, features, aspects, and embodiments are readily combined with each other, as well as varied and modified. Therefore, this disclosure should be considered to include such combinations, variations, and modifications. The scope of this disclosure is not intended to be limited to this detailed description, but is defined by the appended claims. Future applications claiming priority to this application may claim the disclosed subject matter in different ways and, in general, may include any set of one or more elements as disclosed or otherwise shown herein.

Claims

1. A processor-implemented method for registering voice wake-up (WOV) key phrases, the method comprising: The processor-based system generates a WOV key phrase model based on user-provided WOV registered key phrases. The WOV key phrase model is used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrases spoken by the user and to trigger the operation of the automatic speech recognition (ASR) processor in response to the WOV key phrase detection. as well as The processor-based system updates the ASR language model based on the WOV registered key phrases provided by the user. The ASR language model is used by the ASR processor to recognize the spoken expressions of the user. Updating the ASR language model includes generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to the WOV registered key phrases provided by the user, and embedding the generated ASR key phrase model into the ASR language model.

2. The method according to claim 1, wherein, The user-provided WOV registration key phrases are provided as text entries, and the method further includes: performing a character-to-phoneme conversion on the text entries to generate the WOV key phrase model.

3. The method according to any one of claims 1-2, wherein, Triggering the ASR processor includes waking it up from a low-power idle state to a high-power recognition state.

4. The method according to claim 3, wherein, When the ASR processor is in the higher power consumption identification state, the WOV processor consumes less power than the ASR processor.

5. A system for registering key phrases for voice wake-up (WOV), the system comprising: The WOV key phrase model generation circuit is used to generate a WOV key phrase model based on the WOV registered key phrases provided by the user. The WOV key phrase model is used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrase spoken by the user and to trigger the operation of the automatic speech recognition (ASR) processor in response to the WOV key phrase detection. An ASR model update circuit is used to update the ASR language model based on the WOV registered key phrases provided by the user. The ASR language model is used by the ASR processor to recognize the speech expressions spoken by the user. as well as An ASR key phrase model generation circuit is used to generate an ASR key phrase model by applying a phoneme-syllable-based statistical language model to the user-provided WOV registered key phrases, and to embed the generated ASR key phrase model into the ASR language model.

6. The system according to claim 5, wherein, The user-provided WOV registration key phrases are provided as text entries. The system also includes a character-to-phoneme conversion circuit for converting the text entries into phonemes for generating the WOV key phrase model.

7. The system according to any one of claims 5-6, wherein, Triggering the ASR processor includes waking it up from a low-power idle state to a high-power recognition state.

8. The system according to claim 7, wherein, When the ASR processor is in the higher power consumption identification state, the WOV processor consumes less power than the ASR processor.

9. A system for registering key phrases for voice wake-up (WOV), the system comprising: A component for generating a WOV key phrase model based on user-provided WOV registered key phrases, the WOV key phrase model being used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrase spoken by the user and to trigger the operation of the automatic speech recognition (ASR) processor in response to the WOV key phrase detection. as well as A component for updating an ASR language model based on user-provided WOV registered key phrases, the ASR language model being used by the ASR processor to recognize spoken expressions by the user, wherein updating the ASR language model includes: generating an ASR key phrase model by applying a phoneme-syllable-based statistical language model to the user-provided WOV registered key phrases, and embedding the generated ASR key phrase model into the ASR language model.

10. The system according to claim 9, wherein, The user-provided WOV registration key phrases are provided as text entries, and the system further includes: a component for performing a character-to-phoneme conversion on the text entries to generate the WOV key phrase model.

11. The system according to any one of claims 9-10, wherein, Triggering the ASR processor includes waking it up from a low-power idle state to a high-power recognition state.

12. The system according to claim 11, wherein, When the ASR processor is in the higher power consumption identification state, the WOV processor consumes less power than the ASR processor.

13. A processor-implemented method for registering voice wake-up (WOV) key phrases, the method comprising: The processor-based system generates a WOV key phrase model based on user-provided WOV registered key phrases. The WOV key phrase model is used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrases spoken by the user and to trigger the operation of the automatic speech recognition (ASR) processor in response to the WOV key phrase detection. as well as The processor-based system updates the ASR language model based on the WOV registered key phrases provided by the user. The ASR language model is used by the ASR processor to recognize the speech expressions spoken by the user. Updating the ASR language model includes: performing a conversion of the sub-speech units of the WOV key phrase model and embedding the converted WOV key phrase model into the ASR language model.

14. The method according to claim 13, wherein, The user-provided WOV registration key phrases are provided as text entries, and the method further includes: performing a character-to-phoneme conversion on the text entries to generate the WOV key phrase model.

15. The method according to any one of claims 13-14, wherein, Triggering the ASR processor includes waking it up from a low-power idle state to a high-power recognition state.

16. The method according to claim 15, wherein, When the ASR processor is in the higher power consumption identification state, the WOV processor consumes less power than the ASR processor.

17. A system for registering key phrases for voice wake-up (WOV), the system comprising: The WOV key phrase model generation circuit is used to generate a WOV key phrase model based on the WOV registered key phrases provided by the user. The WOV key phrase model is used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrase spoken by the user and to trigger the operation of the automatic speech recognition (ASR) processor in response to the WOV key phrase detection. An ASR model update circuit is used to update the ASR language model based on the WOV registered key phrases provided by the user. The ASR language model is used by the ASR processor to recognize the speech expressions spoken by the user. as well as A model conversion circuit is used to perform the conversion of sub-speech units of the WOV key phrase model and to embed the converted WOV key phrase model into the ASR language model.

18. The system according to claim 17, wherein, The user-provided WOV registration key phrases are provided as text entries. The system also includes a character-to-phoneme conversion circuit for converting the text entries into phonemes for generating the WOV key phrase model.

19. The system according to any one of claims 17-18, wherein, Triggering the ASR processor includes waking it up from a low-power idle state to a high-power recognition state.

20. The system according to claim 19, wherein, When the ASR processor is in the higher power consumption identification state, the WOV processor consumes less power than the ASR processor.

21. A system for registering key phrases for voice wake-up (WOV), the system comprising: A component for generating a WOV key phrase model based on user-provided WOV registered key phrases, the WOV key phrase model being used by the WOV processor to detect the correct sequence of sub-speech units of the WOV key phrase spoken by the user and to trigger the operation of the automatic speech recognition (ASR) processor in response to the WOV key phrase detection. as well as A component for updating an ASR language model based on WOV registered key phrases provided by the user, the ASR language model being used by the ASR processor to recognize spoken expressions by the user, wherein updating the ASR language model includes: performing a conversion of sub-speech units of the WOV key phrase model, and embedding the converted WOV key phrase model into the ASR language model.

22. The system according to claim 21, wherein, The user-provided WOV registration key phrases are provided as text entries, and the system further includes: a component for performing a character-to-phoneme conversion on the text entries to generate the WOV key phrase model.

23. The system according to any one of claims 21-22, wherein, Triggering the ASR processor includes waking it up from a low-power idle state to a high-power recognition state.

24. The system according to claim 23, wherein, When the ASR processor is in the higher power consumption identification state, the WOV processor consumes less power than the ASR processor.

Images