Always-on keyword detector

Abstract

This document provides an integrated circuit that, in some embodiments, includes: a dedicated host processor; a neuromorphic coprocessor; and a communication interface between the host processor and the coprocessor, configured to transfer information between them. The dedicated host processor can operate as a standalone host processor. The neuromorphic coprocessor includes an artificial neural network. The coprocessor is configured to enhance the dedicated processing of the host processor via the artificial neural network. In this embodiment, the host processor is a keyword identifier processor configured to transmit one or more detected words to the coprocessor via the communication interface. The coprocessor is configured to transmit recognized words or other sounds to the host processor.

Description

[0001] This is a divisional application. The parent application is entitled "Always-On Keyword Detector", filed on December 28, 2018, with application number 201880090472.7.

[0002] priority

[0003] This application claims the benefit of priority to U.S. Patent Application No. 16 / 235,396, filed December 28, 2018, which claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 611,512, filed December 28, 2017, entitled “Always-On Keyword Detector,” which are incorporated herein by reference in their entirety. Technical Field

[0004] The field of this disclosure generally relates to neuromorphic computing. More specifically, the field of the invention relates to an apparatus and method for a low-power, always-on keyword detector. Background Technology

[0005] Traditional central processing units (“CPUs”) process instructions based on “clock time.” Specifically, a CPU operates by transmitting information at regular time intervals. Based on complementary metal-oxide-semiconductor (“CMOS”) technology, silicon-based chips can be manufactured with more than 5 billion transistors per die, these transistors having features as small as 10nm. Advances in CMOS technology have been attributed to advancements in parallel computing, which is ubiquitous in personal computers and cell phones containing multiple processors.

[0006] However, as machine learning becomes commonplace in numerous applications, including bioinformatics, computer vision, video games, marketing, medical diagnostics, and online search engines, traditional CPUs often cannot provide sufficient processing power while maintaining low power consumption. Specifically, machine learning, a subfield of computer science, involves software with the ability to learn from and predict data. Furthermore, a branch of machine learning includes deep learning, which involves utilizing deep (multi-layered) neural networks.

[0007] Currently, research is underway to develop direct hardware implementations of deep neural networks, which could include systems attempting to simulate “silicon” neurons (e.g., “neuromorphic computing”). Neuromorphic chips (e.g., silicon computing chips designed for neuromorphic computing) operate by processing instructions in parallel using bursts of current transmitted at non-uniform intervals (e.g., in contrast to conventional sequential computers). As a result, neuromorphic chips require significantly less power to process information, specifically artificial intelligence (“AI”) algorithms. To achieve this, a neuromorphic chip can contain five times as many transistors as a conventional processor while consuming up to one-two-thousandth of the power. Therefore, the development of neuromorphic chips involves providing a chip with enormous processing power that consumes far less power compared to conventional processors. Furthermore, neuromorphic chips are designed to support dynamic learning in contexts of complex and unstructured data.

[0008] There is a continued need to develop and enhance dedicated processing capabilities, such as those found in keyword detectors for hands-free operation of mobile devices. This paper provides a system and method for enhancing the dedicated processing capabilities of a dedicated processor that can operate as a stand-alone processor utilizing a microcontroller interface. Summary of the Invention

[0009] This document discloses an integrated circuit for detecting keywords, comprising: a keyword identifier host processor that operates as a standalone host processor and is configured to identify one or more words in a received audio stream; a neuromorphic coprocessor including an artificial neural network configured to identify one or more desired keywords among the one or more words received from the host processor; and a communication interface between the host processor and the coprocessor configured to transmit information between them.

[0010] In a further embodiment, the neuromorphic coprocessor includes a database of known keywords, which can identify one or more desired keywords among one or more words.

[0011] In another embodiment, new keywords can be added to the database, and the new keywords can be distinguished from existing keywords.

[0012] In a further embodiment, the integrated circuit is configured to operate using battery power.

[0013] This document further discloses a method for detecting keywords within an audio stream, comprising: receiving an audio stream in the form of an electrical signal destined for a keyword identifier host processor; transmitting the electrical signal from the host processor to a neuromorphic coprocessor; identifying one or more desired keywords within the electrical signal; transmitting one or more desired keywords from the neuromorphic coprocessor to the host processor; and, upon receiving one or more desired keywords from the neuromorphic coprocessor, sending one or more output signals from the host processor.

[0014] In yet another embodiment, the method further includes: a neuromorphic coprocessor comprising a database of known keywords, thereby enabling the identification of one or more desired keywords within an electrical signal.

[0015] In more embodiments, the method further includes: one or more keywords consisting of predefined acoustic signals other than speech.

[0016] This document further discloses an integrated circuit comprising: a dedicated host processor capable of operating as a standalone host processor; a neuromorphic coprocessor including an artificial neural network configured to enhance the dedicated processing of the host processor; and a communication interface between the host processor and the coprocessor configured to transmit information between them.

[0017] In a further embodiment, the host processor is a keyword identifier processor configured to transmit an audio stream in the form of an electrical signal to a coprocessor via a communication interface, and the coprocessor is configured to enhance the host processor's dedicated processing by providing one or more detected keywords to the host processor via the communication interface.

[0018] In a further embodiment, the communication interface between the host processor and the coprocessor is a Serial Peripheral Interface (“SPI”) bus or an internal integrated circuit (“I”) bus. 2 C” bus.

[0019] In an additional further embodiment, the artificial neural network is disposed in an array of analog multipliers comprising multiple two-quadrant multipliers in the memory sector of the integrated circuit.

[0020] Further embodiments include storing the synaptic weights of the artificial neural network in the firmware of the integrated circuit, and configuring the firmware for cloud-based upgrades to update the synaptic weights of the artificial neural network.

[0021] Additional further embodiments include configuring the integrated circuit to operate using battery power.

[0022] This document further discloses an integrated circuit for detecting keywords, comprising: a dedicated host processor configured to identify one or more sounds within a received audio stream and transmit the one or more sounds to a neuromorphic coprocessor; an artificial neural network including a neuromorphic coprocessor configured to identify one or more desired sounds among the one or more sounds received from the host processor; and a communication interface between the host processor and the coprocessor configured to transmit information between them.

[0023] In a further embodiment, the integrated circuit is configured to remain in a low-power, always-on state, thereby keeping the integrated circuit constantly ready to receive audio streams.

[0024] In yet another embodiment, the host processor is configured to output one or more signals upon detecting one or more desired sounds.

[0025] In an additional embodiment, the integrated circuit is configured for implementation within a mobile device.

[0026] In more embodiments, the integrated circuit is configured to be powered by current leakage from the battery including the mobile device.

[0027] In several embodiments, the integrated circuit is configured to remain in a fully-aware state, thereby keeping the integrated circuit continuously ready to receive audio streams. Attached Figure Description

[0028] The accompanying drawings refer to embodiments of this disclosure, in which:

[0029] Figure 1 Schematic diagrams illustrating exemplary embodiments of systems for designing and updating neuromorphic integrated circuits (“ICs”) according to some embodiments are provided;

[0030] Figure 2 Schematic diagrams illustrating exemplary embodiments of analog multiplier arrays according to some embodiments are provided;

[0031] Figure 3 Schematic diagrams illustrating exemplary embodiments of analog multiplier arrays according to some embodiments are provided;

[0032] Figure 4 A schematic diagram illustrating an exemplary embodiment of a microcontroller interface between a coprocessor of a neuromorphic IC and a host processor of an application-specific IC, according to some embodiments, is provided.

[0033] Figure 5A schematic diagram illustrating an exemplary method for detecting spoken keywords using neuromorphic ICs according to some embodiments is provided; and

[0034] Figure 6 A block diagram illustrating components of an exemplary mobile device, including a keyword detector, is provided according to some embodiments.

[0035] While this disclosure is susceptible to various modifications and alternatives, specific embodiments thereof have been shown by way of example in the accompanying drawings and will be described in detail herein. The invention should be understood as not being limited to the specific forms disclosed, but rather, the invention is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure. Detailed Implementation

[0036] In the following description, certain terms are used to describe the features of the invention. For example, in some cases, the term "logic" may refer to hardware, firmware, and / or software configured to perform one or more functions. As hardware, logic may include circuitry with data processing or storage functions. Examples of such circuitry may include, but are not limited to, microprocessors, one or more processor cores, programmable gate arrays, microcontrollers, controllers, application-specific integrated circuits (ASICs), wireless receivers, transmitter and / or transceiver circuitry, semiconductor memories, or combinational logic.

[0037] The term "process" can include instances of computer programs (e.g., sets of instructions, also referred to herein as applications). In one embodiment, a process can include one or more threads that execute concurrently (e.g., each thread may execute the same or different instructions concurrently).

[0038] The term "processing" can include executing a binary file or script, or launching an application in which objects are processed, where launching should be interpreted as putting the application into an open state, and in some implementations, simulating typical actions of human interaction with the application.

[0039] The term "object" generally refers to a collection of data, whether in transit (e.g., via a network) or at rest (e.g., stored), which often has a logical structure or organization that allows it to be categorized or typed. In this document, the terms "binary file" and "binary" will be used interchangeably.

[0040] The term "file" is used broadly to refer to a set or collection of data, information, or other content used with a computer program. A file can be accessed, opened, stored, manipulated, or otherwise processed as a single entity, object, or unit. A file can contain other files and can contain related or unrelated content, or no content at all. A file can also have a logical format and / or be part of a file system with a logical structure or organization of multiple files. A file can have a name, sometimes simply referred to as a "filename," and often has attached properties or other metadata. Many types of files exist, such as data files, text files, program files, and directory files. Files can be generated by the user of the computing device or by the computing device itself. Access to and / or manipulation of files can be mediated by the operating system of the computing device and / or one or more applications. A file system can organize the files of a computing device on a storage device. A file system can enable the tracking of files and enable access to those files. A file system can also enable operations on files. In some embodiments, operations on files can include file creation, file modification, file opening, file reading, file writing, file closing, and file deletion.

[0041] The term "host processor" includes primary processors such as the CPU or digital signal processor (DSP) of an IC in a system. A host processor is an additional host processor that can operate independently but benefits from neuromorphic computing provided by a neuromorphic IC or its processor through a microcontroller interface.

[0042] The term "coprocessor" refers to an additional processor that interfaces with the host processor via a microcontroller interface. This additional processor can be configured to perform functions that are impossible to achieve using only the host processor, or functions that the coprocessor can perform faster or with lower power.

[0043] The term "enhancement filter" includes filters configured to suppress unwanted noise in a signal by selectively attenuating or amplifying certain components of the signal on a time-varying basis. Similarly, "enhancement filtering" includes filters used to suppress unwanted noise in a signal by selectively attenuating or amplifying certain components of the signal on a time-varying basis.

[0044] Finally, the terms “or” and “and / or” as used herein should be interpreted as inclusive or meaning either one or any combination thereof. Therefore, “A, B, or C” or “A, B, and / or C” means “any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition will only occur if the combination of elements, functions, steps, or actions is inherently mutually exclusive in some way.

[0045] Now for reference Figure 1 According to some embodiments, a schematic diagram of a system 100 for designing and updating neuromorphic ICs is provided. As shown, system 100 may include an emulator 110, a neuromorphic synthesizer 120, and a cloud 130 configured for designing and updating neuromorphic ICs (such as neuromorphic IC 102). As further shown, designing and updating neuromorphic ICs may include creating a machine learning architecture using emulator 110 based on a specific problem. As those skilled in the art will appreciate, cloud-based computer systems may include, but are not limited to, systems that can provide Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and / or Infrastructure as a Service (“IaaS”) resources. Neuromorphic synthesizer 120 may then transform the machine learning architecture into a netlist pointing to the electronic components of neuromorphic IC 102 and the nodes to which the electronic components are connected. Additionally, neuromorphic synthesizer 120 may transform the machine learning architecture into a Graph Database System (“GDS”) file that details the IC layout of neuromorphic IC 102. The neuromorphic IC 102 can be manufactured using current IC manufacturing technologies from its netlist and GDS file. Once manufactured, the neuromorphic IC 102 can be deployed to work on a specific problem, for which it is designed. While the initially manufactured neuromorphic IC 102 may include initial firmware with custom synaptic weights between nodes, this initial firmware can be updated by the cloud 130 as needed to adjust the weights. Since the cloud 130 is configured to update the firmware of the neuromorphic IC 102, it is not necessary for everyday use.

[0046] With accuracy matching or exceeding that of comparable software solutions, the energy efficiency of a neuromorphic IC (such as neuromorphic IC 102) can be up to 100 times greater than that of a graphics processing unit (“GPU”) solution, and up to 280 times greater than that of a digital CMOS solution. This makes such a neuromorphic IC suitable for battery-powered applications.

[0047] Neuromorphic ICs (such as neuromorphic IC 102) can be configured for use in dedicated standard products (“ASSPs”), including but not limited to keyword detection, speech recognition, voice tagging, one or more audio filters, gesture recognition, image recognition, video object classification and segmentation, or autonomous vehicles including drones. For example, if the specific problem is to detect one of the keywords, the simulator 110 can create a machine learning architecture regarding one or more aspects of spoken word tagging. The neuromorphic synthesizer 120 can then transform this machine learning architecture into a netlist and GDS file corresponding to a neuromorphic IC for word tagging, which can be manufactured according to current IC manufacturing techniques. Once the neuromorphic IC for word tagging is manufactured, it can be deployed to work on keyword spotting instances, such as in a mobile device, as further detailed herein.

[0048] Neuromorphic ICs (such as neuromorphic IC 102) can be deployed in toys, sensors, wearable devices, augmented reality (“AR”) systems or devices, virtual reality (“VR”) systems or devices, mobile systems or devices, electrical appliances, Internet of Things (“IoT”) devices, or auditory systems or devices.

[0049] Now for reference Figure 2 According to some embodiments, a schematic diagram of an analog multiplier array 200 is provided. This analog multiplier array can be based on a digital NOR flash memory array because the core of the analog multiplier array can be similar to, or the same as, the core of the digital NOR flash memory array. That is, at least the selection and readout circuitry of the analog multiplier array is different from that of the digital NOR array. For example, the output current is routed to the next layer as an analog signal, rather than being converted to bits via bit lines to a sense amplifier / comparator. Word lines are driven by analog input signals rather than by a digital address decoder. Furthermore, the analog multiplier array 200 can be used in a neuromorphic IC (such as neuromorphic IC 102). For example, a neural network can be incorporated into the analog multiplier array 200 within the memory sectors of the neuromorphic IC.

[0050] Because the analog multiplier array 200 is an analog circuit, the input and output current values ​​(or signal values) can vary over a continuous range, rather than simply being turned on or off. This is useful for storing weights or coefficients of a neural network relative to digital bits. In some embodiments, the weights are multiplied by the input current values ​​231, 232, 233, 234 in the core to provide output current values, which are combined to derive the neural network's decisions. Furthermore, the analog multiplier array 200 can utilize standard programming and erasing circuitry to generate tunneling and erasing voltages. In many embodiments, the input current values ​​231, 232, 233, 234 are provided by the input generator 230. Those skilled in the art will recognize that such input current values ​​can be generated or obtained from various devices or other components within the system. Additionally, the charge stored within the multiplier 210 can shift the voltage on the floating gate, according to the weights w i，j To scale the drain current.

[0051] Now for reference Figure 3 According to some embodiments, a schematic diagram of an analog multiplier array 300 is provided. The analog multiplier array 300 may use two transistors (e.g., a positive transistor and a negative transistor), such as, but not limited to, a metal-oxide-semiconductor field-effect transistor (“MOSFET”) or a field-effect transistor (“FET”), to perform a two-quadrant multiplication of signed weights (e.g., positive or negative weights) with a non-negative input current value. In many embodiments, the input current value may be provided by a separate input generator 330. In some embodiments, the input generation of 330 may be similar to... Figure 2 The input generation of the input generator is depicted in the diagram. If the input current value provided by the input generator 330 is multiplied by a positive or negative weight, the product or output current value can be positive or negative, respectively. The positively weighted product can be stored in the first column (e.g., corresponding to I in the analog multiplier array 300). Out0+ The negatively weighted product can be stored in the first column (e.g., the column corresponding to I in the analog multiplier array 300), and the negatively weighted product can be stored in the second column (e.g., the column corresponding to I in the analog multiplier array 300). Out0- In the column). As an example rather than a limitation, I can be... Out0+ and I Out0- The differential current output 310 is taken and then provided to multiple current sensing circuits, including but not limited to current mirrors, charge integrators, and / or transimpedance amplifiers. The aforementioned differential outputs 310 and 320 can provide positive and negative weighted products, or the output signal value can be taken as a differential current value to provide information useful for decision-making.

[0052] Because each output current from the positive or negative transistor is wired to ground and is proportional to the product of the input current value and the positive or negative weight, the power consumption of the positive or negative transistor is zero or close to zero when the input current value or weight is zero or close to zero. That is, if the input signal value is "0" or if the weight is "0", the corresponding transistor in the analog multiplier array 300 will not consume power. This is important because in many neural networks, most of these values ​​or weights are typically "0", especially after training. Therefore, energy is saved when there is nothing to do or continue. This differs from differential pair-based multipliers, which consume a constant current (e.g., by means of a tail bias current) regardless of the input signal.

[0053] Now for reference Figure 4 A schematic diagram is provided illustrating a microcontroller interface 400 between a coprocessor of a neuromorphic IC 402 and a host processor of a dedicated IC 404, according to some embodiments. Although the neuromorphic IC 402 and the dedicated IC 404 are... Figure 4 While these are shown as two distinct ICs, it should be understood that the aforementioned ICs can alternatively be embodied in a single monolithic IC. Therefore, the description of the microcontroller interface 400 between the coprocessor of the neuromorphic IC 402 and the host processor of the dedicated IC 404 should also be adopted to describe the microcontroller interface 400 between the neuromorphic coprocessor of the single monolithic IC and the dedicated host processor; that is, unless the context explicitly indicates otherwise.

[0054] like Figure 4 As shown, the dedicated IC 404 can be, but is not limited to, an IC for a speech recognition system or device including a keyword spotter. The IC for the keyword spotter may include a keyword identifier host processor 440 as a dedicated host processor, a microphone 410, a filter bank 420, a filter bank output channel 430, a post-processed word signal input 450, and a signal output 460. However, it should be understood that the neuromorphic coprocessor can interface with any one or more of the multiple dedicated host processors. Example embodiments of neuromorphic coprocessors interfaced with keyword identifier host processors are provided merely for illustrative purposes. It should be understood that extensions to other ASSPs may require certain modifications depending on the specific functionality of the other ASSP.

[0055] like Figure 4As further shown, the neuromorphic IC 402, or a portion of the aforementioned single monolithic IC corresponding to the neuromorphic IC 402, may include, but is not limited to: interface circuitry 470, a deep speech enhancement network 480 with cloud-updateable synaptic weights, and a database 490 of desired keywords. The neuromorphic IC 402 and the dedicated IC 404 can be configured to communicate via a bus such as a Serial Peripheral Interface (“SPI”) or an internal integrated circuit (“I”). 2 Communication is achieved via a digital inter-chip interface 405 such as a C” bus. In several embodiments, the synaptic weights of the deep speech enhancement network 480 may be updated periodically or in response to a manual update process. In further embodiments, the synaptic weights may be dynamically modified based on feedback from the user and / or the system.

[0056] The keyword identifier host processor can be configured to transmit frequency elements or signal spectrum information to a neuromorphic coprocessor via an SPI bus in the form of a Fourier transform or similar frequency decomposition for audio signal processing tasks (such as audio enhancement or denoising). The neuromorphic coprocessor can be configured to subsequently perform a word recognition task and transmit the results back to the keyword identifier host processor via the SPI bus. In some embodiments, the neuromorphic coprocessor may have access to a database including a list of keywords, whereby spoken keywords can be identified. In some embodiments, the results sent back to the keyword identifier host processor may include a weighted signal for the detected desired word. The keyword identifier host processor can be configured to subsequently transmit one or more signals indicating that a desired keyword has been detected. In some embodiments, these one or more signals may include signals sent to other dedicated ICs configured to perform specific tasks upon detection of the desired keyword.

[0057] It is contemplated that the dedicated IC 404 and the neuromorphic IC 402 can be implemented in a single system or system-on-a-chip (“SoC”) configuration. It is also contemplated that some embodiments may have the dedicated IC 404 and / or the neuromorphic IC 402 implemented remotely via a network connection using a digital interface 405.

[0058] Now for reference Figure 5A schematic diagram is provided illustrating a method 500 for detecting spoken keywords or other desired sounds using a neuromorphic IC according to certain embodiments. As shown, the method includes: 1) a first step 510, wherein a keyword identifier host processor calculates a frequency component signal; 2) a second step 520, wherein the keyword identifier host processor transmits the frequency component signal to a neuromorphic coprocessor; 3) a step 530, wherein the neuromorphic coprocessor processes the frequency component signal by generating an identified word signal from the frequency component signal to infer and identify a desired keyword; 4) a step 540, wherein the neuromorphic coprocessor transmits the identified keyword signal to the keyword identifier host processor; and 5) a step 550, wherein the neuromorphic coprocessor waits until the next time step.

[0059] It should be understood that method 500 is not limited to detecting speech, but rather method 500 can be used to detect any type of desired sound. For example, in some embodiments, method 500 can be implemented with security capabilities. In such an embodiment, the first step 510 may include: the host processor identifying undesirable sounds, such as unwanted intrusion into a restricted area. In step 530, the coprocessor may perform a sound recognition task, and then in step 540 transmit the identified sound to the host processor. Thus, upon detecting an undesirable sound, the host processor may output a signal to another IC, which may trigger an alarm.

[0060] Figure 6 This is a block diagram illustrating the components of an exemplary mobile device 600, which may include, as combined with... Figure 4 A disclosed keyword detector. In the illustrated embodiment, mobile device 600 includes one or more microphones 604, at least one processor 608, a keyword detector 612, a memory storage device 616, and one or more communication devices 620. In some embodiments, mobile device 600 may further include additional or other components necessary for the operation of mobile device 600. In some embodiments, mobile device 600 may include fewer components that perform functions similar to or equivalent to those described herein.

[0061] In some embodiments, once an acoustic signal is received (e.g., captured by one or more microphones 604), it can be converted into an electrical signal, which can then be converted into a digital signal by keyword detector 612 for processing, according to some embodiments. The processed signal can be transmitted to processor 608 for further processing. In some embodiments, some of the microphones 604 may be one or more digital microphones operable to capture acoustic signals and output digital signals. Some of the one or more digital microphones may provide voice activity detection or vocalization detection, and buffer audio data significantly prior to vocalization detection.

[0062] The keyword detector 612 may be operable to process acoustic signals. In some embodiments, the acoustic signals are captured by one or more microphones 604. In some embodiments, the acoustic signals detected by one or more microphones 604 may be used by the keyword detector 612 to separate desired speech (e.g., keywords) from ambient noise, thereby providing more robust automatic speech recognition (“ASR”).

[0063] In some embodiments, the keyword detector 612 may include a reference Figure 4 The neuromorphic IC 402 and the dedicated IC 404 are discussed. The keyword detector 612 can be configured to transmit one or more signals indicating the identification of a desired keyword or other sound. In some embodiments, these one or more signals can be sent to a processor 608 or other dedicated IC, which are configured to perform a specific task upon identification of a desired keyword or other sound. Therefore, the keyword detector 612 can be configured to provide hands-free operation of the mobile device 600.

[0064] In some embodiments, the keyword detector 612 can be configured to remain in a low-power, always-on state, thereby keeping the keyword detector 612 continuously ready to detect keywords or other sounds. As will be appreciated, a conventional DSP keyword finder is typically in a less-aware, low-power state until some event in the environment (such as speech) occurs, and then the entire DSP is powered up to full power. With the entire DSP powered up, a conventional DSP keyword finder is then typically placed in a fully conscious state, thereby enabling keyword detection. However, in the case of word detection, the time required to switch the DSP from low power to full power results in latency, where the keyword finder may miss one or more initial words. In some implementations, latency can lead to adverse results. For example, in instances of DSP detectors configured for security purposes, the time wasted due to latency could result in the loss of crucial evidence regarding a crime that has already been committed.

[0065] Unlike conventional DSP keyword searchers, keyword detector 612 can be configured to remain fully conscious when mobile device 600 is in a low-power state. By using the components described herein, keyword detector 612 may be able to detect changes in the environment (such as speech) without spending time powering on other components before detection can occur. Therefore, keyword detector 612 can be configured to detect keywords or other sounds with reduced and / or near-zero latency. It is contemplated that the full-conscious state of keyword detector 612 can be achieved by operating the keyword searcher within a power envelope that is negligible to the rest of the components including mobile device 600. For example, in some embodiments, keyword detector 612 can be implemented as a minimal component including mobile device 600 such that the keyword detector can be powered by current leakage from the battery. Thus, when components including mobile device 600 are in a low-power sleep mode, keyword detector 612 can continue to operate in a fully conscious state maintained by typical leakage current from the battery.

[0066] Processor 608 may include hardware and / or software operable to execute computer programs and / or logic stored in memory storage device 616. Processor 608 may use floating-point operations, complex operations, and other operations necessary to implement embodiments of this disclosure. In some embodiments, processor 608 of mobile device 600 may include at least one of, for example, a DSP, a graphics processor, an audio processor, a general-purpose processor, etc.

[0067] In various embodiments, the exemplary mobile device 600 may be operable to communicate, for example, via a communication device 620 through one or more wired or wireless communication networks. In some embodiments, the mobile device 600 may transmit at least audio signals (voice) via a wired or wireless communication network. In some embodiments, the mobile device 600 may encapsulate and / or encode at least one digital signal for transmission over a wireless network such as a cellular network.

[0068] Furthermore, it should be understood that the mobile device 600, and in particular the keyword detector 612, is not limited to detecting keywords for hands-free operation of the mobile device. For example, in some embodiments, the mobile device 600 can be configured for security-related implementations. Thus, the mobile device 600 can be configured to detect predefined unpleasant sounds, such as the sound of breaking glass, within a restricted area. Upon detecting an unpleasant sound, the keyword detector 612 can be configured to output one or more signals to other components indicating that an unpleasant sound has been detected. In some embodiments, for example, the keyword detector 612 can trigger an alarm system upon detecting an unpleasant sound.

[0069] Although the invention has been described with reference to specific variations and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the described variations or drawings. Furthermore, where the foregoing methods and steps indicate certain events occurring in a certain order, those skilled in the art will recognize that the order of certain steps can be modified, and such modifications are made according to variations of the invention. Additionally, where possible, certain steps can be performed simultaneously in parallel processes, and certain steps can be performed sequentially as described above. The patent is intended to cover as many variations of the invention as exist within the spirit of this disclosure or equivalent to the invention found in the claims. Therefore, this disclosure is to be understood as not being limited to the specific embodiments described herein, but only to the scope of the appended claims.

Claims

1. An integrated circuit for detecting keywords or sounds, comprising: The host processor is configured to receive a data stream and calculate the frequency component signal of the data stream; The coprocessor is configured to identify one or more desired keywords or sounds within the calculated frequency component signal; The coprocessor is configured to remain in a low-power, always-on state to be continuously ready to detect the one or more desired keywords or sounds. The coprocessor includes an enhanced network with cloud-updateable synaptic weights, and The interface between the host processor and the coprocessor is configured to transmit information between them.

2. The integrated circuit of claim 1, wherein the coprocessor is a neuromorphic processor.

3. The integrated circuit of claim 2, wherein the neuromorphic processor comprises an artificial neural network.

4. The integrated circuit of claim 1, wherein the coprocessor includes a database of known keywords, whereby the one or more desired keywords can be utilized by the coprocessor.

5. The integrated circuit of claim 4, wherein additional keywords are added to the database.

6. The integrated circuit of claim 1, wherein the integrated circuit is configured to operate relying on current leakage from the battery.

7. A method for detecting keywords within a data stream, comprising: The host processor receives the data stream in the form of electrical signals; Calculate the frequency component signal of the data stream; Transmit the calculated frequency component signal to the coprocessor; It remains in a low-power, always-on state to be continuously ready to detect one or more desired keywords; Identify one or more desired keywords within the transmitted frequency component signal; Provides enhanced networks with cloud-updateable synaptic weights; as well as Transmit one or more desired keywords to the host processor.

8. The method of claim 7, wherein the coprocessor includes a database for identifying known keywords of electrical signals corresponding to one or more keywords.

9. The method of claim 7, wherein identifying the one or more expected keywords is performed by an artificial neural network.

10. The method of claim 7, wherein the one or more keywords consist of a plurality of predefined acoustic signals other than speech.

11. An integrated circuit, comprising: Host processor; A coprocessor, comprising an artificial neural network configured to compute the frequency component signals of a data stream; One or more desired keywords are detected in the data stream, which is received by the host processor in the form of electrical signals. The coprocessor is configured to remain in a low-power, always-on state to be continuously ready to detect the one or more desired keywords. The coprocessor includes an enhanced network with cloud-updateable synaptic weights, and The interface between the host processor and the coprocessor is configured to transmit information between them.

12. The integrated circuit of claim 11, wherein the host processor is a pattern signal identifier processor configured to transmit the data stream in the form of electrical signals to the coprocessor at the interface, and wherein the coprocessor is configured to enhance the processing of the host processor by providing one or more detected patterns within the data stream to the host processor at the interface.

13. The integrated circuit of claim 11, wherein the interface between the host processor and the coprocessor is a serial peripheral interface bus or an internal integrated circuit bus.

14. The integrated circuit of claim 11, wherein the artificial neural network is disposed in an analog multiplier array of a plurality of two-quadrant multipliers in a memory sector of the integrated circuit.

15. The integrated circuit of claim 11, wherein the synaptic weights of the artificial neural network are stored in the firmware of the integrated circuit.

16. The integrated circuit of claim 15, wherein the firmware is configured to receive updates to the synaptic weights of the artificial neural network.

17. The integrated circuit of claim 11, wherein the integrated circuit is configured to operate using battery power.

18. The integrated circuit of claim 11, wherein the integrated circuit is configured to remain in a low-power, always-on state, thereby keeping the integrated circuit continuously ready to receive the data stream.

19. The integrated circuit of claim 11, wherein the host processor is configured to generate an output signal upon detection of the one or more desired keywords.

20. The integrated circuit of claim 11, wherein the integrated circuit is configured to remain in a fully conscious state to receive the data stream.

Images