Domain-adaptive speech recognition using artificial intelligence

Abstract

Provided herein are methods, systems, and computer program products for domain-adaptive speech recognition using artificial intelligence. The computer-implemented method includes generating a set of language data candidates by processing a sequence of phonemes associated with input speech data using an artificial intelligence-based data transformation model, each language data candidate including one or more graphemes; determining a subset of graphemes from the set of language data candidates for a target pair of phonemes and graphemes; generating a first speech recognition output by processing the subset of graphemes using at least one biased language model and an artificial intelligence-based speech recognition model; generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the graphemes from the target pair; and performing an automated action based on the second speech recognition output.

Description

[Technical Field]

[0001] This application relates generally to information technology, and more particularly to source language data processing. More specifically, speech recognition presents numerous problems. For example, conventional speech recognition techniques are commonly resource intensive and error prone. Summary of the Invention

[0002] In at least one embodiment, a technique for domain-adaptive speech recognition using artificial intelligence is provided. An exemplary computer-implemented method includes generating a set of language data candidates by processing a sequence of phonemes associated with input speech data using an artificial intelligence-based data transformation model, each language data candidate including one or more graphemes, and determining a subset of graphemes from the set of language data candidates for one or more target pairs of phonemes and one or more graphemes. The method also includes generating a first speech recognition output by processing at least a portion of the subset of graphemes using at least one biased language model and an artificial intelligence-based speech recognition model. The method further includes generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair, and performing one or more automated actions based at least in part on the second speech recognition output.

[0003] Another embodiment of the present invention, or elements thereof, may be implemented in the form of a computer program product tangibly embodying computer-readable instructions that, when executed, cause a computer to perform a plurality of method steps described herein. Furthermore, another embodiment of the present invention, or elements thereof, may be implemented in the form of a system including a memory and at least one processor coupled to the memory and configured to perform the described method steps. Still further, another embodiment of the present invention, or elements thereof, may be implemented in the form of means for performing the method steps or elements thereof described herein, which means may include hardware modules or a combination of hardware and software modules, with the software modules being stored on a tangible computer-readable storage medium (or multiple such media).

[0004] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 2 illustrates an exemplary speech recognition workflow, according to an exemplary embodiment of the present invention.

[0006] [Figure 2] FIG. 1 illustrates a neural network training procedure according to an exemplary embodiment of the present invention.

[0007] [Figure 3] FIG. 1 is a flow diagram illustrating a technique according to an exemplary embodiment of the present invention.

[0008] [Figure 4] FIG. 1 illustrates a computing environment in which at least one embodiment of the present invention may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0009] As described herein, at least one embodiment includes domain-adaptive speech recognition using artificial intelligence. One such embodiment includes training a first generator that receives a sequence of phonemes and outputs a plurality of word candidates, each candidate including one or more graphemes. By way of example, in one or more embodiments, the first generator may use and / or include at least a portion of a trained end-to-end automatic speech recognition (ASR) model.

[0010] In one or more embodiments, the generator includes at least one neural network model. As described in further detail below, in the exemplary embodiment shown in FIG. 1, the artificial intelligence-based data conversion model 104 (also referred to herein as the enhanced_sounds_like generator) includes a neural network model, and the prediction portion of the generator reuses the prediction portion of a recurrent neural network transducer (RNN-T) model (e.g., the RNN-T decoder 110). However, in one or more additional embodiments, reusing that portion of the RNN-T model is not required, and the generator may include a non-neural network model, such as, for example, a rule-based generator that can convert phonemes to graphemes.

[0011] Further, for a target pair of a sequence of phonemes and a word of one or more graphemes, at least one embodiment includes using a trained first generator to determine the top N words of one or more graphemes. The top N words may be output, for example, in order of probability. In one such embodiment, the generator reads the sounds_like element of the speech input and outputs the top N enhanced_sounds_like words. One or more embodiments also include training at least one biased language model (LM) using the determined top N words of one or more graphemes. In one such embodiment, an RNN-T model (e.g., a predictive network and speech encoder, such as that shown via element 110 in FIG. 1 ) has a non-separable LM that may be trained using a pair of speech data and text data. Furthermore, the biased LM may be trained with text data only.

[0012] Further, in one such embodiment, the trained biased LM may be used with the trained end-to-end ASR model to generate one or more output transcripts, and at least one of the top N words of one or more graphemes appearing in the one or more transcripts may be replaced with one or more grapheme words from the target pair. By way of example, for English, a user may set display_as to "IEEE" and sounds_like to "AY TR IH P AX L IY," and the generator will convert sounds_like to enhanced_sounds_like: ["I triple E," "eye triple iye"]. Further, in one such embodiment, a language model is trained using these enhanced_sounds_like entries, and the output transcript includes an enhanced_sounds_like such as "I joined I triple E." Also, because the end user is likely expecting "IEEE," word replacement may be performed to flip "I triple E" to "IEEE." In response, in such an example, the end user receives "I joined IEEE."

[0013] As described in further detail herein, in one or more embodiments, the trained end-to-end ASR model may include a trained RNN-T, and the first generator uses the prediction network of the trained RNN-T as its own prediction network. As noted above, in one such embodiment, separate language and auditory models may not be separated, but the prediction network is similar to the language model because, for example, as shown in FIG. 2, the prediction network at 210 may be configured to predict the preceding character (y u-1 ) and read the next character (y u ) Thus, in one such embodiment, the predictive network may act as a language model.

[0014] Additionally, at least one embodiment includes obtaining at least one sequence of phonemes as input for the generator from processing a word of graphemes.

[0015] FIG. 1 illustrates an exemplary speech recognition workflow associated with a domain-adaptive speech recognition system 126, in accordance with an exemplary embodiment of the present invention. Specifically, FIG. 1 illustrates an exemplary embodiment using an LM customization component 102 and a decoding component 114 to recognize the unknown word “IEEE” in English. An exemplary embodiment such as that illustrated in FIG. 1 includes using a converter component, referred to herein as an artificial intelligence-based data conversion model 104 (or enhanced_sounds_like generator), to generate a list of words (each containing one or more graphemes) from a given sequence of phonemes. As illustrated in FIG. 1, the artificial intelligence-based data conversion model 104 includes a phoneme encoder and a predictive network in common with the RNN-T decoder 110, which enables the artificial intelligence-based data conversion model 104 to create graphemes 106 that mimic the behavior of the RNN-T decoder 110.

[0016] 1 , speech data 116 from decoding component 114 is decoded by RNN-T decoder 110 in conjunction with biased LM 108 into a list of words representing initial speech recognition results 118. One or more words in initial speech recognition results 118 are also replaced with at least a portion of the original input speech data 103 processed by converter 112 to obtain final speech recognition results 120. In one or more embodiments, converter 112 converts enhanced_sounds_like output (e.g., graphemes 106) to display_as in the original input speech data 103.

[0017] One or more embodiments include using the phoneme and grapheme pairs to train an artificial intelligence-based data conversion model 104. In at least one embodiment, the artificial intelligence-based data conversion model 104 may include an encoder-decoder network. As also shown in FIG. 1 in connection with the LM customization component 102, a user adds original input speech data 103, including one or more pairs of display_as data (w) and sounds_like data (p). The trained model (i.e., the artificial intelligence-based data conversion model 104) selects up to the top N graphemes 106 from a larger set of graphemes based at least in part on the probability of generating a character sequence from the input phonemes (p).

number

number

[0018] 1, decoding the input speech data may include using an RNN-T decoder 110 in conjunction with a biased LM 108 to read and process the original input speech data 103 and output an initial speech recognition result 118 (e.g., one or more transcripts). One or more embodiments may also include using a RNN-T decoder 110 in conjunction with a biased LM 108 to read and process the original input speech data 103 and output an initial speech recognition result 118 (e.g., one or more transcripts).

number

[0019] 2 is a diagram illustrating a neural network training procedure according to an exemplary embodiment of the present invention. Specifically, FIG. 2 illustrates the neural network training procedure for one or more phonetic-character pairs (e.g., one or more y u-1 and x i...T pairs) to train the RNN-T decoder 210, and to train the artificial intelligence-based data transformation model 204, as described in more detail below. For example, in at least one embodiment, training the artificial intelligence-based data transformation model 204 includes initializing a prediction network of the artificial intelligence-based data transformation model 204 with the trained prediction network of the RNN-T decoder 210, freezing the prediction network of the artificial intelligence-based data transformation model 204 while training the artificial intelligence-based data transformation model 204, and u-1 and p i...T The neural network training includes training the artificial intelligence-based data transformation model 204 using a pair of neural network weights (such as a pair of neural networks). Training of the neural network is iterative, with each iteration including a forward process and a backward process (also referred to as backpropagation). The freezing phase disables the backward process in such training. To this end, the weights of the frozen portion of the neural network are not changed during training. Furthermore, in exemplary embodiments such as those shown in FIG. 2 (and FIG. 1), the RNN-T decoder 210 and the artificial intelligence-based data transformation model 204 use a common and / or shared prediction network.

[0020] In one or more embodiments, the speech recognition models detailed herein are not limited to RNN-Ts, but may include, for example, one or more types of end-to-end ASR models. Furthermore, in connection with biased LMs such as those detailed herein, the input is not limited to one or more individual words, but may also include one or more sentences and / or one or more compound words.

[0021] For illustrative purposes only, consider an example in which an end user uses English and provides original input speech data 103 with display_as: "IEEE" and sounds_like: "AY TR IH P AX L IY." The artificial intelligence-based data conversion model 104 converts sounds_like: "AY TR IH P AX L IY" into enhanced_sounds_like: ["I triple E", "eye triple iye"] (i.e., grapheme 106), and the biased LM 108 is implemented using the enhanced_sounds_like grapheme 106. Furthermore, a converter 112 from enhanced_sounds_like: ["I triple E", "eye triple iye"] to display_as: "IEEE" is provided. During the runtime (decoding) process, the RNN-T decoder 110 may output "I joined I triple E," and word replacement is performed using the converter 112 to convert "I triple E" to "IEEE." In that way, the end user receives "I joined IEEE." Accordingly, as can be seen through such an example, one or more embodiments include training the biased LM 108 with the original user input 103 of display_as and sounds_like and then inverting the output to the original word (display_as), instead of training the biased LM 108 with easy-to-learn graphemes 106.

[0022] Also, in at least one embodiment, the artificial intelligence-based data transformation model may include at least one RNN-T transformer model having a predictive network of an RNN-T speech recognition network, although such model type merely represents one example and other model types may be implemented. Furthermore, while the exemplary embodiment shown in FIG. 1 includes the use of English, one or more embodiments are not limited to English but rather may include the use of various other languages ​​(e.g., Japanese, etc.). Furthermore, in such embodiments, varying pronunciations (within a given language) of one or more words may be utilized in conjunction with the speech recognition models detailed herein.

[0023] 3 is a flow diagram illustrating a technique according to one embodiment of the present invention. Step 302 includes generating a set of language data candidates by processing a sequence of phonemes associated with input speech data using an artificial intelligence-based data conversion model, each language data candidate including one or more graphemes. In at least one embodiment, the artificial intelligence-based data conversion model includes at least one encoder-decoder neural network. Furthermore, in one or more embodiments, the artificial intelligence-based data conversion model implements a predictive network common to artificial intelligence-based speech recognition models.

[0024] One or more embodiments may also include training an artificial intelligence-based data conversion model using the one or more grapheme-phoneme pairs. Further, in at least one embodiment, the input speech data may include one or more of one or more individual words, one or more sentences, and one or more compound words.

[0025] Step 304 includes determining a subset of graphemes from the set of candidate language data for a target pair of one or more phonemes and one or more graphemes.

[0026] Stage 306 includes generating a first speech recognition output by processing at least a portion of the subset of graphemes using at least one biased language model and an artificial intelligence-based speech recognition model. In at least one embodiment, the artificial intelligence-based speech recognition model includes a recurrent neural network transducer. Additionally or alternatively, the artificial intelligence-based speech recognition model may include at least one end-to-end automatic speech recognition model.

[0027] One or more embodiments may also include training at least one biased language model using at least a portion of the subset of graphemes.

[0028] Step 308 includes generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair.

[0029] Stage 310 includes performing one or more automated actions based at least in part on the second speech recognition output. In at least one embodiment, performing the one or more automated actions includes automatically training at least one of an artificial intelligence-based data conversion model and an artificial intelligence-based speech recognition model using feedback data associated with the second speech recognition output. Additionally or alternatively, performing the one or more automated actions may include outputting the second speech recognition output to at least one user in response to the input speech data.

[0030] Additionally, in one or more embodiments, software implementing the techniques shown in FIG. 3 may be offered as a service in a cloud environment.

[0031] As used herein, a "model" should be understood to refer to a digitally stored electronic set of associated executable instructions and data values ​​that is capable of receiving and responding to a program or other digital call, invocation, or request for resolution based on specified input values ​​to yield one or more output values ​​that may serve as the basis for computer-implemented recommendations, output data displays, machine controls, etc. While those skilled in the art find it convenient to express models using mathematical formulas, that form of expression does not limit the models disclosed herein to abstract concepts; instead, each model herein has practical application in a computer in the form of stored executable instructions and data that implement the model using a computer.

[0032] The techniques shown in FIG. 3 also include providing a system, as described herein, including separate software modules, each embodied on a tangible computer-readable, recordable storage medium. All of the modules (or any subset thereof) may be on the same medium, or, for example, each may be on a different medium. The modules may include any or all of the components shown in the figure and / or described herein. In one embodiment of the present invention, the modules may be executed, for example, on a hardware processor. Method steps are then implemented using the separate software modules of the system executing on the hardware processor, as described above. Additionally, a computer program product may include a tangible computer-readable, recordable storage medium having code adapted to be executed to implement at least one method step described herein, including providing a system with separate software modules.

[0033] 3 may be implemented via a computer program product that may include computer-usable program code stored on a computer-readable storage medium at a data processing system, the computer-usable program code downloaded over a network from a remote data processing system. Also, in one embodiment of the present invention, the computer program product may include computer-usable program code stored on a computer-readable storage medium at a server data processing system, the computer-usable program code downloaded over a network to the remote data processing system for use on the computer-readable storage medium with the remote system.

[0034] An embodiment of the present invention or elements thereof may be implemented in the form of an apparatus including a memory and at least one processor coupled to the memory, and configured to perform the exemplary method steps.

[0035] Various aspects of the present disclosure are described through narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, two operations shown in successive flowchart blocks may be performed in the reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner, again depending on the technology involved.

[0036] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or pits / lands formed on a major surface of a disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated over wires, and / or other transmission media. As will be appreciated by those skilled in the art, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the above does not make a storage device transient because data is not transient while it is stored.

[0037] Computing environment 400 includes an example of an environment for the execution of at least some of the computer code involved in performing the methods of the present invention, such as domain-adaptive speech recognition code 426. In addition to code 426, computing environment 400 includes, for example, a computer 401, a wide area network (WAN) 402, an end user device (EUD) 403, a remote server 404, a public cloud 405, and a private cloud 406. In this embodiment, computer 401 includes a set of processors 410 (including processing circuitry 420 and cache 421), a communications fabric 411, volatile memory 412, persistent storage 413 (including an operating system 422 and the above-identified code 426), a set of peripheral devices 414 (including a set of user interface (UI) devices 423, storage 424, and a set of Internet of Things (IoT) sensors 425), and a network module 415. Remote server 404 includes a remote database 430. The public cloud 405 includes a gateway 440, a cloud orchestration module 441, a set of host physical machines 442, a set of virtual machines 443, and a set of containers 444.

[0038] Computer 401 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database such as remote database 430. As is well understood in the field of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. While in this presentation of computing environment 400, to keep the presentation as simple as possible, the detailed discussion focuses on a single computer, specifically computer 401. Computer 401 may be located in a cloud, although it is not depicted in FIG. 4 within the cloud. However, computer 401 is not required to be in the cloud except to the extent that it may be expressly indicated.

[0039] Processor set 410 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 420 may be distributed across multiple packages, e.g., multiple cooperating integrated circuit chips. Processing circuitry 420 may implement multiple processor threads and / or multiple processor cores. Cache 421 is located on the processor chip package and is typically memory used for data or code that should be available for quick access by threads or cores executing on processor set 410. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all caches for a processor set may be located “off-chip.” In some computing environments, processor set 410 may be designed to operate with qubits and perform quantum computing.

[0040] Computer-readable program instructions are typically loaded into computer 401 and cause a series of operational steps to be performed by processor set 410 of computer 401, thereby enabling a computer-implemented method, such that the instructions so executed instantiate the method specified in the computer-implemented method flowcharts and / or narrative descriptions contained herein (collectively, the "methods of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 421 and other storage media discussed below. The program instructions and associated data are accessed by processor set 410 to control and direct the execution of the methods of the present invention. In computing environment 400, at least some of the instructions for executing the methods of the present invention may be stored in code 426 in persistent storage 413.

[0041] Communications fabric 411 is the signal-conducting pathway that allows various components of computer 401 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as switches and conductive pathways that make up buses, bridges, physical input / output ports, etc. Other types of signal communication pathways may be used, such as fiber optic and / or wireless communication pathways.

[0042] Volatile memory 412 may be any type of volatile memory now known or later developed. Examples include dynamic type RAM or static type RAM. Typically, volatile memory 412 is characterized by random access, although this is not required unless expressly indicated. In computer 401, volatile memory 412 is located in a single package and is internal to computer 401, although alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 401.

[0043] Persistent storage 413 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data is maintained regardless of whether power is supplied to computer 401 and / or directly to persistent storage 413. While persistent storage 413 can be ROM, typically at least a portion of persistent storage allows data to be written, data to be erased, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 422 may take several forms, such as various known proprietary operating systems employing a kernel or open-source Portable Operating System Interface-type operating systems. The code included in code 426 typically includes at least some of the computer code involved in performing the methods of the present invention.

[0044] Peripheral device set 414 includes the set of peripheral devices of computer 401. Data communication connections between peripheral devices and other components of computer 401 may be implemented in various manners, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cable (such as a Universal Serial Bus (USB)-type cable), insertion-type connections (e.g., a Secure Digital (SD) card), connections made over a local area communication network, and even connections made over a wide area network such as the Internet. In various embodiments, UI device set 423 may include components such as a display screen, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage 424 may be external storage, such as an external hard drive, or insertable storage, such as an SD card. Storage 424 may be persistent and / or volatile. In some embodiments, storage 424 may take the form of a quantum computing storage device that stores data in the form of qubits. In embodiments where computer 401 is required to have a large amount of storage (e.g., computer 401 stores and manages a large database locally), this storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. IoT sensor set 425 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0045] Network module 415 is a collection of computer software, hardware, and firmware that enables computer 401 to communicate with other computers over WAN 402. Network module 415 may include hardware such as a modem or Wi-Fi signal transceiver, software that packetizes and / or depacketizes data for communication network transmission, and / or web browser software that communicates data over the Internet. In some embodiments, the network control and network forwarding functions of network module 415 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of network module 415 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for implementing the methods of the present invention may be downloaded to computer 401 from an external computer or external storage device, typically via a network adapter card or network interface included in network module 415.

[0046] WAN 402 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, now known or later developed. In some embodiments, WAN 402 may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0047] End-user device 403 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 401) and may take any of the forms discussed above in connection with computer 401. EUD 403 typically receives useful and useful data from the operation of computer 401. For example, in the hypothetical case where computer 401 is designed to provide recommendations to the end user, the recommendations would typically be communicated from computer 401's network module 415 over WAN 402 to EUD 403. In this manner, EUD 403 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 403 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, etc.

[0048] Remote server 404 is any computer system that provides at least some data and / or functionality to computer 401. Remote server 404 may be controlled and used by the same entity that operates computer 401. Remote server 404 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 401. For example, in the hypothetical case where computer 401 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 401 from remote database 430 of remote server 404.

[0049] Public cloud 405 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct, active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct, active management of computing resources in public cloud 405 is performed by computer hardware and / or software in cloud orchestration module 441. Computing resources provided by public cloud 405 are typically implemented by virtual computing environments running on various computers comprising host physical machine set 442, which is the universe of physical computers in and / or available to public cloud 405. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 443 and / or containers from container set 444. It is understood that these VCEs may be stored as images and transferred among and between various hosts of physical machines either as images or after instantiation of the VCE. Cloud orchestration module 441 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. Gateway 440 is a collection of computer software, hardware, and firmware that enables public cloud 405 to communicate over WAN 402.

[0050] Some further description of VCEs is provided here. A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of an operating system in which the kernel allows the existence of multiple isolated user space instances, called containers. These isolated user space instances typically behave as actual computers from the perspective of programs running within them. A computer program running on a typical operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and of the devices assigned to the container; this feature is known as containerization.

[0051] Private cloud 406 is similar to public cloud 405, except that the computing resources are available only for use by a single enterprise. While private cloud 406 is shown in communication with WAN 402, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both public cloud 405 and private cloud 406 are part of a larger hybrid cloud.

[0052] In computing environment 400, computer 401 is shown as connected to the Internet (see WAN 402). However, in one or more embodiments of the invention, computer 401 is isolated from communicating over a communications network, is not connected to the Internet, and operates as a stand-alone computer. In these embodiments, network module 415 of computer 401 may not be necessary, or even desirable, to ensure isolation and prevent external communications from reaching computer 401. Stand-alone computer embodiments, and in at least some applications of the invention, are potentially advantageous because they are typically more secure. In other embodiments, computer 401 is connected to a secure WAN or secure LAN instead of WAN 402 and / or the Internet. In these network-connected (i.e., non-stand-alone) embodiments, system designers may wish to take appropriate security measures, now known or to be developed in the future, to reduce the risk that incoming network communications will not cause a security breach.

[0053] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used herein, specify the presence of stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of other features, steps, operations, elements, components, and / or groups thereof.

[0054] The description of various embodiments of the present invention has been presented for purposes of illustration and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method comprising: generating a set of language data candidates by processing a sequence of phonemes associated with the input speech data using an artificial intelligence-based data transformation model, each language data candidate including one or more graphemes; determining a subset of graphemes from said set of candidate language data for a target pair of one or more phonemes and one or more graphemes; generating a first speech recognition output by processing at least a portion of the subset of graphemes using at least one biased language model and an artificial intelligence based speech recognition model; generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair; and performing one or more automated actions based at least in part on the second speech recognition output; Equipped with The method is performed by at least one computing device. Computer-implemented methods.

2. The computer-implemented method of claim 1 , wherein the artificial intelligence-based speech recognition model includes a recurrent neural network transducer.

3. The computer-implemented method of claim 1 , wherein the artificial intelligence-based data transformation model comprises at least one encoder-decoder neural network.

4. The computer-implemented method of claim 1 , wherein the artificial intelligence-based data transformation model implements a predictive network common to the artificial intelligence-based speech recognition model.

5. The computer-implemented method of claim 1 , wherein the artificial intelligence-based speech recognition model comprises at least one end-to-end automatic speech recognition model.

6. training said artificial intelligence-based data conversion model using one or more grapheme-phoneme pairs; The computer-implemented method of claim 1 further comprising:

7. training said at least one biased language model using at least a portion of said subset of graphemes; The computer-implemented method of claim 1 further comprising:

8. 2. The computer-implemented method of claim 1, wherein performing one or more automated actions includes automatically training at least one of the artificial intelligence-based data transformation model and the artificial intelligence-based speech recognition model using feedback data associated with the second speech recognition output.

9. 2. The computer-implemented method of claim 1, wherein performing one or more automated actions comprises outputting the second speech recognition output to at least one user in response to the input speech data.

10. The computer-implemented method of claim 1 , wherein the input speech data includes one or more of one or more individual words, one or more sentences, and one or more compound words.

11. The computer-implemented method of claim 1 , wherein software implementing the method is provided as a service in a cloud environment.

12. a computer-readable storage medium having program instructions embodied thereon, the program instructions causing a computing device to: generating a set of language data candidates by processing a sequence of phonemes associated with the input speech data using an artificial intelligence-based data transformation model, each language data candidate including one or more graphemes; determining a subset of graphemes from said set of candidate language data for a target pair of one or more phonemes and one or more graphemes; generating a first speech recognition output by processing at least a portion of the subset of graphemes using at least one biased language model and an artificial intelligence based speech recognition model; generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair; and performing one or more automated actions based at least in part on the second speech recognition output. a computer program product executable by said computing device to cause said computing device to perform

13. 13. The computer program product of claim 12, wherein the artificial intelligence based speech recognition model comprises a recurrent neural network transducer.

14. 13. The computer program product of claim 12, wherein the artificial intelligence-based data transformation model comprises at least one encoder-decoder neural network.

15. 13. The computer program product of claim 12, wherein the artificial intelligence-based data transformation model implements a predictive network common to the artificial intelligence-based speech recognition model.

16. 13. The computer program product of claim 12, wherein the artificial intelligence-based speech recognition model comprises at least one end-to-end automatic speech recognition model.

17. a memory configured to store program instructions; and operatively coupled to the memory: generating a set of language data candidates by processing a sequence of phonemes associated with the input speech data using an artificial intelligence-based data transformation model, each language data candidate including one or more graphemes; determining a subset of graphemes from the set of candidate language data for a target pair of one or more phonemes and one or more graphemes; generating a first speech recognition output by processing at least a portion of the subset of graphemes using at least one biased language model and an artificial intelligence based speech recognition model; generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair; and performing one or more automated actions based at least in part on the second speech recognition output. a processor that executes said program instructions for A system comprising:

18. 20. The system of claim 17, wherein the artificial intelligence based speech recognition model includes a recurrent neural network transducer.

19. 20. The system of claim 17, wherein the artificial intelligence-based data transformation model comprises at least one encoder-decoder neural network.

20. 20. The system of claim 17, wherein the artificial intelligence-based data transformation model implements a predictive network common to the artificial intelligence-based speech recognition model.

Images