Speech-to-text automatic scaling for live use cases

Abstract

An embodiment is provided for speech-to-text automatic scaling of computational resources. The embodiment may include calculating, for each word in a transcript, a difference between a wall-clock time and the time the word is delivered to a client. The embodiment may also include sending a plurality of the differences to a group of metrics servers. The embodiment may further include requesting current values ​​of the plurality of differences from the group of metrics servers. The embodiment may also include determining whether the current values ​​of the plurality of differences exceed a predefined maximum latency threshold. The embodiment may further include adjusting the allocated computational resources based on a frequency of the current values ​​of the differences exceeding the predefined maximum latency threshold. The embodiment may also include creating a histogram from the current values ​​of the plurality of differences and scaling up the allocated computational resources based on a percentage of data points exceeding the predefined maximum latency threshold.

Description

[Background technology]

[0001] The present invention relates generally to the field of computing, and more particularly to a speech-to-text auto-scaling system.

[0002] Speech-to-text relates to the field of transcribing spoken words into text format. Real use cases include, but are not limited to, closed-captioning, automated agents, and dictation. In these types of cases, a natural and smooth interaction between the user and the cloud-based speech-to-text service is desired. Live interaction between the user and the cloud-based speech-to-text service requires fast and efficient delivery of the recognition transcript by the speech recognition engine. Backend servers running in remote data centers preferably have sufficient headroom, i.e., available computational resources, to accommodate the dynamic central processing unit (CPU) requirements of the speech recognition engine. Therefore, to allocate computational resources and maintain latency, it may be essential to have a system that dynamically generates metrics that correlate with user experience and the computational resources needed at a given time. Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment, a method, computer system, and computer program product are provided for speech-to-text automatic scaling of computational resources. The embodiment may include calculating a difference for each word in a transcript. The calculated difference may be for each word in the transcript between a wall-clock time and the time the word is delivered to a client. The embodiment may also include sending a plurality of the differences to a group of metrics servers, where the group of metrics servers is configured to collect the plurality of differences. The embodiment may further include requesting current values ​​of the plurality of differences from the group of metrics servers. The embodiment may also include determining whether the current values ​​of the plurality of differences exceed a predefined maximum latency threshold. The embodiment may further include adjusting the allocated computational resources based on a frequency of the current values ​​of the differences exceeding the predefined maximum latency threshold. The embodiment may also include creating a histogram from the current values ​​of the plurality of differences. The embodiment may further include, in response to creating the histogram, scaling up the allocated computing resources based on a percentage of data points that exceed the predefined maximum latency threshold.

[0004] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings, in which various features of the drawings are not to scale for clarity in order to facilitate understanding of the present invention by those skilled in the art in conjunction with the detailed description of the invention. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 illustrates an exemplary networked computing environment according to at least one embodiment. [Figure 2]FIG. 2 illustrates an operational flowchart for allocating computational resources in a speech-to-text auto-scaling process in accordance with at least one embodiment. [Figure 3] FIG. 3 is a functional block diagram of the internal and external components of the computer and server depicted in FIG. 1, according to at least one embodiment. [Figure 4] FIG. 4 illustrates a cloud computing environment according to one embodiment of the present invention. [Figure 5] FIG. 5 illustrates abstraction model layers according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0006] Detailed embodiments of the claimed structures and methods are disclosed herein. However, it should be understood that the disclosed embodiments are merely exemplary of the claimed structures and methods, which may be embodied in various forms. However, the present invention may be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0007] FIELD OF THE INVENTION Embodiments of the present invention relate to the field of computing, and more particularly, to speech-to-text auto-scaling systems. The exemplary embodiments described below provide systems, methods, and program products that utilize, among other things, speech-to-text and natural language processing (NLP) to dynamically generate metrics correlating user experience and the computational resources required at any given time, and then allocate the necessary computational resources accordingly to maintain latency within acceptable limits. It should be understood that "computational resources" and "backend servers" are used interchangeably herein. Thus, the present embodiments have the potential to advance the field of speech-to-text auto-scaling by enabling natural and seamless interaction between users and cloud-based speech-to-text services.

[0008] As mentioned previously, speech-to-text relates to the field of transcribing spoken words into text format. Real-time guarantees, especially user-perceived latency, are central to the user experience in all speech-to-text use cases involving live interaction between the user and the cloud-based speech-to-text service. Practical use cases include, but are not limited to, closed captioning, automated agents, and dictation. In these types of cases, the recognized transcript must be delivered by the speech recognition engine with little delay. However, between the audio source and the speech recognition engine running in a remote data center, several factors contribute to poor latency, such as the backend server lacking available computing resources to accommodate the dynamic central processing unit (CPU) requirements of the speech recognition engine. This issue is typically addressed by over-allocating computing resources to cover worst-case scenarios where the audio stream is difficult to process. The amount of computing resources (typically CPU) required by the speech recognition engine to generate a real-time transcript stream with low latency varies significantly depending on the audio stream being processed by the backend server. Factors related to the amount of computing resources required may include, but are not limited to, speaker characteristics, style, crosstalk, silence regions, background noise, traffic patterns, and co-located workloads, i.e., other applications or other instances of the speech recognition engine running on the same host machine competing for computing resources. As such, it may be advantageous to proactively take measures to address these factors by, among other things, automatically scaling up or down the number of back-end servers to match available computing resources to incoming traffic from users. Thus, computing resources can be allocated based on need, and latency and costs are maintained within acceptable limits.

[0009] According to at least one embodiment, a delta for each word in a recognition transcript may be calculated. The calculated delta may be the time difference between the wall-clock time when the audio for the word is sent to the speech recognition engine and the time the word is delivered to the client, i.e., the application layer. The delta may be sent to a group of metrics servers configured to collect the delta. A group of horizontal auto-scalers may periodically request current values ​​of the delta from the metrics server. In at least one embodiment, if the current values ​​of multiple deltas exceed a predefined max-latency threshold, the group of horizontal auto-scalers may initiate a scale-up operation and add more computing resources to a speech recognition engine running in a remote data center. The computing resources may be added incrementally until the current values ​​of the deltas fall within an acceptable range for latency. In at least one other embodiment, if the current value of the difference falls below the predefined minimum latency (min-latency) threshold, the group of horizontal auto-scalers may trigger a scale-down operation and reduce the computing resources allocated to the speech recognition engine running in the remote data center. Similarly, the computing resources may be gradually reduced until the current value of the difference falls within an acceptable latency range.

[0010] The present invention may be a system, method, or computer program product, or combination thereof, at any level of technical detail that may be integrated. The computer program product may include one or more computer-readable storage media having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0011] The computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or a ridge structure in a groove in which instructions are recorded, or any suitable combination thereof. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted over an electrical wire.

[0012] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to an individual computing device / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may be comprised of copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing device / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions to the individual computing device / processing device for storage in a computer-readable storage medium.

[0013] The computer-readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, such as object-oriented programming languages, e.g., Smalltalk, C++, etc., or conventional procedural programming languages ​​(e.g., the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any kind of network, such as a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., over the Internet using an Internet Service Provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform aspects of the invention.

[0014] Aspects of the present invention are described herein with reference to flowchart illustrations or block diagrams, or combinations thereof, of methods, apparatus (systems), and computer program products or computer programs according to embodiments of the invention. It will be understood that each block of the flowchart illustrations or block diagrams, or combinations thereof, and combinations of blocks in the flowchart illustrations or block diagrams, or combinations thereof, can be implemented by computer-readable program instructions.

[0015] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts identified in one or more blocks of the flowchart diagrams or block diagrams, or a combination thereof, to produce a machine. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer-programmable data processing apparatus or other device, or a combination thereof, to function in a particular manner, such that a computer-readable storage medium having stored instructions includes an article of manufacture including instructions that implement aspects of the functions / acts identified in one or more blocks of the flowchart diagrams or block diagrams, or a combination thereof.

[0016] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable data processing apparatus, or other device, implement the functions / acts identified in one or more blocks of the flowchart diagrams or block diagrams, or a combination thereof, to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to generate a computer-implemented process.

[0017] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products or computer programs according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may actually be accomplished as a single step performed simultaneously, substantially simultaneously, partially, or fully in a time-overlapping manner, depending on the functionality involved, or the blocks may be performed in the reverse order. It should be noted that each block of the block diagrams or flowchart diagrams or combinations thereof, and combinations of multiple blocks in the block diagrams or flowchart diagrams or combinations thereof, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations, or may execute a combination of special-purpose hardware and computer instructions.

[0018] The exemplary embodiments described below provide systems, methods, and program products for determining in real time whether the allocated computing resources are sufficient to maintain latency within acceptable limits or whether the allocated computing resources require adjustment.

[0019] 1, according to at least one embodiment, an exemplary networked computing environment 100 is depicted. The networked computing environment 100 may include client computing devices 102 and servers 112 interconnected via a communications network 114. According to at least one implementation, the networked computing environment 100 may include multiple client computing devices 102 and servers 112, only one of which is shown for illustrative simplicity.

[0020] The communication network 114 may comprise various types of communication networks, such as a wide area network (WAN), a local area network (LAN), a telecommunications network, a wireless network, a public switched network, or a satellite network, or any combination thereof. The communication network 114 may comprise connections, such as wired communication links, wireless communication links, or fiber optic cables. It will be appreciated that Figure 1 provides only an illustration of one implementation and does not suggest any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0021] The client computing device 102 may include a processor 104 and a data storage device 106 capable of hosting and executing a software program 108 and a speech-to-text automatic scaling program 110A and communicating with a server 112 via a communications network 114, according to one embodiment of the present invention. The client computing device 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of executing programs and accessing a network. As will be described with reference to FIG. 3, the client computing device 102 may include internal components 302a and external components 304a, respectively.

[0022] The server computer 112 may be a laptop computer, a netbook computer, a personal computer (PC), a desktop computer, or any programmable electronic device or any network of programmable electronic devices capable of hosting and executing the speech-to-text auto-scaling program 110B and the database 116 and communicating with the client computing devices 102 via a communications network 114 in accordance with an embodiment of the present invention. As will be described with reference to FIG. 3 , the server computer 112 may include internal components 302b and external components 304b, respectively. The server 112 may also operate in a cloud computing service model, such as software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (IaaS). The server 112 may also be deployed within a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0023] According to this embodiment, speech-to-text auto-scaling programs 110A and 110B may be programs that can calculate, for each word in a transcript, the difference between wall-clock time and the time the word is delivered to a client, i.e., application layer, send a plurality of the differences to a group of metrics servers configured to collect the differences, request current values ​​of the plurality of differences from the metrics servers, determine whether the current values ​​of the plurality of differences exceed a predefined threshold, and adjust the allocation of the computational resources based on the frequency with which the current values ​​of the differences exceed the predefined threshold. The speech-to-text auto-scaling method is described in further detail below in connection with FIG. 2.

[0024] Referring now to FIG. 2, an operational flowchart for allocating computational resources in a speech-to-text autoscaling process 200 is depicted, according to at least one embodiment. In step 202, the speech-to-text autoscaling programs 110A and 110B calculate a delta for each word in the recognition transcript. The calculated delta may be the time difference between the wall-clock time when the audio of the word is sent to the speech recognition engine and the time when the word is delivered to the client. The audio may be considered to be sent to the speech recognition engine at the moment the word is spoken by the user. Similarly, the audio may be considered to be delivered to the client at the moment the word is transcribed. For example, if a word is spoken and time-stamped at 1:00:00 PM, and the same word is transcribed and time-stamped at 1:00:12 PM, the calculated delta may be determined to be 12 seconds. The calculated difference may be an end-to-end latency measurement that represents the quality of service in terms of latency experienced by a user for a particular audio session.

[0025] In at least one other embodiment, a histogram may be created from the current values ​​of multiple such differences to obtain a global estimate of latency. Accordingly, the scaling up or down of the computational resources may be based on the percentage of data points exceeding the predefined maximum latency threshold. For example, if 5% of the calculated differences are greater than X seconds, the speech-to-text autoscaling programs 110A and 110B may scale up the number of backend servers by Y%. The predefined maximum latency threshold and the predefined minimum latency threshold may be modified according to user cases with different sensitivities to latency. For example, in an audio session where crosstalk and background noise are present, the predefined maximum latency threshold may be lowered to allocate more computational resources to that particular session.

[0026] Next, in step 204, the speech-to-text autoscaling programs 110A and 110B send the current values ​​of the differences to a group of metrics servers configured to collect the differences. The current values ​​of the differences may be sent via inter-process communication, where the speech-to-text autoscaling programs 110A and 110B send a request and the group of metrics servers respond to the request. For example, the speech-to-text autoscaling programs 110A and 110B may send the current values ​​of the differences to the metrics server, and the metrics server may accept the transmission of these current values.

[0027] Next, in step 206, speech-to-text autoscaling programs 110A and 110B cause the group of horizontal autoscalers to periodically request the current values ​​of the deltas from the metrics server. For example, the request may be made every 10 seconds. According to at least one other embodiment, the timing of the periodic request may be changed. For example, the group of horizontal autoscalers may make a request every 8 seconds or every 12 seconds. The group of horizontal autoscalers may use a metrics application programming interface (API) to obtain the current values ​​of the deltas from the metrics server. The metrics API may be a software intermediary that sends requests and delivers responses, similar to the inter-process communication described above. For example, the group of horizontal autoscalers may request the current values ​​of the deltas from the metrics server every 10 seconds, and the API may send the current values ​​back to the group of horizontal autoscalers.

[0028] In at least one other embodiment, the metrics server may be configured to transmit the current values ​​of the plurality of deltas to the group of horizontal autoscalers without a request from the group of horizontal autoscalers. The group of metrics servers may transmit the current values ​​of the plurality of deltas to the group of horizontal autoscalers using the metrics API described above. For example, the metrics server may transmit the current values ​​of the plurality of deltas to the group of horizontal autoscalers, and the group of horizontal autoscalers may accept the transmission of the current values ​​of the plurality of deltas.

[0029] Next, in step 208, the speech-to-text autoscaling programs 110A and 110B determine whether the current value of the difference exceeds a predefined maximum latency (max-latency) threshold. This determination may be made based on an analysis of whether some differences exceed the predefined maximum latency threshold and some differences fall below the predefined minimum latency (min-latency) threshold. If the speech-to-text autoscaling programs 110A and 110B determine that the current value of the difference exceeds the predefined maximum latency threshold, the speech-to-text autoscaling process proceeds to step 210 and adjusts the allocation of computational resources based on the frequency with which the current value of the difference exceeds the predefined maximum latency threshold.

[0030] In at least one other embodiment, in step 209, speech-to-text autoscaling programs 110A and 110B determine whether the current value of the difference is below the predefined minimum latency threshold. If speech-to-text autoscaling programs 110A and 110B determine that the current value of the difference is below the predefined minimum latency threshold, the speech-to-text autoscaling process proceeds to step 212 and adjusts allocation of computational resources based on the frequency with which the current value of the difference is below the predefined minimum latency threshold.

[0031] Next, in step 210, speech-to-text auto-scaling programs 110A and 110B adjust their allocation of computational resources based on the frequency with which the current value of the difference exceeds the predefined maximum latency threshold. For example, of If the difference is exceeded more than Y times within at least Z seconds, the group of horizontal autoscalers may trigger a scale-up operation and add more computing resources to the speech recognition engine running in the remote data center. A default number of computing resources may be initially allocated to the speech recognition engine before the current value of the difference is available. For example, at the start of an audio session, at least one computing resource may be allocated to the speech recognition engine. The group of horizontal autoscalers may trigger a scale-up operation by sending a signal to a monitoring circuit within the backend server to wake up the backend server. The computing resources may be added incrementally until the current value of the difference is within an acceptable latency range.

[0032] In at least one other embodiment, a maximum number of computational resources may be allocated to a speech recognition engine to prevent over-allocation of computational resources to any one audio session. The speech-to-text autoscaling programs 110A and 110B may be configured to throttle new audio session requests from users to ensure quality of service for already established audio sessions. For example, if 90% of the available computational resources are currently allocated to all audio sessions combined, the speech-to-text autoscaling programs 110A and 110B may throttle new audio session requests.

[0033] In at least one other embodiment, in step 212, speech-to-text auto-scaling programs 110A and 110B adjust allocation of computational resources based on the frequency with which the current value of the difference is below the predefined minimum latency (min-latency) threshold. of If the difference is not exceeded more than Y times for at least Z seconds, the group of horizontal autoscalers may trigger a scale-down operation and reduce the computing resources allocated to the speech recognition engine running in the remote data center. The group of horizontal autoscalers may trigger a scale-down operation by sending a signal to a monitoring circuit within the backend server to power down the backend server. Similarly, the computing resources may be gradually reduced until the current value of the delta is within an acceptable range for latency.

[0034] In at least one other embodiment, a minimum number of computing resources may be allocated to the speech recognition engine so as not to significantly impact the quality of service of the audio session. For example, at least 10% of the available computing resources may be allocated to the speech recognition engine at any given time. Thus, even if the current value of the difference is within an acceptable latency range, a number of computing resources may be devoted to each audio session to ensure quality of service.

[0035] It will be appreciated that Figure 2 is only an illustration of one implementation and is not intended to imply any limitations on how different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

[0036] 3 is a block diagram 300 of internal and external components of the client computing device 102 and server 112 depicted in FIG. 1 in accordance with an embodiment of the present invention. It will be appreciated that FIG. 3 provides only an illustration of one implementation and does not suggest any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0037] Data processing systems 302 and 304 represent any electronic device capable of executing machine-readable program instructions. Data processing systems 302 and 304 may represent smartphones, computer systems, PDAs, or other electronic devices. Examples of computing systems, environments, or configurations, or combinations thereof, that may be represented by data processing systems 302 and 304 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputer systems, and distributed cloud computing environments comprising any of the above systems or devices.

[0038] The client computing device 102 and the server 112 may include respective sets of internal components 302a and 302b and external components 304a and 304b, as shown in Figure 3. Each of the sets of internal components 302 includes one or more processors 320 on one or more buses 326, one or more computer-readable RAMs 322, and one or more computer-readable ROMs 324, as well as one or more operating systems 328 and one or more computer-readable tangible storage devices 330. The one or more operating systems 328, software programs 108, and speech-to-text automatic scaling program 110A on the client computing device 102, and the speech-to-text automatic scaling program 110B on the server 112, are stored in one or more respective computer-readable tangible storage devices 330 for execution by one or more of the respective processors 320 via one or more of the respective RAMs 322 (which typically include cache memory). 3, each of the computer-readable tangible storage devices 330 is an internal hard drive magnetic disk storage device. Alternatively, each of the computer-readable tangible storage devices 330 is a semiconductor storage device, such as ROM 324, EPROM, flash memory, or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0039] Each set of internal components 302a and 302b also includes a R / W drive or interface 332 for reading from and writing to one or more portable computer-readable tangible storage devices 338, such as CD-ROMs, DVDs, memory sticks, magnetic tapes, magnetic disks, optical disks, or semiconductor storage devices. Software programs, such as speech-to-text automatic scaling programs 110A and 110B, can be stored on one or more of the respective portable computer-readable tangible storage devices 338, readable via the respective R / W drive or interface 332, and loaded into the respective hard drives 330.

[0040] Each set of internal components 302a and 302b also includes a network adapter or interface 336, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G or 4G wireless interface card, or other wired or wireless communication link. The software program 108 and speech-to-text autoscaling program 110A in the client computing device 102 and the speech-to-text autoscaling program 110B in the server 112 can be downloaded to the client computing device 102 and the server 112 from an external computer via a network (e.g., the Internet, a local area network, or other wide area network) and the respective network adapters or interfaces 336. From the network adapters or interfaces 336, the software program 108 and speech-to-text autoscaling program 110A in the client computing device 102 and the speech-to-text autoscaling program 110B in the server 112 are loaded into the respective hard drives 330. The network may include copper wire, fiber optics, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof.

[0041] Each of the sets of external components 304a and 304b may include a computer display monitor 344, a keyboard 342, and a computer mouse 334. The external components 304a and 304b may also include touch screens, virtual keyboards, touchpads, pointing devices, and other human interface devices. Each of the sets of internal components 302a and 302b also includes a device driver 340 for interfacing to the computer display monitor 344, the keyboard 342, and the computer mouse 334. The device driver 340, the R / W drive or interface 332, and the network adapter or interface 336 include hardware and software (stored in the storage device 330 or the ROM 324, or a combination thereof).

[0042] Although this disclosure includes detailed descriptions related to cloud computing, it should be understood in advance that implementation of the teachings recited herein is not limited to a cloud computing environment. Rather, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0043] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. The cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0044] The features are as follows:

[0045] On-demand self-service: Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, as needed, without requiring human interaction with the provider of the service.

[0046] Broad network access: Functionality is available over the network and accessed via standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0047] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, and various physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge of the exact location of the resources provided, but are said to be location-independent in that they may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center).

[0048] Rapid Elasticity: Capabilities can be provisioned quickly and elastically, sometimes automatically, scaled out quickly, released quickly, and scaled in quickly. To the consumer, the capabilities available for provisioning are often unlimited and can be purchased in any quantity at any time.

[0049] Measured Services: Cloud systems automatically control and optimize resource usage by using metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services being used.

[0050] The service model is as follows:

[0051] Software as a Service (SaaS): The ability to offer consumers the ability to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through a thin-client interface, such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings.

[0052] Platform as a Service (PaaS): The capability offered to consumers to deploy consumer-created or acquired applications, created using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure (e.g., including networks, servers, operating systems, or storage), but does have control over the deployed applications and, in some cases, the application-hosting environment configuration.

[0053] Infrastructure as a Service (IaaS): The capability offered to consumers to provision processing, storage, network, and other basic computing resources on which they can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating systems, storage, deployed applications, and in some cases, limited control over selecting network components (e.g., host firewalls).

[0054] The deployment models are as follows:

[0055] Private Cloud: Cloud infrastructure is operated exclusively for an organization. The cloud infrastructure may be managed by the organization or a third party, and may reside on-premises or off-premises.

[0056] Community Cloud: Cloud infrastructure is shared by several organizations and supports a specific community with common interests (e.g., mission, security requirements, policies, and compliance considerations). The cloud infrastructure may be managed by the organizations or a third party and may reside on-premises or off-premises.

[0057] Public Cloud: Cloud infrastructure is available to the general public or large industry groups and is owned by organizations that sell cloud services.

[0058] Hybrid Cloud: A cloud infrastructure is a blend of two or more clouds (private, community, or public) that remain unique entities but are brought together by standardized or proprietary technologies (e.g., cloud bursting for load balancing between clouds) that enable data and application portability.

[0059] A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure comprising a network of interconnected nodes.

[0060] Referring now to FIG. 4, an exemplary cloud computing environment 40 is illustrated. As illustrated, the cloud computing environment 40 includes one or more cloud computing nodes 100 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or mobile phone 44A, a desktop computer 44B, a laptop computer 44C, or an automobile computer system 44N, or any combination thereof, may communicate. The nodes 100 may communicate with each other. They may be physically or virtually grouped in one or more networks (not shown), such as a private cloud, community cloud, public cloud, or hybrid cloud, or any combination thereof, as described herein. This enables the cloud computing environment 40 to provide infrastructure, platform, or software, or any combination thereof, as a service without the cloud consumer having to maintain resources on their local computing device. It is understood that the types of computing devices 44A-N shown are intended to be illustrative only, and that computing node 100 and cloud computing environment 40 communicate with any type of computerized device (e.g., using a web browser) over any type of network or network-addressable connection or combination thereof.

[0061] Referring now to Figure 5, there is shown a set of functional abstraction layers 500 provided by cloud computing environment 40. It should be understood that the components, layers, and functions shown in Figure 5 are intended to be merely exemplary, and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0062] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include a mainframe 61, a RISC (Reduced Instruction Set Computer) architecture-based server 62, a server 63, a blade server 64, a storage device 65, and a network and networking component 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0063] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including, for example, virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0064] In one example, management layer 80 may provide several functions, as described below. Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks and protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides allocation and management of cloud computing resources so that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-provisioning and procurement of cloud computing resources where future requirements are predicted according to SLAs.

[0065] The workload tier 90 provides examples of functions for which the cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this tier include mapping and navigation 91; software development and lifecycle management 92; virtual classroom instruction delivery 93; data analytics processing 94; transaction processing 95; and natural language speech-to-text autoscaling 96. Natural language speech-to-text autoscaling 96 may involve dynamically generating metrics that correlate with user experience and the computational resources needed at any given time, and allocating the necessary computational resources to keep latency within acceptable bounds.

[0066] The description of various embodiments of the present disclosure has been presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification have been selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-based method comprising: calculating, for each word in the transcript, the difference between the time the audio corresponding to that word was sent to the speech recognition engine generating the transcript (hereinafter referred to as wall clock time) and the time the transcript of that audio corresponding to that word was delivered to the client; sending the plurality of said differences to a group of metrics servers, wherein the group of metrics servers is configured to collect the plurality of said differences; requesting current values ​​of a plurality of said differences from said one group of metrics servers; determining whether the current value of the plurality of differences exceeds a predefined maximum latency threshold; increasing computational resources allocated to the speech recognition engine in response to a determination that the current value of the difference exceeds the predefined maximum latency threshold and in response to the current value of the difference exceeding the maximum latency threshold more than Y times in Z seconds. The method comprising:

2. The method of claim 1 , wherein the predefined maximum latency threshold is configured to be varied according to user cases with different sensitivities to latency.

3. The method of claim 1 , wherein the allocated computing resources are added incrementally until the current value of the difference is within an acceptable range for latency.

4. determining whether the current value of the plurality of differences is below a predefined minimum latency threshold; and reducing computational resources allocated to the speech recognition engine in response to a determination that the current value of the difference is below a predefined minimum latency threshold and in response to a determination that the current value of the difference does not exceed the predefined minimum latency threshold more than Y times within Z seconds. The method of claim 1 further comprising:

5. The method of claim 4 , wherein the predefined minimum latency threshold is configured to be varied according to user cases with different sensitivities to latency.

6. The method of claim 1 , wherein the allocated computing resources are gradually reduced until the current value of the difference is within an acceptable range for latency.

7. Calculating the difference creating a histogram from the current values ​​of the differences; Including, The method comprises: determining the percentage of data points from the histogram that exceed the predefined maximum latency threshold; Further comprising: increasing the allocated computing resources when the percentage of the data points that exceed the maximum latency threshold is greater than a predetermined value. The method of claim 1.

8. 1. A computer system comprising: comprising one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and a plurality of program instructions stored on at least one computer-readable tangible storage medium of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories; wherein the computer system: calculating, for each word in the transcript, the difference between the time the audio corresponding to that word was sent to the speech recognition engine generating the transcript (hereinafter referred to as wall clock time) and the time the transcript of that audio corresponding to that word was delivered to the client; sending the plurality of said differences to a group of metrics servers, wherein the group of metrics servers is configured to collect the plurality of said differences; requesting current values ​​of a plurality of said differences from said one group of metrics servers; determining whether the current value of the plurality of differences exceeds a predefined maximum latency threshold; increasing computational resources allocated to the speech recognition engine in response to a determination that the current value of the difference exceeds the predefined maximum latency threshold and in response to the current value of the difference exceeding the maximum latency threshold more than Y times in Z seconds. The computer system is capable of executing a method comprising:

9. 9. The computer system of claim 8, wherein the predefined maximum latency threshold is configured to be varied according to user cases having different sensitivities to latency.

10. 9. The computer system of claim 8, wherein the allocated computing resources are added incrementally until the current value of the difference is within an acceptable range for latency.

11. determining whether the current value of the plurality of differences is below a predefined minimum latency threshold; and reducing computational resources allocated to the speech recognition engine in response to a determination that the current value of the difference is below a predefined minimum latency threshold and in response to a determination that the current value of the difference does not exceed the predefined minimum latency threshold more than Y times within Z seconds. The computer system of claim 8 further comprising:

12. The computer system of claim 11, wherein the predefined minimum latency threshold is configured to be varied according to user cases having different sensitivities to latency.

13. 9. The computer system of claim 8, wherein the allocated computing resources are gradually reduced until the current value of the difference is within an acceptable range for latency.

14. Calculating the difference creating a histogram from the current values ​​of the differences; Including, The method comprises: determining the percentage of data points from the histogram that exceed the predefined maximum latency threshold; Further comprising: increasing the allocated computing resources when the percentage of the data points that exceed the maximum latency threshold is greater than a predetermined value.

9. The computer system of claim 8.

15. A computer program comprising: calculating, for each word in the transcript, the difference between the time the audio corresponding to that word was sent to the speech recognition engine generating the transcript (hereinafter referred to as wall clock time) and the time the transcript of that audio corresponding to that word was delivered to the client; sending the plurality of said differences to a group of metrics servers, wherein the group of metrics servers is configured to collect the plurality of said differences; requesting current values ​​of a plurality of said differences from said one group of metrics servers; determining whether the current value of the plurality of differences exceeds a predefined maximum latency threshold; increasing computational resources allocated to the speech recognition engine in response to a determination that the current value of the difference exceeds the predefined maximum latency threshold and in response to the current value of the difference exceeding the maximum latency threshold more than Y times in Z seconds. The computer program causing a processor to execute each step of the method comprising:

16. The computer program product of claim 15 , wherein the predefined maximum latency threshold is configured to be varied according to user cases with different sensitivities to latency.

17. 16. The computer program product of claim 15, wherein the allocated computing resources are added incrementally until the current value of the difference is within an acceptable range for latency.

18. determining whether the current value of the plurality of differences is below a predefined minimum latency threshold; and reducing computational resources allocated to the speech recognition engine in response to a determination that the current value of the difference is below a predefined minimum latency threshold and in response to a determination that the current value of the difference does not exceed the predefined minimum latency threshold more than Y times within Z seconds.

16. The computer program of claim 15, further comprising:

19. 20. The computer program product of claim 18, wherein the predefined minimum latency threshold is configured to be varied according to user cases having different sensitivities to latency.

20. 16. The computer program product of claim 15, wherein the allocated computing resources are gradually reduced until the current value of the difference is within an acceptable range for latency.

Images