Speech-to-text autoscaling for live use cases

Abstract

A method for automatic scaling of speech-to-text computing resources. The method includes: calculating a delta between the wall-clock time of each word in a transcript and the time the word is delivered to a client (202); submitting the delta to a metrics server group (204); requesting a current value of the delta from the metrics server group (206); determining whether the current value of the delta exceeds a predefined maximum latency threshold (208); and adjusting allocated computing resources based on the frequency of the current value of the delta exceeding the predefined maximum latency threshold (210). The method may also include creating a histogram from the current value of the delta and proportionally increasing the allocated computing resources based on the percentage of data points that fall above the predefined maximum latency threshold.

Description

Background Art

[0001] The present invention relates generally to the field of computing, and more particularly to a speech-to-text automatic scaling system.

[0002] Speech to text relates to the field of transcribing words into text form based on spoken language. Real use cases may include but are not limited to closed captioning, automatic proxy and dictation. In the case of these types, it is expected that the natural and smooth interaction between the user and the cloud-based speech to text service will be achieved. The live interaction between the user and the cloud-based speech to text service requires that the speech recognition engine delivers the recognition transcript quickly and efficiently. It is expected that the back-end server running in the remote data center has enough headroom, i.e., available computing resources, to adapt to the dynamic central processing unit (CPU) requirements of the speech recognition engine. Therefore, it is necessary to make the system in place to dynamically generate the metrics related to the user experience and the computing resources required at any given time, so as to allocate computing resources and retain time delay. Summary of the Invention

[0003] According to one embodiment, a method, computer system, and computer program product for automatic scaling of speech-to-text computing resources are provided. This embodiment may include calculating a delta for each word in a transcript. The calculated delta may be the time between the wall clock time of each word in the transcript and the time when the word is delivered to the client. This embodiment may also include submitting the delta to a metric server group, wherein the metric server is configured to collect the delta. This embodiment may also include requesting a current value of the delta from the metric server group. This embodiment may also include determining whether the current value of the delta exceeds a predefined maximum latency threshold. This embodiment may also include adjusting the allocated computing resources based on the frequency of the current value of the delta exceeding the predefined maximum latency threshold. This embodiment may also include creating a histogram from the current value of the delta. This embodiment may also include, in response to creating the histogram, performing a proportional increase (scaling-up) of the allocated computing resources based on the percentage of data points that fall above the predefined maximum latency threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] These and other objects, features, and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which will be read in conjunction with the accompanying drawings. The various features of the drawings are not to scale, as they are illustrated for the sake of clarity to facilitate understanding of the present invention by those skilled in the art in conjunction with the detailed description. In the drawings:

[0005] Figure 1 An exemplary networked computer environment is shown in accordance with at least one embodiment.

[0006] Figure 2 An operational flow diagram for allocating computing resources during speech-to-text autoscaling is shown, in accordance with at least one embodiment.

[0007] Figure 3 is a functional block diagram of a software build quality assessment process according to at least one embodiment.

[0008] Figure 4 A cloud computing environment according to an embodiment of the present invention is depicted.

[0009] Figure 5 Abstract model layers according to an embodiment of the present invention are depicted. DETAILED DESCRIPTION

[0010] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it is understood that the disclosed embodiments are merely illustrative of the claimed structures and methods that may be implemented in various forms. However, the present invention may be implemented in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Details of well-known features and techniques may be omitted from the description to avoid unnecessarily obscuring the presented embodiments.

[0011] Embodiments of the present invention relate to the field of computing, and more specifically to speech-to-text automatic scaling systems. The exemplary embodiments described below provide a system, method, and program product to utilize speech-to-text and natural language processing (NLP) to dynamically generate metrics related to user experience and the computing resources required at any given time, and thus allocate the computing resources necessary to keep latency within an acceptable range. It will be understood that "computing resources" and "backend servers" are used interchangeably herein. Therefore, the present embodiment has the ability to improve the technical field of speech-to-text automatic scaling by allowing natural and smooth interaction between users and cloud-based speech-to-text services.

[0012] As previously mentioned, speech-to-text involves transcribing spoken words into text form. Real-time guarantees, especially user-perceived latency, are important for the user experience in all speech-to-text use cases involving live interactions between users and cloud-based speech-to-text services. Real-world use cases may include, but are not limited to, closed captioning, automated proxying, and dictation. In these types of situations, the recognized transcript must be delivered by the speech recognition engine with almost no delay. However, between the audio source and the speech recognition engine running in a remote data center, several factors contribute to latency degradation, such as the lack of headroom in available computing resources on the back-end server to accommodate the dynamic central processing unit (CPU) requirements of the speech recognition engine. This problem is typically addressed by over-allocating computing resources to cover the worst-case scenario of an intractable audio stream. The amount of computing resources (typically CPU) required for the speech recognition engine to generate the transcript stream in real time with low latency varies greatly between the audio streams processed by the back-end server. Factors related to the amount of computing resources required may include, but are not limited to, speaker characteristics, style, crosstalk, quiet zones, background noise, business models, and collocated workloads, i.e., other applications or other instances competing for computing resources with the speech recognition engine running on the same host. Therefore, it may be advantageous to proactively take steps to address these factors by automatically scaling up or down the number of backend servers to match available computing resources to the traffic from users, thereby allocating computing resources based on need while keeping latency and costs within acceptable limits.

[0013] According to at least one embodiment, a delta for each word in a recognized transcript can be calculated. The calculated delta can be the time difference between the wall-clock time when the audio for the word is sent to the speech recognition engine and the time when the word is delivered to the client (i.e., the application layer). The delta can be submitted to a metric server group configured to collect the delta. The horizontal autoscaler group can periodically request the current value of the delta from the metric server. In at least one embodiment, if the current value of the delta exceeds a predefined maximum latency (max-latency) threshold, the horizontal autoscaler group can trigger a scale-up operation and add more computing resources to the speech recognition engine running in the remote data center. Computing resources can be added incrementally until the current value of the delta falls within an acceptable latency range. In at least one other embodiment, if the current value of the delta falls below a predefined minimum latency (min-latency) threshold, the horizontal autoscaler group can trigger a scale-down operation and reduce the computing resources allocated to the speech recognition engine running in the remote data center. Similarly, computing resources can be incrementally reduced until the current value of the delta falls within an acceptable latency range.

[0014] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, the computer-readable program instructions being used to cause a processor to perform various aspects of the present invention.

[0015] A computer-readable storage medium can be a tangible device that can retain and store instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted by a wire.

[0016] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the corresponding computing / processing device.

[0017] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages (including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages (such as "C" programming language or similar programming languages). The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider through the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute the computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions.

[0018] Aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the present invention. It will be understood that each block of the flowcharts and / or block diagrams and combinations of blocks in the flowcharts and / or block diagrams can be implemented by computer-readable program instructions.

[0019] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can direct the computer, programmable data processing device, and / or other equipment to operate in a specific manner, such that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0020] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0021] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, segment or portion of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the box may not occur in the order noted in the accompanying drawings. For example, two boxes shown in succession can actually be performed substantially simultaneously, or these boxes can sometimes be performed in reverse order, depending on the functions involved. It will also be noted that each box in the block diagram and / or flowchart illustration and the combination of boxes in the block diagram and / or flowchart illustration can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0022] The exemplary embodiments described below provide a system, method, and program product for determining in real time whether allocated computing resources are sufficient to keep latency within an acceptable range, or whether the allocated computing resources need to be adjusted.

[0023] refer to Figure 1 , according to at least one embodiment, an exemplary networked computer environment 100 is depicted. The networked computer environment 100 may include a client computing device 102 and a server 112 interconnected via a communication network 114. According to at least one implementation, the networked computer environment 100 may include multiple client computing devices 102 and servers 112, only each of which is shown for clarity of illustration.

[0024] The communication network 114 may include various types of communication networks, such as a wide area network (WAN), a local area network (LAN), a telecommunications network, a wireless network, a public switched network, and / or a satellite network. The communication network 114 may include connections such as wired, wireless communication links, or fiber optic cables. It is understood that Figure 1 This merely provides an illustration of one implementation and does not imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the described environment may be made based on design and implementation requirements.

[0025] According to one embodiment of the present invention, the client computing device 102 may include a processor 104 and a data storage device 106 capable of hosting and running a software program 108 and a speech-to-text auto-scaling program 110A, and communicating with a server 112 via a communication network 114. The client computing device 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop, a tablet computer, a desktop computer, or any other type of computing device capable of running programs and accessing a network. Figure 3As discussed, the client computing device 102 can include internal components 302a and external components 304a, respectively.

[0026] According to embodiments of the present invention, the server computer 112 may be a laptop computer, a netbook computer, a personal computer (PC), a desktop computer, or any programmable electronic device or any network of programmable electronic devices capable of hosting and running the speech-to-text auto-scaling program 110B and the database 116 and communicating with the client computing device 102 via the communication network 114. Figure 3 As discussed, the server computer 112 can include internal components 302b and external components 304b, respectively. The server 112 can also operate in a cloud computing service model, such as software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (IaaS). The server 112 can also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0027] According to the present embodiment, the speech-to-text auto-scaling program 110A, 110B may be a program capable of calculating the delta between the wall-clock time of each word in the transcript and the time when the word is delivered to the client (i.e., the application layer), submitting the delta to a group of metric servers configured to collect the delta, requesting the current value of the delta from the metric server, determining whether the current value of the delta exceeds a predefined threshold, and adjusting the allocation of computing resources based on the frequency of the current value of the delta exceeding the predefined threshold. Figure 2 The speech-to-text autoscaling method is explained in further detail.

[0028] Now refer to Figure 2 , depicts an operational flow diagram for allocating computing resources in a speech-to-text autoscaling process 200, according to at least one embodiment. At 202, the speech-to-text autoscaling program 110A, 110B calculates a delta for each word in the recognized transcript. The calculated delta may be the time difference between the wall clock time when the audio for the word is sent to the speech recognition engine and the time when the word is delivered to the client (i.e., the application layer). The audio may be considered to be sent to the speech recognition engine when the user speaks the word. Likewise, the audio may be considered to be delivered to the client when the word is transcribed. For example, if a word is spoken and timestamped at 1:00:00 PM, and the same word is transcribed and timestamped at 1:00:12 PM, the calculated delta may be determined to be 12 seconds. The calculated delta may be an end-to-end latency measurement that expresses the quality of service in terms of the latency experienced by the user for a particular audio session.

[0029] In at least one other embodiment, to obtain a global estimate of latency, a histogram can be created from the current values of the deltas. Thus, the scaling up or scaling down of computing resources can be based on the percentage of data points that fall above a predefined maximum latency threshold. For example, if 5% of the calculated deltas are above X seconds, the speech-to-text auto-scaling program 110A, 110B can scale up the number of backend servers by Y%. The predefined maximum latency threshold and the predefined minimum latency threshold can be changed based on user scenarios with different sensitivities to latency. For example, in an audio session in the presence of crosstalk and background noise, the predefined maximum latency threshold can be lowered to devote more computing resources to that particular session.

[0030] Then, at 204, the speech-to-text auto-scaling program 110A, 110B submits the current values of the deltas to a set of metric servers configured to collect the deltas. The current values of the deltas may be submitted via inter-process communication, where the speech-to-text auto-scaling program 110A, 110B sends a request and the set of metric servers responds to the request. For example, the speech-to-text auto-scaling program 110A, 110B may send the current values of the deltas to the metric servers, and the metric servers may accept the submission of these current values.

[0031] Next, at 206, the speech-to-text autoscaler 110A, 110B causes the horizontal autoscaler group to periodically request the current value of the increment from the metrics server. For example, the request may be made every 10 seconds. According to at least one other embodiment, the timing of the periodic request may vary. For example, the horizontal autoscaler group may make the request every 8 seconds or every 12 seconds. The horizontal autoscaler group may utilize a metrics application programming interface (API) to obtain the current value of the increment from the metrics server. The metrics API may be a software intermediary that sends requests and delivers responses, similar to the inter-process communication described above. For example, the horizontal autoscaler group may request the current value of the increment from the metrics server every 10 seconds, and the API may deliver the current value back to the horizontal autoscaler group.

[0032] In at least one other embodiment, the metrics server can be configured to send the current value of the delta to the horizontal autoscaler group without a request from the horizontal autoscaler group. The metrics server group can send the current value of the delta to the horizontal autoscaler group using the metrics API described above. For example, the metrics server can submit the current value of the delta to the horizontal autoscaler group, and the horizontal autoscaler group can accept the submission of the current value of the delta.

[0033] Then, at 208, the speech-to-text auto-scaling program 110A, 110B determines whether the current value of the increment exceeds a predefined maximum latency threshold. This determination can be made based on an analysis of how many increments exceed the predefined maximum latency threshold and how many increments fall below a predefined minimum latency threshold. If the speech-to-text auto-scaling program 110A, 110B determines that the current value of the increment exceeds the predefined maximum latency threshold, the speech-to-text auto-scaling process proceeds to step 210 to adjust the allocation of computing resources based on the frequency of the current value of the increment exceeding the predefined maximum latency threshold.

[0034] In at least one other embodiment, the speech-to-text auto-scaling program 110A, 110B determines whether the current value of the increment falls below a predefined minimum latency threshold at 209. If the speech-to-text auto-scaling program 110A, 110B determines that the current value of the increment falls below the predefined minimum latency threshold, the speech-to-text auto-scaling process proceeds to step 212 to adjust computing resource allocation based on the frequency of the current value of the increment falling below the predefined minimum latency threshold.

[0035] Next, at 210, the speech-to-text autoscaling programs 110A, 100B adjust the allocation of computing resources based on the frequency with which the current value of the increment exceeds a predefined maximum latency threshold. For example, if the predefined maximum latency threshold is exceeded more than Y times over a period of at least Z seconds, the horizontal autoscaler group can trigger a scale-up operation and add more computing resources to the speech recognition engine running in the remote data center. Before the current value of the increment is available, a default number of computing resources can be initially allocated to the speech recognition engine. For example, at least one computing resource can be allocated to the speech recognition engine at the beginning of an audio session. The horizontal autoscaler group can trigger the scale-up operation by sending a signal to a monitoring circuit within the backend server to direct the backend server. Computing resources can be added incrementally until the current value of the increment falls within an acceptable latency range.

[0036] In at least one other embodiment, a maximum amount of computing resources can be allocated to the speech recognition engine to prevent computing resources from being over-allocated to any one audio session. The speech-to-text auto-scaling program 110A, 110B can be configured to throttle new audio session requests from users in order to preserve the quality of service for already established audio sessions. For example, if 90% of the available computing resources are currently allocated to all combined audio sessions, the speech-to-text auto-scaling program 110A, 110B can throttle new audio session requests.

[0037] In at least one other embodiment, at 212, the speech-to-text autoscaling programs 110A, 110B adjust the allocation of computing resources based on the frequency with which the current value of the increment falls below a predefined minimum latency threshold. For example, if the predefined minimum latency threshold has not been exceeded more than Y times for at least Z seconds, the horizontal autoscaler group can trigger a scale-down operation and reduce the computing resources allocated to the speech recognition engine running in the remote data center. The horizontal autoscaler group can trigger the scale-down operation by sending a signal to monitoring circuitry within the backend server to power off the backend server. Similarly, computing resources can be incrementally reduced until the current value of the increment falls within an acceptable latency range.

[0038] In at least one other embodiment, a minimum amount of computing resources can be allocated to the speech recognition engine so as not to significantly impact the quality of service of the audio session. For example, at any given time, at least 10% of the available computing resources can be allocated to the speech recognition engine. Thus, even if the current value of the increment falls within an acceptable latency range, a significant amount of computing resources can be dedicated to each audio session to ensure quality of service.

[0039] I understand. Figure 2 This merely provides an illustration of one implementation and does not imply any limitation on how different embodiments may be implemented. Many modifications may be made to the described environment based on design and implementation requirements.

[0040] Figure 3 According to an embodiment of the present invention Figure 1 300 of the internal and external components of the client computing device 102 and the server 112 are depicted in FIG. It should be understood that Figure 3 This merely provides an illustration of one implementation and does not imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the described environment may be made based on design and implementation requirements.

[0041] Data processing systems 302 and 304 represent any electronic device capable of executing machine-readable program instructions. Data processing systems 302 and 304 may represent smartphones, computer systems, PDAs, or other electronic devices. Examples of computing systems, environments, and / or configurations that may be represented by data processing systems 302 and 304 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputer systems, and distributed cloud computing environments that include any of the above.

[0042] The client computing device 102 and the server 112 may include Figure 3, and a respective set of internal components 302a, b and external components 304a, b are shown in FIG. Each of the set of internal components 302 includes one or more processors 320, one or more computer-readable RAMs 322, and one or more computer-readable ROMs 324 on one or more buses 326, one or more operating systems 328, and one or more computer-readable tangible storage devices 330. The one or more operating systems 328, software programs 108, and speech-to-text auto-scaling program 110A in the client computing device 102 and the speech-to-text auto-scaling program 110B in the server 112 are stored on one or more respective computer-readable tangible storage devices 330 for execution by the one or more respective processors 320 via one or more respective RAMs 322 (typically including cache memory). Figure 3 In the embodiment shown in FIG, each of the computer-readable tangible storage devices 330 is a disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 330 is a semiconductor memory device such as ROM 324, EPROM, flash memory, or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0043] Each set of internal components 302a, b also includes a R / W drive or interface 332 to read from and write to one or more portable computer-readable tangible storage devices 338 (such as a CD-ROM, DVD, memory stick, tape, magnetic disk, optical disk, or semiconductor storage device). Software programs such as the speech-to-text auto-scaling program 110A, 110B can be stored on one or more corresponding portable computer-readable tangible storage devices 338, read via the corresponding R / W drive or interface 332, and loaded into the corresponding hard drive 330.

[0044] Each set of internal components 302a, b also includes a network adapter or interface 336, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G or 4G wireless interface card, or other wired or wireless communication link. The software program 108 and the speech-to-text automatic scaling program 110A in the client computing device 102 and the speech-to-text automatic scaling program 110B in the server 112 can be downloaded from an external computer to the client computing device 102 and the server 112 via a network (e.g., the Internet, a local area network, or other wide area network) and the corresponding network adapter or interface 336. From the network adapter or interface 336, the software program 108 and the speech-to-text automatic scaling program 110A in the client computing device 102 and the speech-to-text automatic scaling program 110B in the server 112 are loaded into the corresponding hard drive 330. The network can include copper wire, fiber optics, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.

[0045] Each of the sets of external components 304a, b may include a computer display monitor 344, a keyboard 342, and a computer mouse 334. External components 304a, b may also include a touch screen, a virtual keyboard, a touchpad, a pointing device, and other human-computer interface devices. Each of the sets of internal components 302a, b also includes a device driver 340 to interface with the computer display monitor 344, the keyboard 342, and the computer mouse 334. The device driver 340, the R / W driver or interface 332, and the network adapter or interface 336 comprise hardware and software (stored in the storage device 330 and / or ROM 324).

[0046] It is understood in advance that although the present disclosure includes detailed descriptions about cloud computing, the implementation of the teachings recorded herein is not limited to a cloud computing environment. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0047] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the service provider. The cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0048] Features are as follows:

[0049] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed without manual interaction with the service provider.

[0050] Wide Area Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0051] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. This is location-independent in the sense that consumers typically do not control or know the exact location of the provided resources, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0052] Rapid elasticity: In some cases, the ability to scale out quickly and in quickly can be provided quickly and elastically. To the consumer, the capacity available for provisioning often appears unlimited and can be purchased in any quantity at any time.

[0053] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the provider and consumer of the utilized service.

[0054] The service model is as follows:

[0055] Software as a Service (SaaS): The ability provided to consumers is to use the provider's applications running on cloud infrastructure. Applications are accessible from a variety of client devices through a thin-client interface such as a web browser (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0056] Platform as a Service (PaaS): The capability provided to consumers is to deploy consumer-created or acquired applications onto cloud infrastructure. These applications are built using programming languages and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but do have control over the deployed applications and possibly the configuration of the application hosting environment.

[0057] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but do have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0058] The deployment model is as follows:

[0059] Private cloud: The cloud infrastructure is operated solely for the organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0060] Community cloud: Cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0061] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by the organization that sells cloud services.

[0062] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a unique entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0063] The cloud computing environment is service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure consisting of a network of interconnected nodes.

[0064] Now refer to Figure 4 , depicts an illustrative cloud computing environment 40. As shown, the cloud computing environment 40 includes one or more cloud computing nodes 100 with which a local computing device used by a cloud consumer can communicate, such as a personal digital assistant (PDA) or cellular phone 44A, a desktop computer 44B, a laptop computer 44C, and / or an automobile computer system 44N. The nodes 100 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 40 to provide infrastructure, platform, and / or software as a service for which the cloud consumer does not need to maintain resources on a local computing device. It should be understood that Figure 4The types of computing devices 44A-N shown in FIGURE 4 are intended for illustration only, and computing node 100 and cloud computing environment 40 may communicate with any type of computerized device over any type of network and / or network-addressable connection (eg, using a web browser).

[0065] Now refer to Figure 5 , shows a set of functional abstraction layers 500 provided by the cloud computing environment 40. It should be understood in advance that Figure 5 The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0066] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host computers 61; servers based on RISC (Reduced Instruction Set Computer) architecture 62; servers 63; blade servers 64; storage devices 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0067] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0068] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides for dynamic procurement of computing and other resources for performing tasks within a cloud computing environment. Metering and pricing 82 provides for cost tracking when utilizing resources in a cloud computing environment, as well as billing or invoicing for the consumption of those resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. A user portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides for allocation and management of cloud computing resources so that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides for pre-scheduling and procurement of cloud computing resources, where future requirements are anticipated according to the SLAs.

[0069] The workload layer 90 provides examples of functionality that can utilize a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and automatic scaling of speech-to-text via natural language descriptions 96. Automatic scaling of speech-to-text via natural language descriptions 96 can involve dynamically generating metrics related to the user experience and the computing resources required at any given time, and allocating the computing resources necessary to keep latency within acceptable limits.

[0070] The description of various embodiments of the present invention has been provided for the purpose of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, practical applications, or improvements over existing technologies in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A computer-based method for allocating computing resources, the method comprising: calculating the delta between the wall clock time of each word in the transcript and the time at which the word was delivered to the client; submitting the delta to a metric server group configured to collect the delta; requesting a current value of the delta from the set of metric servers; determining whether the current value of the increment exceeds a predefined maximum latency threshold; as well as In response to determining that the current value of the increment exceeds the predefined maximum latency threshold, the allocated computing resources are adjusted based on a frequency of the current value of the increment exceeding the predefined maximum latency threshold.

2. The method according to claim 1, wherein The predefined maximum delay threshold is configured to change according to user situations with different sensitivities to delay.

3. The method according to claim 1, wherein The allocated computing resources are incrementally added until the current value of the increment falls within an acceptable latency range.

4. The method according to claim 1, wherein Adjusting the allocated computing resources is based on the frequency of the current value of the delta falling below a predefined minimum latency threshold.

5. The method according to claim 4, wherein The predefined minimum delay threshold is configured to change according to user situations with different sensitivities to delay.

6. The method according to claim 1, wherein The allocated computing resources are incrementally decreased until the current value of the increment falls within an acceptable latency range.

7. The method according to claim 1, wherein Calculating the delta between the wall clock time and the time at which the word was delivered to the client for each word in the transcript further comprises: creating a histogram from the current values of the increments; and The scaling up of the allocated computing resources is performed based on a percentage of data points falling above the predefined maximum latency threshold.

8. A computer system, comprising: One or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more tangible storage media, the program instructions for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising: calculating the delta between the wall clock time of each word in the transcript and the time at which the word was delivered to the client; submitting the delta to a metric server group configured to collect the delta; requesting a current value of the delta from the set of metric servers; determining whether the current value of the increment exceeds a predefined maximum latency threshold; and In response to determining that the current value of the increment exceeds the predefined maximum latency threshold, the allocated computing resources are adjusted based on a frequency of the current value of the increment exceeding the predefined maximum latency threshold.

9. The computer system according to claim 8, wherein: The predefined maximum delay threshold is configured to change according to user situations with different sensitivities to delay.

10. The computer system according to claim 8, wherein: The allocated computing resources are incrementally added until the current value of the increment falls within an acceptable latency range.

11. The computer system according to claim 8, wherein: Adjusting the allocated computing resources is based on the frequency of the current value of the delta falling below a predefined minimum latency threshold.

12. The computer system according to claim 11, wherein: The predefined minimum delay threshold is configured to change according to user situations with different sensitivities to delay.

13. The computer system according to claim 8, wherein: The allocated computing resources are incrementally decreased until the current value of the increment falls within an acceptable latency range.

14. The computer system according to claim 8, wherein: Calculating the delta between the wall clock time and the time at which the word was delivered to the client for each word in the transcript further comprises: creating a histogram from the current values of the increments; and The scaling up of the allocated computing resources is performed based on a percentage of data points falling above the predefined maximum latency threshold.

15. A computer program product, comprising: One or more computer-readable tangible storage media and program instructions stored on at least one of the one or more tangible storage media, the program instructions being executable by a processor capable of performing a method comprising: calculating the delta between the wall clock time of each word in the transcript and the time at which the word was delivered to the client; submitting the delta to a metric server group configured to collect the delta; requesting a current value of the delta from the set of metric servers; determining whether the current value of the increment exceeds a predefined maximum latency threshold; and In response to determining that the current value of the increment exceeds the predefined maximum latency threshold, the allocated computing resources are adjusted based on a frequency of the current value of the increment exceeding the predefined maximum latency threshold.

16. The computer program product of claim 15, wherein: The predefined maximum delay threshold is configured to change according to user situations with different sensitivities to delay.

17. The computer program product of claim 15, wherein: The allocated computing resources are incrementally added until the current value of the increment falls within an acceptable latency range.

18. The computer program product of claim 15, wherein: Adjusting the allocated computing resources is based on the frequency of the current value of the delta falling below a predefined minimum latency threshold.

19. The computer program product of claim 18, wherein: The predefined minimum delay threshold is configured to change according to user situations with different sensitivities to delay.

20. The computer program product of claim 15, wherein: The allocated computing resources are incrementally decreased until the current value of the increment falls within an acceptable latency range.

Images