Adaptive frame batching for reducing speech recognition latency

Abstract

An embodiment may include collecting a first batch of acoustic feature frames of an audio signal, the number of acoustic feature frames of the first batch being equal to the first batch size, inputting the first batch into a speech recognition network, in response to detecting a word hypothesis output by the speech recognition network, collecting a second batch of acoustic feature frames of the audio signal, the number of acoustic feature frames of the second batch being equal to a second batch size that is larger than the first batch size, and inputting the second batch into the speech recognition network.

Description

Background Art

[0001] Neural network-based models are often used to perform automatic speech recognition (ASR). In some examples, a neural network-based acoustic model is trained to extract polyphone senone discriminant features from input audio frames and classify the polyphones based on the extracted features. The decoder generates word hypotheses based on the classification and outputs the corresponding text.

[0002] Input audio frames can be batched into batches of two or more frames to allow joint processing and recognition with the goal of improving accuracy and performance. However, batching requires the system to wait to receive all frames of the batch before submitting the batch to the ASR. Regardless of the processing speed of the ASR, this wait can result in undesirable user-perceived delays.

[0003] Traditional ASR systems can use several physically distinct models to meet different latency requirements within a given deployment. This approach multiplies the processor-related training and deployment costs. It is desirable for the system to improve latency without resorting to parallel models. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1A is a block diagram of a speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0005] Figure 1B is a block diagram of a speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0006] Figure 1C is a block diagram of a speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0007] Figure 1D is a block diagram of a speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0008] Figure 2 is a flow diagram of a process for initial delay-sensitive adaptive batching according to some embodiments.

[0009] Figure 3A is a block diagram of a speech recognition system that employs terminal delay-sensitive adaptive batching during operation according to some embodiments.

[0010] Figure 3B is a block diagram of a speech recognition system that employs terminal delay-sensitive adaptive batching during operation according to some embodiments.

[0011] FIG. 4A to FIG. 4CA flow chart of a process for initial and terminal delay-sensitive adaptive batching according to some embodiments is included.

[0012] Figure 5A is a block diagram of a look-ahead-enabled speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0013] Figure 5B is a block diagram of a look-ahead enabled speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0014] Figure 5C is a block diagram of a look-ahead enabled speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0015] Figure 5D is a block diagram of a look-ahead enabled speech recognition system that employs initial delay-sensitive adaptive batching during operation, according to some embodiments.

[0016] Fig. 6A is a block diagram of a look-ahead enabled speech recognition system employing initial delay-sensitive adaptive batching and adaptive look-ahead during operation according to some embodiments.

[0017] Figure 6B is a block diagram of a look-ahead enabled speech recognition system employing initial delay-sensitive adaptive batching and adaptive look-ahead during operation according to some embodiments.

[0018] Figure 6C is a block diagram of a look-ahead enabled speech recognition system employing initial delay-sensitive adaptive batching and adaptive look-ahead during operation according to some embodiments.

[0019] Fig.6D is a block diagram of a look-ahead enabled speech recognition system employing initial delay-sensitive adaptive batching and adaptive look-ahead during operation according to some embodiments.

[0020] Fig. 7A and Figure 7B A flow chart of a process for initial and delay-sensitive adaptive batching and adaptive look-ahead according to some embodiments is included.

[0021] Figure 8 is a block diagram of a cloud-based speech recognition service according to some embodiments.

[0022] Fig. 9 is a block diagram of a speech recognition system according to some embodiments. DETAILED DESCRIPTION

[0023] The following description is provided to enable anyone skilled in the art to make and use the described embodiments. However, various modifications will still be apparent to those skilled in the art.

[0024] According to some embodiments, the number of frames that are simultaneously input to the acoustic model (i.e., the batch size) is dynamically controlled during ASR processing. For example, a small batch size is used before generating word hypotheses. Using a small batch size can provide lower latency than a large batch size because the time required to collect a small batch is less than the time required to collect a large batch of frames of the same size. The batch size can then be increased after the word hypotheses are generated, which can improve processing efficiency because processing a large number of frames simultaneously is more CPU-efficient and cache-efficient than processing a smaller number of frames. Therefore, word hypotheses can be presented to the user quickly while retaining most of the processing advantages of a larger batch size.

[0025] Some embodiments may operate in conjunction with a model that employs look-ahead frames within each batch of input frames, where the number of look-ahead frames is related to the degree of overlap between successive batches of input frames. These embodiments may also dynamically control the batch size as described above to provide low initial user-perceived latency, and further provide improved processing efficiency after a first assumption.

[0026] Some models are operable to receive input batches of various sizes and include various numbers of lookahead frames. Embodiments using such models can provide low initial user-perceived latency and subsequently provide improved processing efficiency as described above by dynamically increasing the batch size in response to a first hypothesis. In addition, improved recognition accuracy can be achieved by increasing the number of lookahead frames in response to a first hypothesis.

[0027] In this regard, such an acoustic model may be trained in view of a specific number of look-ahead frames that is greater than the number of look-ahead frames initially used. Thus, while the initial operation is suboptimal, changing the number of look-ahead frames to the number of look-ahead frames upon which the acoustic model is trained may result in improved recognition accuracy. Embodiments may also allow the number of look-ahead frames to be changed while keeping the batch size constant.

[0028] Additionally or alternatively, some embodiments monitor input frames independently of hypothesis generation to detect an end-of-speech state. Upon detecting this state, the batch size may be reduced to avoid waiting for non-speech frames that would otherwise be added to the current batch of input frames. In response to detecting the end of a stream, a target maximum processing time, or other conditions, the batch size may be reduced additionally or alternatively. Such embodiments may therefore support early exit from the batch process and reduce delays in the generation of final hypotheses.

[0029] Figure 1A A speech recognition system 100 employing initial delay-sensitive adaptive batching during operation according to some embodiments is illustrated. The system 100 may be implemented using any suitable combination of hardware and software components. Each illustrated function of the system 100 or each function described herein may be implemented by one or more computing devices (e.g., computer servers), storage devices (e.g., hard disks or solid-state disk drives), and other hardware known in the art. These components may be located remotely from one another and may be elements of one or more cloud computing platforms, including but not limited to software as a service, platform as a service, and infrastructure as a service platforms. According to some embodiments, one or more components are implemented by one or more dedicated virtual machines executing one or more trained neural network models.

[0030] The frame generation unit 110 receives an audio signal and generates a frame-level amplitude spectrum or an original acoustic feature frame corresponding to a corresponding frame of the audio signal, as known in the art. For example, assuming that the audio frame size is 25 ms, the frame generation unit 110 generates a first original acoustic feature frame s based on the first 25 ms of the incoming audio signal. 0 .in this regard, Figure 1A The beginning of ASR based on an incoming audio signal is illustrated.

[0031] The next original acoustic feature frame may be generated based on the next 25 ms of the incoming audio signal. In some embodiments, temporally adjacent original acoustic feature frames may represent overlapping frames of the audio signal. For example, the first original acoustic feature frame s 0 The first acoustic feature frame may be generated based on the first 25 ms of the incoming audio signal, while the next original acoustic feature frame may be generated based on the 15 ms to 40 ms frame of the incoming audio signal, and the third original acoustic feature frame may be generated based on the 30 ms to 55 ms frame of the incoming audio signal.

[0032] According to some embodiments, the generated original acoustic feature frames are Mel-frequency cepstral coefficients or logarithmic filter bank features known in the art. The frame generation unit 110 can implement any suitable signal processing to determine the original acoustic feature frames corresponding to the corresponding frames of the incoming audio signal. For example, the frame generation unit 110 can implement a custom designed fast Fourier transform and custom filter bank parameters.

[0033] The batch processor 120 operates to collect the original acoustic feature frames into batches and provide the batches to the trained acoustic model 130. The trained acoustic model 130 may include any acoustic model capable of processing batches of input frames having various batch sizes. For example, the trained acoustic model 130 may include a unidirectional long short-term memory model (LSTM), a bidirectional LSTM (BLSTM) model, or a layer trajectory LSTM (ltLSTM), but the embodiment is not limited thereto.

[0034] According to some embodiments described below, the trained acoustic model 130 is also capable of processing batches of input frames including lookahead frames. The trained acoustic model 130 may support only a fixed number of lookahead frames (e.g., a context layer trajectory LSTM (cltLSTM) model) or may support batches including different numbers of lookahead frames (e.g., a delay-controlled BLSTM (LC-BLSTM) model).

[0035] Decoder 140 forms word hypotheses based on the output of trained acoustic model 130. According to some non-exhaustive examples, model 130 outputs polyphone posteriors in response to a received batch of input frames. Based on the posteriors and the hidden Markov model, decoder 140 generates word hypotheses based on the output of trained acoustic model 130. According to some non-exhaustive examples, model 130 outputs polyphone posteriors in response to a received batch of input frames. 0 ...s n Output the most likely word sequence

[0036] The batch processor 120 may determine a batch size and / or a number of lookahead frames based on which to collect the batches. As described herein, this determination may be based on word hypotheses generated by the decoder 140 and / or based on end-of-speech states detected in other ways. Thus, the batch processor 120 may implement functionality to determine a batch size (and, in some embodiments, a number of lookahead frames) based on received working hypotheses and / or end-of-speech signals.

[0037] Figure 1B 1 illustrates the continued operation of the system 200 according to the present example. As shown, the frame generation unit 110 has generated a frame s based on a second (potentially overlapping) frame of the input audio signal. 1 . Assume that the initial batch size is set to 2.

[0038] The batch processor 120 has previously received Figure 1A The frames shown in 0 Since the initial batch size is 2, the batch processor 120 divides the frame s 0 and 1The posterior is collected into batches 150, and batches 150 are provided to the trained acoustic model 130. The model 130 outputs corresponding posteriors based on its architecture and trained parameter values ​​known in the art. The posteriors are received by the decoder 140, which proceeds to attempt to form a hypothesis.

[0039] According to some embodiments, the above process is continued as shown with respect to subsequently received audio signal frames. Figure 1C The diagram shows the generation of frame s based on corresponding frames of the audio signal. 10 At this point, the batch processor 120 has already sent frame s 8 and 9 The frames are collected into batches 152 and passed to the model 130 .

[0040] The hypothesis decoder 140 now generates a non-silent, non-noise hypothesis based on the batch received up to the current point. As illustrated, the hypothesis is returned to the batch processor 120. Based on the generation of the hypothesis, the batch processor 120 determines to increase the batch size.

[0041] Changing the batch size based on the assumptions and / or the determination of the new batch size may be performed by a component other than the batch processor 120. Thus, such a component may communicate the new batch size to the batch processor 120 so that the batch processor 120 may cause subsequent batches to conform to the new batch size.

[0042] The determination of the new batch size may be triggered by the first generated word hypothesis, but the embodiment is not limited thereto. The determination may be triggered by the nth generated word hypothesis, by the generation of a particular subset (or subsets) of word hypotheses, or by any other function of the word hypotheses output by the decoder 140.

[0043] Figure 1D FIG. 4 illustrates the implementation of a new batch size according to some embodiments. Assume that the new batch size is 4, but the embodiments are not limited thereto. Therefore, Figure 1C After the output of batch 152 shown in FIG. , batch processor 120 converts frame s 10 to 14 are collected into batch 154, and batch 154 is provided to model 130. In contrast, if the batch size remains at 2, batch processor 120 will only batch frame s 10 and 11 The data are collected into a batch and the batch is provided to the model 130 .

[0044] The illustrated process may continue, with batches consisting of four frames, until ASR terminates (e.g., the end of the audio signal has been reached). According to some embodiments, the batch size is reduced to the initial batch size (e.g., two) when silence is detected based on the output of decoder 140, and increased again as described above in response to a word hypothesis.

[0045] In some embodiments, the batch size may be gradually increased to a target batch size. With respect to the above example, the batch processor 120 may increase the batch size from 2 to 3 over a specific time period, and may then increase the batch size from 3 to 4. The increase may be implemented based not on the elapsed time but on the generation of further hypotheses, or in addition to the elapsed time.

[0046] Figure 2 200 is a flow chart of a process 200 for providing dynamic batch processing according to some embodiments. Process 200 and other processes described herein may be performed using any suitable combination of hardware and software. The software program code embodying these processes may be stored by any non-transitory tangible medium, including a fixed disk, volatile or non-volatile random access memory, a DVD, a flash drive, or a tape, and executed by any number of processing units, including but not limited to processors, processor cores, and processor threads. Embodiments are not limited to the examples described herein.

[0047] Process 200 may be initiated in response to a request to perform ASR on an audio signal. Process 200 may be performed, for example, by a cloud-based ASR service that receives the request and the audio signal from a cloud-based conference provider. According to some embodiments, the audio signal may be received as a near real-time stream. In some embodiments, process 200 is performed continuously to process any signal presented on an incoming audio channel.

[0048] At S210, an initial batch size for processing the audio signal is determined. The initial batch size may be a predefined experimentally determined value. The initial batch size may be selected from a number of values ​​based on a desired initial delay, where a lower delay corresponds to a smaller batch size. Figures 1A to 1D In the example of , an initial batch size of two frames is determined at S210 .

[0049] At S220, raw acoustic feature frames of the audio signal are collected. The number of raw acoustic feature frames collected is equal to the determined initial batch size. Therefore, with respect to the above example, the process is paused at S220 while the batch processor 120 waits to receive frame s from the frame generation unit 110. 0 and 1 .

[0050] At S230, the collected frames are input as a batch into a speech recognition network. The speech recognition network may include any system capable of generating word hypotheses based on a batch of acoustic frames. According to some embodiments, Figures 1A to 1D The trained acoustic model 130 and decoder 140 comprise a speech recognition network.

[0051] Next, at S240, it is determined whether the speech recognition network has generated a word hypothesis. More specifically, S240 may include determining whether the speech recognition network has generated a non-silent, non-noise hypothesis based on the batches received up to the current point. If not, the process returns to S220 to collect another batch of frames that conforms to the initial batch size. Thus, the process loops between S220, S230, and S240 until the speech recognition network generates a word hypothesis. As described above, the determination at S240 is not limited to identifying the first generated word hypothesis.

[0052] Although illustrated as a linear process, the determination at S240 may occur independently of and in parallel with the process loop between S220 and S230. For example, the collection and input of frame batches may continue until interrupted by a separate determination of a generated word hypothesis, after which the process proceeds to S250.

[0053] At S250, a next batch size for processing the audio signal is determined. The next batch size may be predetermined based on experiments and / or selected from several possible batch sizes. According to some embodiments, the next batch size is eighty frames.

[0054] As about Figure 1D As described, the raw acoustic feature frames of the next batch size are collected at S260 and input to the speech recognition network at S270. According to some embodiments, the process loops between S260 and S270 to collect and input frame batches until silence is detected at S280. The silence detection at S280 may include detecting the absence of word hypotheses generated by the speech recognition network over a period of time, or and / or may be based on heuristics.

[0055] According to the illustrated example, detecting silence at S280 causes process 200 to return to S210 to reset the batch size to the initial batch size.Therefore, if subsequent frames of the audio signal include speech, the user-perceived delay in processing the speech may be lower than the delay caused by the larger batch size.

[0056] Switching from an initial batch size to a next batch size may introduce significant delays in processing. In response to the determination at S240, some embodiments may gradually increase the batch size to a target batch size. For example, for each n iterations of S260, the batch size may be increased by a predefined number until the target batch size is reached.

[0057] Process 200 may terminate when the end of the audio signal is reached. In some embodiments, process 200 terminates in response to detecting an end-of-speech state. Figure 3A and Figure 3B This process is illustrated in accordance with some embodiments.

[0058] System 300 adds a speech end detector 360 to the above components of system 100. The speech end detector 360 receives the original acoustic frame directly from the frame generation unit 310 and determines whether the speech has ended based on it. The speech end detector 360 can include any system for detecting the speech end state based on the input frame, and can include one or more trained neural networks. In one example, the detector 360 is an LSTM classifier formulated and trained independently of the decoder hypothesis, which predicts the end of the query based only on past frames. The detector 360 can operate in conjunction with an operating heuristic.

[0059] In one example of operation, components 310, 320, 330, and 340 have operated as described above to modify the initial batch size to a next batch size of four frames. Thus, batch processor 320 has collected frames s 10 to 13 The batches 350 are used as input to the model 330 . Figure 3A Also shown is the frame s 16 The frame generation unit 310 is thus already generated by the batch processor 320 and the detector 360. 14 and 15 , which have been collected by the batch processor 320 in anticipation of generating the next four-frame batch.

[0060] It will be assumed that the end-of-speech detector 360 responds to frame s 16 In response, speech end detector 360 notifies batch processor 320 of the detected state. Batch processor 320 then collects frame s 16 As the last frame of the current batch. Figure 3B The above operation is illustrated, wherein the batch processor 320 generates a batch file containing frames s 14 、s 15 and 16The batch 352 of the last batch is generated, and the batch 352 is input to the model 330. Therefore, the batch size of the last batch is reduced to avoid waiting for non-speech frames, which would otherwise be added to the input frames of the current batch. Such an embodiment can therefore support early exit from the batch processing process and reduce delays in the generation of the final hypothesis. As mentioned above, according to some embodiments, the last batch size can be reduced in response to other conditions, including but not limited to detecting an end-of-stream condition and detecting a maximum elapsed recognition time.

[0061] FIG. 4A to FIG. 4C A flow chart of a process 400 for providing dynamic batching according to some embodiments is included. The process 400 may be implemented by the system 300 of FIG. 3, but embodiments are not limited thereto.

[0062] Steps S405, S410, S415, S420, and S425 may be performed similarly to corresponding steps in steps S210, S220, S230, S240, and S250 of process 200. As described, an initial batch size for processing an incoming audio signal is determined at S405 and a corresponding batch of raw acoustic feature frames of the audio signal is collected at S410.

[0063] At S415, the collected batch of frames is input to the speech recognition network. Next, at S420, it is determined whether the speech recognition network has generated a suitable word hypothesis. If not, the process returns to S410 to collect another batch of frames that meets the initial batch size. If so, the next batch size for processing the audio signal is determined at S425 as described above.

[0064] Next, at S430, it is determined whether a speech end state has been detected based on the next original acoustic feature frame. S430 can be performed by a speech end detector that operates independently of the speech recognition network, such as Figure 3A and Figure 3B Described. Reference Figure 3A , S430 may include receiving a frame from the frame generation unit 310 and evaluating a speech end state based on the received frame (and, for example, based on previously received frames and / or other heuristics). If a speech end state is not detected, the process continues to S435 to collect the next original acoustic frame at S435. The frame may be collected at S435 to be included in the next batch of frames. In this regard, it is determined at S440 whether a batch that meets the next batch size has been collected. If not, the process returns to S430.

[0065] It will be assumed that the determination at S430 continues to be negative because the process loops between S430, S435 and S440 until the desired number of frames has been collected. The process then continues to S445 to input the collected current batch of frames into the speech recognition network. If no silence is detected at S450, the process returns to S430. Therefore, the process loops between S430 and S450 to collect and input the frame batches of the current batch size until silence is detected at S450 or an end-of-speech state is detected at S430. Detection of silence at S430 causes the process to return to S405 to reset the batch size to the initial batch size. As mentioned above, if the next received frame of the audio signal includes speech, the user-perceived delay of processing the speech can be lower than the delay caused by a larger batch size.

[0066] If a speech end state is detected at S430 during the collection of the next batch size, the flow proceeds from S430 to S455. At S455, the next original acoustic frame based on which the speech end state is detected is collected. Figure 3A and Figure 3B In the example, at S430 based on the frame s 16 The speech end state is detected, and then the batch processor 320 collects the frames s at S455. 16 As the last frame of the current batch. The collected frames of the current batch (s 14 、s 15 and 16 ) is then input to the model 330 at S460, even if the size of the current batch is smaller than the size of the next batch determined at S425.

[0067] The determination at S420 can be independent of the flow loop between S410 and S415 and occur in parallel therewith. Similarly, the speech end determination at S430 can also or alternatively be independent of the flow loop from S435 to S450 and occur in parallel therewith.

[0068] FIG. 5A to FIG. 5D A look-ahead enabled speech recognition system 500 employing initial delay-sensitive adaptive batching during operation according to some embodiments is illustrated. The system 500 includes a trained acoustic model 530 that supports "look-ahead" input frames as known in the art. The trained acoustic model 530 may include any model architecture that employs look-ahead frames that are known or become known, including but not limited to a delay-controlled bidirectional long short-term memory (LC-BLSTM) acoustic model and a context-layer trajectory long short-term memory (cltLSTM) acoustic model. As known in the art, the cltLSTM model may only support a fixed number of look-ahead frames, while the LC-BLSTM model may support batches including different numbers of look-ahead frames.

[0069] Figure 5A An example is shown in which the batch processor 520 collects a batch consisting of six raw acoustic feature frames, of which four frames (ie, frame s 2 、s 3 、s 4 and 5 ) is the lookahead frame. In some embodiments, the batch size is defined as a window including the start frame and the end frame, and the lookahead frame is defined by an overlap value. For example, batch 550 may be defined by a window {s 0 ,s 5} and an overlap value of 4 (frames).

[0070] Figure 5B The collection and input of the next batch 552 according to this example is illustrated. Batch 552 is the same as the last four frames (s 2 、s 3 、s 4 and 5 ) overlaps and includes the preceding frame s 4 、s 5 、s 6 and 7 Therefore, batch processor 520 must wait for frame s before generating batch 552. 6 and 7 .

[0071] As described above, the model 130 outputs a posteriori corresponding to the received batch based on its architecture and trained parameter values ​​known in the art. The posteriori are received by a decoder (not shown), which attempts to form word hypotheses based on the received posteriori.

[0072] It will now be assumed that the decoder is based on up to Figure 5B The batch processor 520 (or another component) generates a non-silent, non-noise hypothesis based on the batch received at the point illustrated in FIG. Based on the generation of the hypothesis, the batch processor 520 (or another component) determines to increase the batch size.

[0073] Figure 5C Implementation of a new batch size according to some embodiments is illustrated. Figure 5C In the example of , the number of preceding frames is fixed, but the embodiment is not limited thereto. Therefore, this example can illustrate the operation of using the cltLSTM model, which uses a fixed number of preceding frames.

[0074] like Figure 5C As shown in , the new batch size is eight frames. The next batch 554 thus includes eight frames and overlaps with the last four frames of the previous batch 552. The embodiment is not limited thereto. Batch 554 includes the preceding frames s 8 、s 9 、s 10 and11 Therefore, in Figure 1B After the output of the batch 552 shown in FIG. , the batch processor 520 collects the frames s 8 to 11 And add these frames to the already received frames s 4 to 7 to generate batch 554.

[0075] Figure 5D The operation based on the modified batch size is illustrated. The batch processor 520 has collected frames s 12 to 15 And add these frames to the already received frames s 8 to 11 to generate batch 556. Batch 556 has a batch size of eight frames and includes four leading frames.

[0076] As described above, in some embodiments, the batch size may be gradually increased to a target batch size. For example, these increases may be in response to the generation of further hypotheses and / or in response to the time elapsed since the first hypothesis.

[0077] FIG. 5A to FIG. 5D shows the operation of changing the batch size while keeping the number of lookahead frames fixed. In contrast, FIG. 6A to FIG. 6D A look-ahead enabled speech recognition system 600 employing initial delay sensitive adaptive look-ahead is illustrated.

[0078] The frame generation unit 610 receives an audio signal and generates a frame-level amplitude spectrum or an original acoustic feature frame corresponding to a corresponding frame of the audio signal. The batch processor 620 collects the generated original acoustic feature frames into batches with an initial specified batch size (or window) and a specified number of preceding frames (or overlaps). The trained acoustic model 630 receives the batch and outputs a polyphone posteriori, which is used, for example, by the decoder 640 to generate word hypotheses. The trained acoustic model 630 supports batches including different numbers of preceding frames, and therefore may include an LC-BLSTM model in some embodiments.

[0079] exist Fig. 6A In the example, the initial batch size is 3, and each batch includes two previous frames (i.e., batch 650 of frames s 1 and 2 ). Figure 6B The output of the next batch 652 is shown, which includes three frames and overlaps the two preceding frames of batch 650. It will be assumed that the decoder 640 is now generating a non-silence, non-noise hypothesis based on the batches received up to the current point.

[0080] In response and as Figure 6C and Fig.6D As shown in , the batch size becomes 4, and the number of leading frames per batch becomes 3. Therefore, the leading frames s of batch 654 and batch 652 are 2 and 3 The four frames of batch 654 include three preceding frames s 3 、s 4 and 5 Given the now changed batch size and the number of preceding frames, batch processor 620 collects the next batch 656 of four frames. The four frames of batch 656 include the same preceding frames as batch 654. 3 、s 4 and 5 Overlapping frames, and also including the preceding frame s 4 、s 5 and 6 .

[0081] Thus, embodiments may increase or decrease the batch size and / or the number of look-ahead frames per batch.Increasing or decreasing either parameter may occur once or gradually in response to a detected hypothesis.

[0082] According to some embodiments, the acoustic model 630 and the decoder 640 use Figure 6C and Fig.6D The acoustic model 630 and decoder 640 are trained with a batch size and number of lookahead frames, and therefore operate optimally after the first word hypothesis is detected. Therefore, the accuracy of the acoustic model 630 and decoder 640 may not be optimal when operating in conjunction with the initial batch size and the initial number of lookahead frames, such as Fig. 6A and Figure 6B as a trade-off for shorter user-perceived latency.

[0083] Fig. 7A and Figure 7B A flow chart of a process 700 for providing dynamic batching according to some embodiments is included. The process 700 may be implemented by the system 600, but embodiments are not limited thereto.

[0084] At S710, an initial batch size for processing the input audio signal is determined. Also determined at S710 is an initial number of lookahead frames to be included in each batch. The initial batch size may be specified as a window size, and the number of lookahead frames may be defined as a specific fraction (e.g., 1 / 2) of the window size.

[0085] At S720, original acoustic feature frames of the audio signal are collected based on the initial batch size and the initial number of preceding frames. Fig. 6AAs shown in the example of , and assuming that the initial batch size is three frames, the process is suspended at S720 while the batch processor 620 waits to receive frame s from the frame generation unit 610. 0 、s 1 and 2 .

[0086] At S730, the collected frames are input as a batch to a speech recognition network. The speech recognition network may include any system capable of generating word hypotheses based on a batch of acoustic frames including preceding frames.

[0087] Next, at S740, original acoustic feature frames of the audio signal are collected based on the initial batch size and the initial number of preceding frames. The frames collected at S740 overlap with the preceding frames of the previously collected frames. Figure 6B , a batch 652 may be collected at S740, consisting of three frames and having the same precedent frame s as the batch 650. 1 and 2 At S750, the frames collected at S740 are input as a batch into a speech recognition network.

[0088] The collection of frames that overlap with the preceding frames of the previously collected frames may include setting the start of the window of the current batch of frames to the start of the overlapping frames of the previous batch. For example, where the overlap is half the window size, the window of the current batch of frames is set to start from the middle of the window of the previous batch.

[0089] S760 includes determining whether a word hypothesis has been generated by the speech recognition network, and can be implemented as described above. If a word hypothesis has not been generated, the process returns to S740 to collect another batch of frames that conform to the initial batch size, including the previous frame and overlapping with the previous frame of the previous batch. Therefore, the process loops between S740, S750 and S760 until the speech recognition network generates a word hypothesis.

[0090] Once the determination at S760 is positive, then at S770, determine the next batch size for processing the audio signal. The determination of the next batch size may also include determining the next number of the lead frames. Therefore, S770 may include determining the new batch size and / or the new number of the lead frames of each batch. In certain embodiments, S770 includes determining the next window size, and the next lead size is determined as a function of the next window size. As mentioned above, only when performing process 700 in conjunction with a speech recognition network that can process batches including any number of lead frames, the new number of the lead frames may be determined.

[0091] As about Figure 6C and Fig.6DAs described, at S775, raw acoustic feature frames of the next batch size overlapping with the previous look-ahead frame are collected and input to the speech recognition network at S780. The process loops between S775 and S780 to collect and input frame batches until silence is detected at S785. The process then returns to S710 to reset the batch size and the number of look-ahead frames to their initial values ​​in order to reduce the user-perceived delay in processing the subsequently received speech.

[0092] As described with respect to process 400, embodiments of process 700 may further incorporate a speech end detector for independently detecting a speech end state. Based on this detection, the size of the final batch may be reduced (e.g., the window may be shortened) to include only until speech end frames.

[0093] A neural network (e.g., deep learning, deep convolution, or loop) according to some embodiments includes a series of "neurons" arranged into a network, such as LSTM nodes. A neuron is an architecture used in data processing and artificial intelligence (especially machine learning), which includes a memory that can determine when to "remember" and when to "forget" the values ​​stored in the memory based on the input weights provided to a given neuron. Each neuron used in this article is configured to accept a predefined number of inputs from other neurons in the network to provide relationship and sub-relationship outputs for the content of each frame being analyzed. Individual neurons can be linked together and / or organized into tree structures in various configurations of neural networks to provide interaction and relationship learning modeling for understanding how each frame in a discourse is related to each other.

[0094] For example, an LSTM used as a neuron includes several gates for processing input vectors, memory cells, and output vectors. The input gate and output gate control the information flowing into and out of the memory cell, respectively, while the forget gate can optionally remove information from the memory cell based on the input of the earlier link unit in the neural network. The weights and bias vectors for each gate are adjusted during the training phase, and once the training phase is completed, those weights and biases are finalized for normal operation. Neurons and neural networks can be constructed programmably (e.g., via software instructions) or via dedicated hardware that connects each neuron to form a neural network.

[0095] Figure 8 A distributed transcription system 800 is illustrated in accordance with some embodiments. The system 800 may be cloud-based, and its components may be implemented using on-demand virtual machines, virtual servers, and cloud storage instances.

[0096] The speech-to-text service 810 may be implemented as a cloud service that provides transcription of speech audio signals received via the cloud 820. The speech-to-text service 810 may include a batch processor component, an ASR model, and a decoder that operate as described above to provide ASR using delay-sensitive adaptive batching as described herein.

[0097] Each of the client devices 830 and 832 can be operated to request services such as a search service 840 and a voice assistant service 850. Services 840 and 850 can in turn request speech-to-text functionality from the speech-to-text service 810. According to some embodiments, the client device (e.g., smart speaker, smartphone) includes all components of a system such as system 100 or system 300, and operation does not require communication with a cloud service. In some embodiments, the client device includes at least one of a frame generation component, a batch processor component, an ASR model, a decoder, and an optional end-of-speech detector, and the client device calls a remote system (e.g., a cloud service) to access functionality of any of these components that are not implemented on the client device.

[0098] Fig. 9 is a block diagram of a system 900 according to some embodiments. The system 900 may include a general purpose computing device and may execute program code to provide ASR using delay-sensitive adaptive batching as described herein. According to some embodiments, the system 900 may be implemented by a standalone device such as a personal computer, a smart speaker, and a smartphone, or a cloud-based virtual server.

[0099] The system 900 includes a processing unit 910, which is operably coupled to a communication device 920, a persistent data storage system 930, one or more input devices 940, one or more output devices 950, and a memory 960. The processing unit 910 may include one or more processors, processing cores, etc., for executing program code. The communication interface 920 may facilitate communication with an external network such as the Internet. (Multiple) input devices 940 may include, for example, a keyboard, a keypad, a mouse or other pointing device, a microphone, a touch screen, and / or an eye tracking device. (Multiple) output devices 950 may include, for example, a display (e.g., a display screen), a speaker, and / or a printer.

[0100] Data storage system 930 may include any number of suitable persistent storage devices, including combinations of magnetic storage devices (e.g., magnetic tape, hard drives, and flash memory), optical storage devices, read-only memory (ROM) devices, etc. Memory 960 may include random access memory (RAM), storage class memory (SCM), or any other fast access memory.

[0101] The batch processor 932 and the decoder 936 may include program code executable to provide functionality as described herein. According to some embodiments, the batch processor 932 and / or the decoder 936 may include one or more trained neural networks. The node operation library 934 may include program code executable to provide an acoustic model defined by hyperparameters 938 and trained parameter values ​​939, as known in the art. The data storage device 930 may also store data and other program code necessary to provide additional functionality and / or operation of the system 900, such as device drivers, operating system files, etc.

[0102] As is known in the art, each functional component and process described herein may be implemented at least in part in computer hardware, in program code, and / or in one or more computing systems that execute such program code. Such a computing system may include one or more processing units that execute processor-executable program code stored in a memory system.

[0103] Processor-executable program code embodying the processes may be stored by any non-transitory tangible medium, including fixed disks, volatile or non-volatile random access memory, DVDs, flash drives, or tapes, and executed by any number of processing units, including but not limited to processors, processor cores, and processor threads. Embodiments are not limited to the examples described below.

[0104] The above-mentioned figures represent the logical architecture for describing the system according to some embodiments, and actual implementation may include more or different components arranged in other ways. Other topologies may be used in combination with other embodiments. In addition, each component or device described herein may be implemented by any number of devices communicating via any number of other public and / or dedicated networks. Two or more such computing devices may be positioned away from each other and may communicate with each other via (multiple) networks and / or dedicated connections of any known manner. Each component or device may include any number of hardware and / or software elements suitable for providing the functions described herein and any other functions. For example, any computing device used in the implementation of the system according to some embodiments may include a processor to execute program code so that the computing device operates as described herein.

[0105] The figures described herein do not imply a fixed order to the methods illustrated, and the embodiments may be practiced in any feasible order. In addition, any method described herein may be performed by hardware, software, or any combination of these methods. For example, a computer-readable storage medium may store instructions thereon that, when executed by a machine, result in the execution of a method according to any embodiment described herein.

[0106] Those skilled in the art will appreciate that various adaptations and modifications of the above-described embodiments may be configured without departing from the claims.It is therefore to be understood that the claims may be practiced in ways other than as specifically described herein.

Claims

1. A system comprising: Processing unit; as well as a storage device comprising program code which, when executed by the processing unit, causes the system to: determining a first batch size for processing the audio signal; collecting a first batch, the first batch comprising a first number of original acoustic feature frames of the audio signal, the first number being equal to the first batch size; Inputting the first batch into a speech recognition network; determining a second batch size based on word hypotheses output by the speech recognition network, the second batch size being larger than the first batch size; collecting a second batch, the second batch comprising a second number of acoustic feature frames of the audio signal, the second number being equal to the second batch size; as well as The second batch is input to the speech recognition network.

2. The system of claim 1 , wherein determining the second batch size based on the word hypothesis comprises: A first non-silent word hypothesis output by the speech recognition network is detected and the second batch size is determined in response to the detection.

3. The system of claim 1 , wherein the program code, when executed by the processing unit, causes the system to: collecting a plurality of batches, each batch of the plurality of batches comprising the first number of acoustic feature frames of the audio signal; The plurality of batches are input to the speech recognition network until the word hypotheses are output by the speech recognition network.

4. The system of claim 1 , wherein the program code, when executed by the processing unit, causes the system to: detecting an end-of-speech frame; and The size of the next collection batch of acoustic feature frames is reduced to use the speech end frame as the last frame of the next collection batch.

5. The system of claim 1 , wherein the first batch size comprises a first number of lookahead frames, and the first batch comprises a first lookahead frame, the number of the first lookahead frames of the first batch being equal to the first number of lookahead frames, the program code when executed by the processing unit causes the system to: collecting a third batch, the third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first preceding frame of the first batch, Wherein determining the second batch size comprises: determining a second number of preceding frames, and The second batch includes second preceding frames, and the number of the second preceding frames in the second batch is equal to the second number of the preceding frames.

6. The system of claim 5, wherein the speech recognition network comprises a delay-controlled bidirectional long short-term memory model.

7. The system of claim 1 , wherein the first batch size comprises a first number of lookahead frames, and the first batch comprises a first lookahead frame, the number of the first lookahead frames of the first batch being equal to the first number of lookahead frames, the program code when executed by the processing unit causes the system to: collecting a third batch, the third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first preceding frame of the first batch, The second batch includes second preceding frames, and the number of the second preceding frames in the second batch is equal to the first number of the preceding frames.

8. The system of claim 7, wherein the speech recognition network comprises a contextual layer trajectory long short-term memory model.

9. A computer-implemented method comprising: Collecting a first batch of acoustic feature frames of the audio signal, the number of the acoustic feature frames in the first batch being equal to the first batch size; Inputting the first batch into a speech recognition network; In response to detecting a word hypothesis output by the speech recognition network, collecting a second batch of acoustic feature frames of the audio signal, the number of the acoustic feature frames in the second batch equal to a second batch size, the second batch size being larger than the first batch size; and The second batch is input to the speech recognition network.

10. The method of claim 9, wherein detecting the word hypothesis comprises: A first non-silent word hypothesis output by the speech recognition network is detected.

11. The method according to claim 9, further comprising: Detect speech end frame; In response to detecting the speech end frame, collecting a third batch of acoustic feature frames of the audio signal, the third batch using the speech end frame as the last frame of the third batch; and The third batch is input to the speech recognition network.

12. The method of claim 9, wherein the first batch size comprises a first number of preceding frames, and the first batch comprises a first preceding frame, the number of the first preceding frames of the first batch being equal to the first number of preceding frames, the method further comprising: collecting a third batch, the third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first preceding frame of the first batch, The second batch includes second preceding frames, and the number of the second preceding frames in the second batch is equal to the second number of the preceding frames.

13. The method of claim 12, wherein the speech recognition network comprises a delay-controlled bidirectional long short-term memory model.

14. The method of claim 9, wherein the first batch size comprises a first number of preceding frames, and the first batch comprises a first preceding frame, the number of the first preceding frames of the first batch being equal to the first number of preceding frames, the method further comprising: collecting a third batch, the third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first preceding frame of the first batch, The second batch includes second preceding frames, and the number of the second preceding frames in the second batch is equal to the first number of the preceding frames.

15. The method of claim 14, wherein the speech recognition network comprises a contextual layer trajectory long short-term memory model.

16. A non-transitory computer readable medium storing program code to be executed by a processing unit to provide: Speech recognition network; Batch processors to: Collecting a first batch of acoustic feature frames of the audio signal, the number of the first batch of acoustic feature frames being equal to the first batch size; inputting the first batch into the speech recognition network; In response to detecting a word hypothesis output by the speech recognition network, collecting a second batch of acoustic feature frames of the audio signal, the number of the second batch of acoustic feature frames equal to a second batch size, the second batch size being larger than the first batch size; and The second batch is input to the speech recognition network.

17. The non-transitory computer readable medium of claim 16 storing program code to be executed by a processing unit to provide: End of speech detector to: receiving an acoustic feature frame of the audio signal independently of receipt of the acoustic feature frame by the batch processor; as well as Detecting a speech end frame based on the received acoustic feature frame; In response to the detection of the speech end frame, the batch processor is also used to: collect a third batch of acoustic feature frames of the audio signal, the third batch uses the speech end frame as the last frame of the third batch, and inputs the third batch into the speech recognition network.

18. The non-transitory computer readable medium of claim 16, wherein the first batch size comprises a first number of lookahead frames, and the first batch comprises a first lookahead frame, the number of first lookahead frames of the first batch being equal to the first number of lookahead frames, the batch processor being further configured to: collecting a third batch, the third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first preceding frame of the first batch, wherein the second batch includes second preceding frames, the number of the second preceding frames in the second batch is equal to the second number of preceding frames, and The speech recognition network includes a delay-controlled bidirectional long short-term memory model.

19. The non-transitory computer readable medium of claim 16, wherein the first batch size comprises a first number of lookahead frames, and the first batch comprises a first lookahead frame, the number of first lookahead frames of the first batch is equal to the first number of lookahead frames, the batch processor is further configured to: collecting a third batch, the third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first preceding frame of the first batch, wherein the second batch includes second preceding frames, the number of the second preceding frames of the second batch is equal to the first number of the preceding frames, and; The speech recognition network includes a context layer trajectory long short-term memory model.

20. The non-transitory computer-readable medium of claim 16, wherein detecting the word hypothesis comprises: A first non-silent word hypothesis output by the speech recognition network is detected.

Images