Speech recognition device, speech recognition method, and program

Abstract

According to the present invention, a speech recognition device comprises: a decoder score calculation unit which calculates a decoder score of a token wt; a complete hypothesis generation unit which temporarily connects <eos> and generates a complete hypothesis when the token wt is not the end of a sentence; a BLM score calculation unit which inverts a word order of the complete hypothesis and calculates a BLM score; a symbol delete unit which deletes the temporarily connected <eos> when the token wt is not the end of a sentence; an ISF score calculation unit which calculates an ISF score in a hypothesis w1:t of length t on the basis of an ISF score in a hypothesis w1:t-1 of length t-1, the decoder score, the BLM score, and a BLM score in a hypothesis w1:t-1 of length t-1; and an N best complete hypothesis output unit which outputs N best complete hypothesis on the basis of the ISF score.

Description

[Technical Field]

[0001] The present disclosure relates to a speech recognition device, a speech recognition method, and a program. [Background technology]

[0002] Speech recognition is a technology that converts human speech into a sequence of tokens (such as a sequence of words, subwords, or character strings) using a computer. The introduction of neural network (NN) technology has dramatically improved the accuracy of speech recognition. Conventional speech recognition systems consist of modules such as an acoustic analysis unit, an acoustic model unit, a language model unit, and a search unit, with each module (mainly the acoustic model unit and the language model unit) incorporating individual NN-based modeling. In recent years, however, end-to-end (E2E) speech recognition technology, which models a speech recognition system using a single NN, has been actively researched, and it has been reported that it has achieved speech recognition accuracy that exceeds human capabilities in telephone conversation speech recognition tasks. E2E speech recognition systems use a single NN, but the encoder, decoder, and other components are modularized within the system (see, for example, Non-Patent Document 1).

[0003] E2E speech recognition systems (models) are trained to directly capture the conversion from speech to token sequences using paired data of speech and its corresponding token sequences (text). While it is possible to train a highly accurate E2E speech recognition model using a large amount of paired data, there is a limit to the amount of paired data that can actually be prepared. This is because transcribing the token sequences corresponding to speech is usually done manually, which is costly in terms of both time and money.

[0004] On the other hand, text data alone can be obtained more easily and in larger quantities than paired data. Therefore, a method has been proposed in which a language model (a model that calculates the occurrence probability of a token sequence) trained using a large amount of text data prepared separately from paired data is integrated with an E2E speech recognition model as external knowledge (external language model). The most commonly used integration method is shallow fusion (SF). SF is a simple method that weights and adds the scores (logarithmic probability) output by the main E2E speech recognition model and the scores (also logarithmic probability) output by the external language model during the search process, yet has been proven to be highly effective (see, for example, Non-Patent Documents 1 and 2).

[0005] Since SF is performed sequentially according to the extension of speech recognition hypotheses from the beginning of a sentence in beam search, a conventional forward language model (FLM) is used as the external language model. In the following, "hypothesis" and "token sequence" are used interchangeably. Note that beam search is a process that simultaneously extends and develops multiple hypotheses while retaining the hypothesis that is likely to yield the highest score for the input speech and pruning hypotheses with low probability.

[0006] On the other hand, there is a method called rescoring of speech recognition hypotheses. Rescoring is a method of re-scoring (re-evaluating) multiple hypotheses obtained in the form of an N-best list or lattice using an external model (mainly a language model) for rescoring to obtain a final speech recognition result (one best hypothesis). In this method, a backward language model (BLM) is often used in addition to an FLM, and its effectiveness has been confirmed (see, for example, Non-Patent Document 3). Note that a backward language model (BLM) is a language model trained using data (reversed text data) in which the normal word order used in training an FLM is reversed. This model calculates the probability of occurrence of hypotheses in the reverse word order, from the end of the sentence to the beginning of the sentence.

[0007] The reason why BLM is used in rescoring is that BLM can evaluate the probability of a hypothesis from a different perspective than FLM, and the two models are complementary, so using them together can assign a more accurate language score to a hypothesis. Another reason is that rescoring is performed on hypotheses for which the search process has been completed, i.e., when the token sequence from the beginning of the sentence to the end (or from the end to the beginning) has been completely obtained (referred to as a "complete hypothesis" in this specification), BLM, which uses the end of the sentence as the starting point for score calculation, is easily applicable. Conversely, it is difficult to apply BLM to hypotheses in the middle of the search process whose end of the sentence has not yet been determined (referred to as a "partial hypothesis" in contrast to a complete hypothesis; also, when simply written as "hypothesis," it refers to a "partial hypothesis"). Furthermore, because FLM uses the beginning of the sentence as the starting point for score calculation, it can also be easily applied to partial hypotheses.

[0008] Figure 1 shows an overview of speech recognition processing in an E2E speech recognition model. In this specification, an attention-based encoder-decoder (AED) model is used as the E2E speech recognition model, and a label (token) synchronous beam search is used as the search algorithm (see, for example, Non-Patent Document 4). However, this disclosure is also applicable to other types of E2E speech recognition models and search algorithms. As shown in Figure 1, input speech is first input to the encoder in the E2E speech recognition model and converted into a hidden state vector sequence X. Here, it is rare for the speech waveform to be input as is; it is more common for a filter bank feature sequence extracted from the speech waveform to be input as the input speech.

[0009] Next, X is input to the decoder in the E2E speech recognition model, and a complete hypothesis { <sos> ,w^ 1:T , <eos>}={ <sos>,w1,w2,...,w T , <eos>} is found by the search process, and this is output as the final speech recognition result (1 best perfect hypothesis). Note that in the main text of the specification, "w^" is originally intended to be written as "^" directly above "w", but due to limitations in the functionality of word processing software, it is not possible to write it directly above, so for convenience it is written as "w^".

number

[0010] Figure 2 shows an overview of the speech recognition process in an E2E speech recognition model, where an FLM is used as an external language model using a conventional SF. The process from obtaining X from input speech by the encoder is the same as in Figure 1, but the difference from Figure 1 is that the search process in the decoder takes into account not only the decoder score (see equation (1)) but also the language score (FLM score) calculated by the FLM, as shown in equation (2) below.

number

[0011] FIG. 3 shows the details of the search process using SF in the decoder of FIG. 2. At time t, the length of the hypothesis { <sos>,w 1:t-1 } with token w t and the hypothesis { <sos>,w 1:t }. In this specification, the extension of a hypothesis by one token is considered to be the passage of one time. As a premise, when X is given, the hypothesis { <sos>,w 1:t-1 The score s1 of { has already been calculated at the previous time t-1. At t=0, the score s1 is initialized to 0. <sos>,w 1:t-1 } to w t When concatenating X and { <sos>,w 1:t-1 } given w t The decoder score s2 of { <sos>,w 1:t-1 } given w t The FLM score s3 is calculated by adding the above three scores together as shown in the following formula (3) to obtain the FLM score s3 when X is given. <sos>,w 1:t } and get a score s4.

number

number

[0012] The above explains the details of score calculation when extending a single type of token in a single hypothesis by concatenating them, but in reality, the number of types of tokens that can be concatenated to a single hypothesis is the total number of types, i.e., the vocabulary size. In the following, the vocabulary size is represented as V. The vocabulary includes the following: <eos>is included, but <sos>is not included.

[0013] In Fig. 4, in the search process with SF, at time t, one hypothesis { <sos>,w 1:t-1 }V kinds of tokens w v Concatenate (v=1,...,V) to find the V hypotheses { <sos>,w 1:t-1 ,w v }(v=1,...,V) is shown below. In this expansion, it is necessary to calculate the decoder score s2 (see Fig. 3) for the vocabulary size, but this can be done in bulk using a softmax function (see Non-Patent Document 4, for example). Regarding the FLM score s3 (see Fig. 3), if an LM based on a neural network is used as the FLM, V FLM scores s3 can be calculated in bulk using a softmax function, just like the decoder score s2. Furthermore, if a classical count-based n-gram LM is used as the FLM, the FLM score can be obtained by table lookup (however, when an n-gram LM is used, long hypotheses cannot be considered, so for example, in the case of a 3-gram LM (a language model that calculates the probability of a chain of 3 tokens), the hypothesis { <sos>,w 1:t-1 } to {w t-2,t-1 } (a chain of two tokens). For score calculation of FLM based on NN and n-gram, see, for example, Non-Patent Document 5.

[0014] The above describes score calculation when V types of tokens are concatenated into one hypothesis and expanded into V hypotheses. However, in an actual beam search, multiple hypotheses are expanded into V hypotheses, and these are then pruned. Figure 5 shows the overall processing procedure for beam search incorporating SF. The beam width is B (V>>B). Note that in Figure 5, one best complete hypothesis (the single complete hypothesis with the highest score) is output as the speech recognition result, but it may be desirable to output N best complete hypotheses (N complete hypotheses sorted in descending order from highest score based on score s4). In this case, in S94, instead of selecting only one best complete hypothesis, the (B-b+N) complete hypotheses are sorted in descending order based on score s4, as in step S92, and the top N complete hypotheses are selected and the others are pruned. [Prior art documents] [Non-patent literature]

[0015] [Non-Patent Document 1] Z. T¨uske, G. Saon, K. Audhkhasi, and B. Kingsbury, "Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard," in Proc. Interspeech, 2020, pp. 551-555. [Non-patent document 2] C. Gulcehre et al., "On using monolingual corpora in neural machine translation," arXiv:1503.03535v2 [cs.CL]. [Non-patent document 3] A. Ogawa, N. Tawara, M. Delcroix, and S. Araki, "Lattice rescoring based on large ensemble of complementary neural language models," in Proc. ICASSP, 2022, pp. 6517-6521. [Non-patent document 4] S. Watanabe, T. Hori, S. Kim, JR Hershey, and T. Hayashi, "Hybrid CTC / Attention architecture for end-to-end speech recognition," IEEE / ACM Transactions on Audio, Speech, and Language Processing, vol. 11, no. 8, pp. 1240-1253, Dec. 2017. [Non-patent document 5] M. Sundermeyer, R. Schl¨uter, and H. Ney, "LSTM neural network for language modeling," in Proc. Interspeech, 2012, pp. 194-197. [Non-patent document 6] Y. Bengio, "Practical recommendations for gradient-based training of deep architectures," arXiv:1206.5533v2 [cs.LG]. Summary of the Invention [Problem to be solved by the invention]

[0016] Since SF is also considered a type of rescoring using an external language model, if BLM could be applied to SF in addition to FLM, it would be possible to assign more accurate language scores to hypotheses during the search process, and it would be possible to achieve more accurate beam search.However, since SF extends and develops speech recognition hypotheses from the beginning of a sentence, FLM is used as the language model, and it is difficult to apply BLM to hypotheses during the search process where the end of the sentence has not been determined.

[0017] Therefore, an object of the present disclosure is to provide a speech recognition device that can incorporate BLM into SF. [Means for solving the problem]

[0018] The speech recognition device of the present invention includes a decoder score calculation unit, a complete hypothesis generation unit, a BLM score calculation unit, a symbol deletion unit, an ISF score calculation unit, and an N-best complete hypothesis output unit.

[0019] Let 1,…,t be time, and w1,w2,…,w t-1 ,w t Let be a token at each time in a speech recognition hypothesis, and let w be a hypothesis of length t-1. 1:t-1 ={w1,w2,...,w t-1 } with token w t and hypothesis w of length t. 1:t In the ISF score calculation when extending to t The decoder score is calculated for the speech recognition model of the token w. t A special symbol that indicates the end of a sentence if it is not <eos>The BLM score calculation unit calculates the BLM score by reversing the word order of the complete hypothesis. The symbol deletion unit calculates the BLM score by reversing the word order of the complete hypothesis. t Special symbols temporarily concatenated when not at the end of a sentence <eos>The ISF score calculation is performed by subtracting the hypothesis w 1:t-1 ISF score and token w t and the decoder score of a hypothesis w of length t. 1:t BLM score in and hypothesis w of length t-1 1:t-1 Based on the BLM score in the hypothesis w of length t 1:t The decoder score calculation unit, complete hypothesis generation unit, BLM score calculation unit, symbol deletion unit, and ISF score calculation unit perform their respective processes while increasing t by 1 until no partial hypotheses remain. The N-best complete hypothesis output unit outputs the N-best complete hypotheses based on the ISF scores. [Effects of the Invention]

[0020] According to the speech recognition device of the present invention, BLM can be incorporated into SF. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a diagram showing an overview of speech recognition processing in an E2E speech recognition model. [Figure 2] FIG. 1 is a diagram showing an overview of speech recognition processing in an E2E speech recognition model when FLM is used. [Figure 3] FIG. 3 is a diagram showing an example of score calculation when concatenating a token wt with a hypothesis of length t−1, as a detailed view of the search process in the decoder of FIG. 2; [Figure 4] A diagram showing an example of the calculation of scores (s2 and s3) when expanding one hypothesis into V hypotheses. [Figure 5] 10 is a flowchart showing the processing procedure of a conventional beam search in which SF is introduced. [Figure 6] FIG. 1 is a diagram showing an overview of speech recognition processing in an E2E speech recognition model when BLM is used. [Figure 7] FIG. 7 is a diagram showing an example of score calculation when concatenating a token wt with a hypothesis of length t−1, as a detailed view of the search process in the decoder of FIG. 6; [Figure 8] A diagram showing an example of the calculation of scores (s2 and s6) when expanding one hypothesis into V hypotheses. [Figure 9] FIG. 1 is a block diagram showing an example of the functional configuration of a speech recognition device according to a first embodiment. [Figure 10] FIG. 2 is a block diagram showing an example of the functional configuration of an N-best complete hypothesis output unit according to the first embodiment. [Figure 11] 3 is a flowchart showing the operation of the voice recognition device according to the first embodiment. [Figure 12] 4 is a flowchart showing a beam search operation of the speech recognition device according to the first embodiment. [Figure 13] FIG. 10 is a diagram showing an overview of speech recognition processing in an E2E speech recognition model when both FLM and BLM are used. [Figure 14] FIG. 14 is a diagram showing an example of score calculation when concatenating a token wt with a hypothesis of length t−1, as a detailed view of the search process in the decoder of FIG. 13; [Figure 15] FIG. 10 is a block diagram showing an example of the functional configuration of a speech recognition device according to a second embodiment. [Figure 16] 10 is a flowchart showing the operation of the voice recognition device according to the second embodiment. [Figure 17] 10 is a flowchart showing a beam search operation of the speech recognition device according to the second embodiment. [Figure 18] FIG. 10 is a block diagram showing an example of the functional configuration of a speech recognition device according to a fourth embodiment. [Figure 19] FIG. 13 is a block diagram showing an example of the functional configuration of an N-best complete hypothesis output unit according to the fourth embodiment. [Figure 20] 10 is a flowchart showing the operation of the voice recognition device according to the fourth embodiment. [Figure 21] 10 is a flowchart showing a beam search operation of the speech recognition device according to the fourth embodiment. [Figure 22] FIG. 10 is a block diagram showing an example of the functional configuration of a speech recognition device according to a fifth embodiment. [Figure 23] FIG. 13 is a block diagram showing an example of the functional configuration of an N-best complete hypothesis output unit according to the fifth embodiment. [Figure 24] 10 is a flowchart showing the operation of the voice recognition device according to the fifth embodiment. [Figure 25] 10 is a flowchart showing a beam search operation of the speech recognition device according to the fifth embodiment. [Figure 26] FIG. 13 is a block diagram showing an example of the functional configuration of a speech recognition device according to a sixth embodiment. [Figure 27] FIG. 20 is a block diagram showing an example of the functional configuration of an N-best complete hypothesis output unit according to the sixth embodiment. [Figure 28] 13 is a flowchart showing a beam search operation of the speech recognition device according to the sixth embodiment. [Figure 29] FIG. 20 is a diagram showing an example of a token string generated by the language model learning device according to the ninth embodiment. [Figure 30] FIG. 13 is a block diagram showing an example of the functional configuration of a language model learning device according to ninth and tenth embodiments. [Figure 31] 13 is a flowchart showing the operation of the language model learning device according to the ninth and tenth embodiments. [Figure 32] FIG. 23 is a diagram showing an example of a token string generated by the language model learning device of the tenth embodiment. [Figure 33] FIG. 23 is a diagram showing an example of a token string generated by the language model learning device of the eleventh embodiment. [Figure 34] FIG. 12 is a block diagram showing an example of the functional configuration of a language model learning device according to the eleventh and twelfth embodiments. [Figure 35] 13 is a flowchart showing the operation of the language model learning device according to the eleventh and twelfth embodiments. [Figure 36] FIG. 23 is a diagram showing an example of a token string generated by the language model learning device of the twelfth embodiment. [Figure 37] 10A and 10B are diagrams showing the effect of improving speech recognition accuracy by the speech recognition device described in the embodiment. [Figure 38] FIG. 2 is a diagram showing an example of the functional configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, embodiments of the present disclosure will be described in detail. Note that components having the same functions are assigned the same numbers, and redundant explanations will be omitted. [Example]

[0023] Figure 6 shows an overview of speech recognition processing in an E2E speech recognition model when a BLM is used as an external language model using the ISF of the present disclosure. In Figure 2, the search process in the decoder of the E2E speech recognition model takes into account the language scores calculated using the FLM using the SF, but in Figure 6, the language scores calculated using the BLM using the ISF are considered instead.

[0024] FIG. 7 shows the details of the search process in the decoder of FIG. 6, where at time t, a hypothesis w of length t-1 is 1:t-1 ={w1,w2,...,w t-1 } with token w t and hypothesis w of length t. 1:t The score calculation for extending to is as follows. As a premise, when X is given, w 1:t-1 The score s5 of w has already been calculated at the previous time t-1 (at t=0, the score s5 is initialized to 0). 1:t-1 To w t When concatenating X and { <sos>,w 1:t-1 } given w t Calculate the decoder score s2 of <eos>of{ <sos>,w 1:t } by temporarily concatenating it with (w t but <eos>(Not connected in the case of ), the complete hypothesis { <sos> ,w 1:t , <eos>For this complete hypothesis, we generate the word order { <eos> ,w t:1 , <sos>} and calculate the BLM score s6. After calculating the BLM score s6, <sos>,w 1:t } temporarily concatenated <eos>is deleted (w t but <eos>(If X is given, it will not be deleted.) By adding the weighted scores of the above three, the <sos>,w 1:t }, but in this case, the inverted temporary complete hypothesis { <eos> ,w t-1:1 , <sos>} (BLM score s7 is the search process at time t-1, <sos>,w 1:t-1 } and store it as information accompanying it. That is, the score s8 is obtained by the following formula (5).

number

number

number

[0025] Figure 8 shows the relationship between the time t and one hypothesis { <sos>,w 1:t-1 }V kinds of tokens w v Concatenate (v=1,...,V) to find the V hypotheses { <sos>,w 1:t-1 ,w v }(v=1,...,V) ...

[0026] The functional configuration of a speech recognition device according to the first embodiment incorporating the above-described processing will be described below with reference to Fig. 9. As shown in Fig. 9, the speech recognition device 1 of the present embodiment includes a decoder score calculation unit 11, a complete hypothesis generation unit 12, a BLM score calculation unit 13, a symbol deletion unit 14, an ISF score calculation unit 15, and an N-best complete hypothesis output unit 16.

[0027] As shown in FIG. 10, the N-best complete hypothesis output unit 16 includes a pruning unit 161, a hypothesis classification unit 162, a hypothesis selection unit 163, and a finalization unit 164.

[0028] The detailed operation of each component will be described below with reference to FIGS.

[0029] Hypothesis w of length t-1 1:t-1 Token w t and hypothesis w of length t. 1:t In calculating the ISF score when extending to 1:t-1 Based on the token w t The decoder score for the speech recognition model is calculated (S11).

[0030] The complete hypothesis generator 12 generates the token w t If it is not at the end of the sentence <eos>of{ <sos>,w 1:t } to temporarily connect the complete hypothesis { <sos> ,w 1:t , <eos>} is generated (S12).

[0031] The BLM score calculation unit 13 reverses the word order of the complete hypothesis and calculates the BLM score (S13).

[0032] The symbol deletion unit 14 deletes the token w t Special symbols temporarily concatenated when not at the end of a sentence <eos>(S14).

[0033] The ISF score calculation unit 15 calculates the hypothesis w of length t-1. 1:t-1 ISF score P isf ({ <sos>,w 1:t-1 }|X) and the decoder score P dec (w t |X,{ <sos>,w 1:t-1 }) and BLM score P blm ({ <eos> ,w t:1 , <sos>}) and a hypothesis w of length t-1 1:t-1 BLM score in P blm ({ <eos> ,w t-1:1 , <sos>}), for example, by using equation (5), we can obtain the hypothesis w 1:t ISF score P isf ({ <sos>,w 1:t }|X) is calculated (S15).

[0034] The decoder score calculation unit 11, the complete hypothesis generation unit 12, the BLM score calculation unit 13, the symbol deletion unit 14, and the ISF score calculation unit 15 perform their respective processes while incrementing t by 1 until no partial hypotheses remain. Note that the details will be described later, so the repeated branching is omitted from the figure.

[0035] The N best complete hypotheses output unit 16 outputs the N best complete hypotheses based on the ISF scores (S16).

[0036] The above is the general operation of each component. Below, with reference to Fig. 12, we will describe in detail the processing procedure of beam search (beam width = B) for a total of bV hypotheses, which are obtained by further expanding V hypotheses for each of multiple (b) hypotheses at time t.

[0037] The flowchart in this figure is similar to the flowchart in Figure 5, but differs in that in hypothesis development (S11-S15), scores s2 and s6 are calculated instead of scores s2 and s3, and in that in pruning (S161), pruning is performed based on score s8 instead of score s4.

[0038] The decoder score calculation unit 11, the complete hypothesis generation unit 12, the BLM score calculation unit 13, the symbol deletion unit 14, and the ISF score calculation unit 15 execute the above-mentioned steps S11 to S15 to calculate the scores s2, S6, and S8.

[0039] The pruning unit 161 sorts the hypotheses in descending order based on the score s8, keeps the top B hypotheses, and prunes the other hypotheses (S161).

[0040] The hypothesis classification unit 162 classifies the hypotheses according to whether they are complete hypotheses or not (S162).

[0041] The hypothesis selection unit 163 selects the 1(N) best complete hypotheses from the 1(N) best complete hypotheses thus far = the 1(N) complete hypotheses with the highest scores (if any) and the (Bb) newly obtained complete hypotheses (a total of (B-b+1(N)) complete hypotheses), and prunes the others (S163).

[0042] In the subsequent termination determination, if b=0, that is, if no partial hypotheses remain, the termination processing unit 164 outputs the 1(N) best complete hypotheses (S164), and the processing ends (END).

[0043] In the subsequent termination determination, if b≠0, that is, if a partial hypothesis remains, the speech recognition device 1 increments t, and executes steps S11 to S15, S161, S162, and S163 again, and executes the termination determination again. [Example]

[0044] The speech recognition device of the second embodiment simultaneously performs SF and ISF. Fig. 13 shows an outline of speech recognition processing in an E2E speech recognition model, in which FLM is used as an external language model by conventional SF, and BLM is used as an external language model by ISF.

[0045] FIG. 14 shows the details of the search process in the decoder of FIG. 13, where at time t, a hypothesis w of length t−1 is 1:t-1 ={w1,w2,...,w t-1 } with token w t and hypothesis w of length t. 1:t The score calculation for extending to is as follows. As a premise, when X is given, w 1:t-1 The score s9 of w has already been calculated at the previous time t-1 (at t=0, the score s9 is initialized to 0). 1:t-1 To w t When concatenating X and { <sos>,w 1:t-1 } given w t The decoder score s2 of { <sos>,w 1:t-1 } given w t Calculate the FLM score s3 of <eos>of{ <sos>,w 1:t } by temporarily concatenating it with (w t but <eos>(Not connected in the case of ), the complete hypothesis { <sos> ,w 1:t , <eos>} is temporarily generated. For this complete hypothesis, the word order is <eos> ,w t:1 , <sos>} and calculate the BLM score s6. After calculating the BLM score s6, <sos>,w 1:t } temporarily concatenated <eos>is deleted (w t but <eos>(If X is given, it is not deleted.) By adding the weighted scores of the above four, the <sos>,w 1:t }, but in this case, the inverted temporary complete hypothesis { <eos> ,w t-1:1 , <sos>} (BLM score s7 is the search process at time t-1, <sos>,w 1:t-1 } and store it as information accompanying it. That is, the score s10 is obtained by the following formula (8).

number

[0046] Hereinafter, the detailed operation of each component will be described with reference to FIG. 16. Step S11 is executed in the same manner as in the first embodiment. The FLM score calculation unit 22 calculates the symbol <sos>and hypothesis w of length t-1 1:t-1 Based on the token w at time t t FLM score P flm (w t |{ <sos>,w 1:t-1 )}) is calculated (S22). Steps S12 to S14 are executed in the same manner as in the first embodiment. The SF+ISF score calculation unit 25 calculates the hypothesis w 1:t-1 SF+ISF score P sf+isf ({ <sos>,w 1:t-1 }|X) and the decoder score P dec (w t |X,{ <sos>,w 1:t-1 }) and FLM score P flm (w t |{ <sos>,w 1:t-1 }) and BLM score P blm ({ <eos> ,w t:1 , <sos>}) and BLM score P blm ({ <eos> ,w t-1:1 , <sos>}), for example, by equation (8), we can obtain the hypothesis w 1:t SF+ISF score P sf+isf ({ <sos>,w 1:t }|X) is calculated (S25).

[0047] The overall processing procedure for beam search (beam width = B) incorporating SF and ISF is shown in Figure 17. The flowchart in this figure is similar to the flowchart in Figure 12, but differs in that scores s2, s3, s6, and s10 are calculated in hypothesis development (S11, S22, S12, S13, S14, S25), and that pruning (S161) is performed based on score s10 instead of score s8.

[0048] If α=0 and β>0 in equation (8), SF is not performed and only ISF is performed (equivalent to Example 1). If α>0 and β=0 in equation (8), ISF is not performed and only SF is performed. If α=0 and β=0 in equation (8), neither SF nor ISF is performed.

[0049] As described above, the speech recognition devices described in Examples 1 and 2 can incorporate BLM into SF, allowing for more accurate language scores to be assigned to hypotheses during the search process, thereby achieving more accurate beam search. However, according to the above-mentioned method, it is clear that the calculation cost of ISF using BLM is high. In the following Examples 3 to 6, speech recognition devices that reduce the calculation cost will be disclosed. [Example]

[0050] In the methods of Examples 1 and 2, the BLM score s6 is calculated individually for each vocabulary size, resulting in high computational costs (see FIG. 8). Therefore, in Example 3, the BLM score s6 is calculated using batch calculations that bundle together hypotheses for the vocabulary size (see, for example, Non-Patent Document 6 for details of batch calculations). In the label (token) synchronous beam search (see, for example, Non-Patent Document 4) assumed in this specification, batch calculations are particularly effective (no wasted calculations) because the lengths of each hypothesis during the search process are basically the same, but batch calculations can also be introduced into other search algorithms.

[0051] That is, in the speech recognition device of the third embodiment, the BLM score calculation unit calculates the BLM score s6 by batch calculation of hypotheses corresponding to the vocabulary size, thereby realizing a reduction in the cost of calculating the BLM score. [Example]

[0052] In the above-described embodiment, the BLM score s6 based on the ISF was calculated every time the hypothesis was extended by one token (one time passed), but in the fourth embodiment, the BLM score s6 based on the ISF is calculated every time the hypothesis is extended by I tokens (I time passed). In other words, the BLM score s6 based on the ISF is calculated only when the length (time) t of the hypothesis satisfies t%I=0 (% is the modulo operator). Hereinafter, I is referred to as the interval (I≧1). Equation (7) in the first embodiment is changed to the following equation (9) in this embodiment.

number

[0053] As shown in FIG. 19, the N-best complete hypothesis output unit 46 includes a pruning unit 161, a hypothesis classification unit 162, a complete hypothesis score calculation unit 463, a hypothesis selection unit 163, and a termination processing unit 164, and the components other than the complete hypothesis score calculation unit 463 operate in the same manner as in Examples 1 and 2.

[0054] The detailed operation of each component will be described below with reference to Fig. 20. Steps S11, S22, and S12 are executed in the same manner as in the second embodiment. The BLM score calculation unit 43 calculates the BLM score only when I is an arbitrary interval and the length of the hypothesis t satisfies t%I=0 (% is the modulo operator) (S43). Steps S14 and S25 are executed in the same manner as in the second embodiment.

[0055] Figure 21 shows the overall processing procedure for beam search that introduces ISF based on SF and interval I. The beam width is B. The flowchart in this figure is similar to the flowchart in Figure 17, but differs in that step S43 is performed instead of step S13 in hypothesis expansion, reducing the frequency of BLM score calculation; in step S161, pruning is performed based on score s10 when t%I=0, and based on score s9+s2+α×s3 other than the BLM score when t%I≠0; and step S463 is inserted between steps S162 and S163, where score s10 is calculated for the complete hypothesis when t%I≠0.

[0056] As mentioned above, the effectiveness of rescoring (N-best rescoring and lattice rescoring) for complete hypotheses has been confirmed in the past (see, for example, Non-Patent Document 3). Therefore, in this embodiment as well, in step S463, if t%I≠0 for the complete hypothesis, score s10 is calculated. If t%I=0, that is, if score s10 has already been calculated in hypothesis development (S11, S22, S12, S43, S14, S25), there is no need to calculate it again. By introducing interval I, the calculation cost of ISF becomes approximately 1 / I compared to before introduction.

[0057] If I=1 in equation (9), this embodiment is equivalent to embodiment 2. If α=0 and β>0 in equation (8), SF is not performed, and only ISF is performed in interval I. If α>0 and β=0 in equation (8), ISF is not performed, and only SF is performed. If α=0 and β=0 in equation (8), neither SF nor ISF is performed. [Example]

[0058] In beam search, it is necessary to retain the hypotheses (multiple hypotheses) that are likely to obtain the highest score for the input speech, and to prune other hypotheses with low probability at an early stage. If a hypothesis with high probability is erroneously pruned at an early stage of the search process, it becomes difficult to obtain a highly accurate speech recognition result in the end. Therefore, it is considered important to assign highly accurate scores to hypotheses at an early stage of the search process rather than at a later stage. Therefore, in Example 5, ISF is performed only when the hypothesis length t is shorter than L (t≦L).

[0059] The functional configuration of a speech recognition device according to the fifth embodiment incorporating the above-described processing will be described below with reference to Fig. 22. As shown in the figure, the speech recognition device 5 of this embodiment includes a decoder score calculation unit 11, an FLM score calculation unit 22, a complete hypothesis generation unit 12, a BLM score calculation unit 53, a symbol deletion unit 14, an SF+ISF score calculation unit 25, and an N-best complete hypothesis output unit 56. The components other than the BLM score calculation unit 53 and the N-best complete hypothesis output unit 56 operate in the same way as those of the second embodiment.

[0060] As shown in FIG. 23, the N-best complete hypothesis output unit 56 includes a pruning unit 161, a hypothesis classification unit 162, a complete hypothesis score calculation unit 563, a hypothesis selection unit 163, and a termination processing unit 164, and the components other than the complete hypothesis score calculation unit 563 operate in the same manner as in the first and second embodiments.

[0061] The detailed operation of each component will be described below with reference to Fig. 24. Steps S11, S22, and S12 are executed in the same manner as in the second embodiment. The BLM score calculation unit 53 calculates the BLM score only when the length t of the hypothesis is shorter than L (t≦L), where L is an arbitrary working hypothesis length (S53). Steps S14 and S25 are executed in the same manner as in the second embodiment.

[0062] Figure 25 shows the overall processing procedure for beam search incorporating ISF based on SF and ISF implementation hypothesis length L. The beam width is B. The flowchart in this figure is similar to the flowchart in Figure 17, but differs in that step S53 is performed instead of step S13 in hypothesis expansion, reducing the frequency of BLM score calculation; in step S161, pruning is performed based on score s10 if t≦L, and based on score s9+s2+α×s3 other than the BLM score if t>L; and step S563 is inserted between steps S162 and S163, where score s10 is calculated for the complete hypothesis if t>L.

[0063] If L=∞, this embodiment becomes equivalent to embodiment 2. If α=0 and β>0 in equation (8), SF is not performed, and only ISF is performed with the ISF execution hypothesis length L. If α>0 and β=0 in equation (8), ISF is not performed, and only SF is performed. If α=0 and β=0 in equation (8), neither SF nor ISF is performed. [Example]

[0064] The sixth embodiment is a modification of the second embodiment, in which pruning is performed in two stages. The functional configuration of the speech recognition device of the sixth embodiment, which incorporates two-stage pruning processing, will be described below with reference to Fig. 26. As shown in the figure, the speech recognition device 6 of the sixth embodiment includes a decoder score calculation unit 11, an FLM score calculation unit 22, a complete hypothesis generation unit 12, a BLM score calculation unit 13, a symbol deletion unit 14, an SF+ISF score calculation unit 25, and an N-best complete hypothesis output unit 66. The components other than the N-best complete hypothesis output unit 66 operate in the same way as those of the second embodiment.

[0065] As shown in FIG. 27, the N-best complete hypothesis output unit 66 includes a first pruning unit 661, a second pruning unit 662, a hypothesis classification unit 162, a hypothesis selection unit 163, and a termination processing unit 164, and the components other than the first pruning unit 661 and the second pruning unit 662 operate in the same manner as in Example 2.

[0066] 28 shows the overall processing procedure of beam search when pruning is performed in two stages as a modification of Example 2. The beam width of the first stage pruning is M, and the beam width of the second stage pruning is B (M>B). The flowchart in this figure is similar to the flowchart in Figure 17, but differs in that only S11 and S22, which have low costs in hypothesis development, are executed; in step S661, a first pruning unit 661 performs pruning to leave the top M hypotheses based on a score other than the BLM score, s9 + s2 + α × s3; the BLM score calculation unit 13 calculates BLM scores only for the top M hypotheses from the first stage of pruning, and other components perform pre-processing and post-processing (S12, S13, S14, S25), thereby reducing the frequency of BLM score calculation; and after BLM score calculation, in step S662, a second pruning unit 662 performs pruning to leave the top B hypotheses based on the score S10 from the second stage of pruning.

[0067] In this embodiment, in the hypothesis development and first-stage pruning (S11, S22, S661), the first stage pruning is performed based on the decoder score s2 and FLM score s3, which have low computational costs, leaving M hypotheses. In the BLM score calculation and second-stage pruning (S12, S13, S14, S25), the second stage pruning is performed taking into account the BLM score s6, which has high computational costs, leaving B hypotheses. In other words, instead of calculating BLM scores for all of the extremely large number bV hypotheses as in the second embodiment, the BLM score calculation is performed only for a somewhat narrowed number of M hypotheses, thereby reducing computational costs (bV>>M>B). Possible methods for setting M include, for example, M = B × B.

[0068] If M=bV, this embodiment becomes equivalent to embodiment 2. If α=0 and β>0 in equation (8), SF is not performed (first-stage pruning is performed based only on the decoder score s2), and only ISF is performed. If α>0 and β=0 in equation (8), ISF is not performed and only SF is performed (first-stage pruning is not performed (there is no point in performing it) and only second-stage pruning is performed based on the score s9+s2+α×s3). If α=0 and β=0 in equation (8), neither SF nor ISF is performed (first-stage pruning is not performed and only second-stage pruning is performed based on the score s9+s2). [Example]

[0069] Based on Example 1, it is also possible to modify it to perform two or more of batch calculation (Example 3), ISF based on interval I (Example 4), ISF based on ISF execution hypothesis length L (Example 5), and two-stage pruning (Example 6). [Example]

[0070] Based on Example 2, it is also possible to modify it to perform two or more of batch calculation (Example 3), ISF based on interval I (Example 4), ISF based on ISF execution hypothesis length L (Example 5), and two-stage pruning (Example 6). [Example]

[0071] The following has been manually transcribed: <sos>Starts with <eos>A grammatically and order-correct token sequence ending in is called a "complete token sequence." BLM is trained using a complete token sequence with reversed word order, but as mentioned above, in the ISF of this disclosure, BLM is applied to partial hypotheses (temporarily adding <eos>(These are concatenated to form a complete hypothesis, to which BLM is applied.) Therefore, as shown in Fig. 29, the device of the ninth embodiment generates (augmented) a token sequence assuming that BLM will be applied to ISF, and uses the generated token sequence (hereinafter also referred to as a degenerate token sequence) to learn an ISF-specific BLM.

[0072] The functional configuration of a language model training device that generates the above-mentioned token sequence and trains a language model will be described with reference to Fig. 30. As shown in the figure, a language model training device 900 of embodiment 9 (and embodiment 10, described later) includes a word order reversal unit 901, a degenerate token sequence generation unit 902, an FLM language model training unit 903, a BLM language model training unit 904, and an ISF-specific BLM language model training unit 905.

[0073] The operation of each component will be described below with reference to Fig. 31. The word order reversal unit 901 generates a complete token sequence ({ in the example of Fig. 29) used for FLM training. <sos>, a, b, c, d, e, f, g, h, i, j, <eos>} and reverses the word order of the complete token sequence (S901). The reduced token sequence generation unit 902 obtains the special symbol}, which indicates the end of a sentence, from the complete token sequence with the reversed word order. <eos>A degenerate token sequence is generated by deleting I tokens (I is a natural number, I=1 in this embodiment) next to each token (S902). The FLM language model training unit 903 generates an FLM language model based on the complete token sequence used for FLM training (S903). The BLM language model training unit 904 trains a BLM language model based on a token sequence obtained by reversing the word order of the complete token sequence (S904). The ISF-specific BLM language model training unit 905 trains an ISF-specific BLM language model based on the degenerate token sequence (S905).

[0074] In this embodiment, the interval I is set to 1 (Embodiment 4), and the ISF execution hypothesis length L is set to ∞ (Embodiment 5). By setting in this way, it is possible to generate an inverted token sequence of any length starting from any token, and to learn an ISF-specific BLM that can easily handle any situation in the ISF.

[0075] However, as is clear from Fig. 29, when I = 1 and L = ∞ are set, the amount of training data for the ISF-specific BLM becomes enormous. In the following tenth to twelfth examples, I and L are appropriately set to limit the amount of training data for the ISF-specific BLM. [Example]

[0076] As shown in FIG. 32, the language model learning device of the tenth embodiment limits the amount of learning data for the ISF-specific BLM by setting I to a value greater than 1 (I=2 in FIG. 32) while leaving L at ∞. [Example]

[0077] As shown in Figure 33, the language model learning device of Example 11 limits the amount of learning data for the ISF-specific BLM by leaving I at 1 and setting L to a finite integer value greater than 1 (L = 6 in Figure 33). The functional configuration of a language model training device of Example 11 (and Example 12, described later) will be described with reference to Fig. 34. As shown in the figure, a language model training device 1100 of Example 11 includes a word order reversal unit 901, a degenerate token sequence generation unit 1102, an FLM language model training unit 903, a BLM language model training unit 904, and an ISF-specific BLM language model training unit 905, and the configuration other than the degenerate token sequence generation unit 1102 is the same as in Examples 9 and 10. The language model training device 1100 of this example aims to use only token sequences shorter than the working hypothesis length L for training the ISF-specific BLM language model.

[0078] 35, the operations of the components of the language model learning device 1100 of the eleventh embodiment that are different from those of the language model learning device 900 of the ninth embodiment will be described. The degenerate token sequence generation unit 1102 generates the following from the complete token sequence with the word order reversed: <eos>Remove the token placed next to <eos>and special symbols to indicate the beginning of a sentence <sos>The first degenerate token string is generated so that the length of the token string excluding is equal to L, and thereafter, degenerate token strings are generated by deleting I tokens at a time (I=1 in this embodiment) (S1102). [Example]

[0079] As shown in Figure 36, the language model learning device of Example 12 limits the amount of learning data for the ISF-specific BLM by setting I to a value greater than 1 (I = 2 in Figure 36) and setting L to a value greater than 1 (L = 6 in Figure 36). [Example]

[0080] The speech recognition device of the thirteenth embodiment implements the first to eighth embodiments using the ISF-specific BLM trained in the ninth to twelfth embodiments.

[0081] <Experimental Results> Figure 37 shows the improvement in speech recognition accuracy (reduction in word error rate [%]) achieved by the speech recognition device described in the examples. A neural language model (LSTMLM) based on long short-term memory (LSTM) is used as the language model. Comparing method 0 with method 1 confirms the effectiveness of SF using conventional FLM. Comparing method 0 with method 2 or 3 confirms the effectiveness of ISF using BLM by the speech recognition device described in the examples. Comparing method 2 with method 3 shows that the ISF-specific BLM according to example 9 is more suitable for ISF than the normal BLM. Comparing methods 1, 2, and 3 with method 4 confirms the effectiveness of combining SF using FLM and ISF using (ISF-specific) BLM.

[0082] <Additional Notes> The device disclosed herein may, for example, be a single hardware entity having an input unit to which a keyboard or the like can be connected, an output unit to which an LCD display or the like can be connected, a communication unit to which a communication device (e.g., a communication cable) capable of communicating with an external device can be connected, a CPU (which may also include a central processing unit, cache memory, registers, etc.), memories such as RAM and ROM, an external storage device such as a hard disk, and buses connecting these input unit, output unit, communication unit, CPU, RAM, ROM, and external storage device so that data can be exchanged between them. If necessary, the hardware entity may also be provided with a device (drive) capable of reading and writing to a recording medium such as a CD-ROM. A physical entity equipped with such hardware resources includes a general-purpose computer.

[0083] The external storage device of the hardware entity stores the programs required to realize the above-mentioned functions and the data required for processing these programs (the programs may be stored in a ROM, which is a read-only storage device, for example, instead of an external storage device). Data obtained by processing these programs is stored in RAM, the external storage device, etc. as appropriate.

[0084] In a hardware entity, each program stored in an external storage device (or ROM, etc.) and the data required to process each program are loaded into memory as needed, and interpreted, executed, and processed by the CPU as appropriate, resulting in the CPU realizing a predetermined function (each component represented as a unit, means, etc., above).

[0085] The present disclosure is not limited to the above-described embodiments, and modifications can be made as appropriate without departing from the spirit of the present disclosure. Furthermore, the processes described in the above embodiments may not only be executed in chronological order according to the order described, but may also be executed in parallel or individually depending on the processing capabilities of the devices that execute the processes or as needed.

[0086] As described above, when the processing functions of the hardware entities (apparatuses of the present disclosure) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entities should have are described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

[0087] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 10020 of the computer 10000 shown in Figure 38 and operating the control unit 10010, input unit 10030, output unit 10040, etc.

[0088] The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic recording devices, optical disks, magneto-optical recording media, and semiconductor memories. Specifically, examples of magnetic recording devices include hard disk drives, flexible disks, and magnetic tapes; optical disks include DVDs (Digital Versatile Discs), DVD-RAMs (Random Access Memory), CD-ROMs (Compact Disc Read Only Memory), and CD-Rs (Recordable) / RWs (Rewritable); magneto-optical recording media include MOs (Magneto-Optical discs), and semiconductor memories include EEP-ROMs (Electrically Erasable and Programmable-Read Only Memory).

[0089] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0090] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the received program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0091] In addition, in this embodiment, a hardware entity is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.< / sos> < / eos> < / eos> < / eos> < / eos> < / sos> < / eos> < / eos> < / sos> < / sos> < / sos> < / eos> < / sos> < / eos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / eos> < / sos> < / eos> < / eos> < / sos> < / sos> < / eos> < / eos> < / sos> < / eos> < / sos> < / eos> < / sos> < / sos> < / sos> < / sos> < / eos> < / sos> < / eos> < / sos> < / sos> < / eos> < / eos> < / sos> < / sos> < / eos> < / sos> < / sos> < / eos> < / eos> < / sos> < / sos> < / eos> < / sos> < / eos> < / eos> < / sos> < / sos> < / eos> < / eos> < / sos> < / eos> < / sos> < / eos> < / sos> < / eos> < / eos> < / sos> < / sos> < / sos> < / sos> < / eos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / sos> < / eos> < / sos> < / eos> < / sos> < / eos> < / sos> < / eos> < / sos>

Claims

1. Let 1,…,t be the time, and w 1 ,w 2 ,...,w t-1 , w t Let be a token at each time in a speech recognition hypothesis, and let w be a hypothesis of length t-1. 1:t-1 ={w 1 ,w 2 ,...,w t-1 } with token w t and hypothesis w of length t. 1:t In ISF scoring calculations when extending to The token w t a decoder score calculation unit for calculating a decoder score in the speech recognition model; The token w t A special symbol that indicates the end of a sentence if it is not <eos> a complete hypothesis generation unit that temporarily concatenates the above and generates a complete hypothesis;< / eos> a BLM score calculation unit that calculates a BLM score by reversing the word order of the complete hypothesis; The token w t Special symbols temporarily concatenated when not at the end of a sentence <eos> a symbol removal unit that removes< / eos> Hypothesis w of length t-1 1:t-1 and the ISF score for the token w t and a hypothesis w of length t. 1:t The BLM score in and the hypothesis w of length t-1 1:t-1 Based on the BLM score in 1:t an ISF score calculation unit for calculating an ISF score in the decoder score calculation unit, the complete hypothesis generation unit, the BLM score calculation unit, the symbol deletion unit, and the ISF score calculation unit perform each process while increasing t by 1 until there are no more partial hypotheses; Includes an N-best complete hypothesis output unit that outputs the N-best complete hypotheses based on the ISF scores. Voice recognition device.

2. 2. The speech recognition device according to claim 1, Special symbols to indicate the beginning of a sentence <sos>and hypothesis w of length t-1 1:t-1 Based on the token w at time t t Further comprising an FLM score calculation unit for calculating an FLM score of< / sos> The ISF score calculation unit Hypothesis w of length t-1 1:t-1 SF+ISF score in the token w t and the token w t and the FLM score of a hypothesis w of length t. 1:t-1 The BLM score in and the hypothesis w of length t-1 1:t-1 Based on the BLM score in 1:t Calculate the SF+ISF score in Voice recognition device.

3. 3. The speech recognition device according to claim 2, The BLM score calculation unit Calculate the BLM score by batching hypotheses equal to the vocabulary size Voice recognition device.

4. 3. The speech recognition device according to claim 2, The BLM score calculation unit Let I be any interval, and calculate the BLM score only if the length of the hypothesis, t, satisfies t%I=0 (% is the modulus operator). Voice recognition device.

5. 3. The speech recognition device according to claim 2, The BLM score calculation unit Let L be the length of any hypothesis, and calculate the BLM score only if the length of the hypothesis, t, is shorter than L (t ≤ L). Voice recognition device.

6. 3. The speech recognition device according to claim 2, The BLM score calculation unit The beam width of pruning is set to M, and the BLM score is calculated for only the top M hypotheses based on scores other than the BLM score. Voice recognition device.

7. A speech recognition method executed by a speech recognition device, comprising: Let 1,…,t be the time, and w 1 ,w 2 ,...,w t-1 , w t Let be a token at each time in a speech recognition hypothesis, and let w be a hypothesis of length t-1. 1:t-1 ={w 1 ,w 2 ,...,w t-1 } with token w t and hypothesis w of length t. 1:t In ISF scoring calculations when extending to The token w t a decoder score calculation step of calculating a decoder score for the speech recognition model; The token w t A special symbol that indicates the end of a sentence if it is not <eos> a complete hypothesis generation step in which the following are temporarily concatenated to generate a complete hypothesis;< / eos> a BLM score calculation step of calculating a BLM score by reversing the word order of the complete hypothesis; The token w t Special symbols temporarily concatenated when not at the end of a sentence <eos> a symbol removal step, which removes< / eos> Hypothesis w of length t-1 1:t-1 and the ISF score for the token w t and a hypothesis w of length t. 1:t The BLM score in and the hypothesis w of length t-1 1:t-1 Based on the BLM score in 1:t an ISF score calculation step for calculating an ISF score for the decoder score calculation step, the complete hypothesis generation step, the BLM score calculation step, the symbol deletion step, and the ISF score calculation step are performed while incrementing t by 1 until no partial hypotheses remain; An N-best complete hypothesis output step is included, which outputs the N-best complete hypotheses based on the ISF scores. Speech recognition methods.

8. A program that causes a computer to function as the speech recognition device according to any one of claims 1 to 6.

Images