Voice recognition device, voice recognition method, learning device, learning method, and recording medium

Abstract

A speech recognition device (1) comprises: an output means (111) which, when speech data is inputted, uses a neural network (NN) for outputting a first probability (CP), which is the probability of a character sequence corresponding to a speech sequence expressed by the speech data, and a second probability (PP), which is the probability of a phoneme sequence corresponding to the speech sequence, to output a first probability and a second probability; and an update means which updates the first probability on the basis of the second probability and dictionary data (121) in which registered characters and registered phonemes that are the phonemes of the registered characters have been associated with each other.

Description

[Technical field]

[0001] The disclosure relates to the technical fields of, for example, a voice recognition device and a voice recognition method capable of performing voice recognition processing using a neural network capable of outputting the probability of a character sequence corresponding to a voice sequence indicated by the voice data when voice data is input, a learning device and a learning method capable of learning parameters of a neural network capable of outputting the probability of a character sequence corresponding to a voice sequence indicated by the voice data when voice data is input, and a recording medium on which a computer program for causing a computer to execute the voice recognition method or the learning method is recorded. [Background technology]

[0002] As an example of a voice recognition device, a voice recognition device that performs a voice recognition process to convert voice data into a character sequence corresponding to a voice sequence indicated by the voice data by using a statistical method is known. Specifically, a voice recognition device that performs a voice recognition process using a statistical method performs the voice recognition process by using an acoustic model, a language model, and a pronunciation dictionary. The acoustic model is used to identify the phonemes of the voice indicated by the voice data. For example, a hidden Markov model (HMM) is used as the acoustic model. The language model is used to evaluate the likelihood of occurrence of a word sequence corresponding to the voice sequence indicated by the voice data. The pronunciation dictionary represents constraints on the arrangement of phonemes, and is used to associate the word sequence of the language model with the phoneme sequence identified based on the acoustic model.

[0003] Meanwhile, in recent years, the development of end-to-end speech recognition devices has progressed rapidly. An example of an end-to-end speech recognition device is described in Patent Document 1. The end-to-end speech recognition device is a speech recognition device that performs speech recognition processing by using a neural network that outputs a character sequence corresponding to a voice sequence indicated by the speech data when the speech data is input. Such an end-to-end speech recognition device is capable of performing speech recognition processing without separately preparing an acoustic model, a language model, and a pronunciation dictionary.

[0004] Other prior art documents related to this disclosure include Patent Documents 2 to 4. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2018 / 066436 Brochure [Patent Document 2] JP 2014-232510 A [Patent Document 3] JP 2002-278584 A [Patent Document 4] Japanese Patent Application Publication No. 08-297499 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present disclosure is to provide a speech recognition device, a speech recognition method, a learning device, a learning method, and a recording medium that aim to improve upon the techniques described in prior art documents. [Means for solving the problem]

[0007] One aspect of the voice recognition device includes an output means for outputting a first probability, which is the probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability, which is the probability of a phoneme sequence corresponding to the voice sequence, using a neural network when voice data is input, and outputting the first probability and the second probability, and an update means for updating the first probability based on dictionary data in which registered characters are associated with registered phonemes that are phonemes of the registered characters, and the second probability.

[0008] One aspect of the voice recognition method uses a neural network that, when voice data is input, outputs a first probability, which is the probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability, which is the probability of a phoneme sequence corresponding to the voice sequence, and outputs the first probability and the second probability, and updates the first probability based on dictionary data in which registered characters are associated with registered phonemes, which are phonemes of the registered characters, and the second probability.

[0009] One aspect of the learning device includes an acquisition means for acquiring learning data including first voice data for learning, a correct label of a first character sequence corresponding to a first voice sequence indicated by the first voice data, and a correct label of a first phoneme sequence corresponding to the first voice sequence, and a learning means for using the learning data to learn parameters of a neural network that outputs, when second voice data is input, a first probability which is the probability of a second character sequence corresponding to the second voice sequence indicated by the second voice data, and a second probability which is the probability of a second phoneme sequence corresponding to the second voice sequence.

[0010] One aspect of the learning method includes acquiring learning data including first speech data for learning, a correct label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct label of a first phoneme sequence corresponding to the first speech sequence, and using the learning data to learn parameters of a neural network that outputs, when second speech data is input, a first probability which is the probability of a second character sequence corresponding to a second speech sequence indicated by the second speech data, and a second probability which is the probability of a second phoneme sequence corresponding to the second speech sequence.

[0011] A first aspect of the recording medium is a recording medium having recorded thereon a computer program for causing a computer to execute a speech recognition method that uses a neural network to output, when voice data is input, a first probability, which is the probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability, which is the probability of a phoneme sequence corresponding to the voice sequence, and outputs the first probability and the second probability, and updates the first probability based on dictionary data in which registered characters are associated with registered phonemes, which are phonemes of the registered characters, and the second probability.

[0012] A second aspect of the recording medium is a recording medium having recorded thereon a computer program that causes a computer to execute a learning method for acquiring learning data including first speech data for learning, a correct label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct label of a first phoneme sequence corresponding to the first speech sequence, and using the learning data to learn parameters of a neural network that outputs, when second speech data is input, a first probability which is the probability of a second character sequence corresponding to the second speech sequence indicated by the second speech data, and a second probability which is the probability of a second phoneme sequence corresponding to the second speech sequence. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram showing the configuration of a voice recognition device according to this embodiment. [Diagram 2] FIG. 2 is a table showing an example of character probabilities output by the voice recognition device of this embodiment. [Diagram 3] FIG. 3 is a table showing an example of phoneme probabilities output by the speech recognition device of this embodiment. [Figure 4] FIG. 4 is a data structure diagram showing an example of the data structure of dictionary data used by the voice recognition device of this embodiment. [Diagram 5] FIG. 5 is a flowchart showing the flow of the voice recognition process performed by the voice recognition device. [Figure 6]FIG. 6 is a table showing the most likely phonemes (ie, the phonemes with the highest phoneme probability) at a given time. [Figure 7] FIG. 7 is a table showing character probabilities before the speech recognition device is updated. [Figure 8] FIG. 8 is a table showing the character probabilities after the speech recognizer has been updated. [Figure 9] FIG. 9 is a block diagram showing a configuration of a voice recognition device in a modified example. [Figure 10] FIG. 10 is a block diagram showing the configuration of the learning device of this embodiment. [Figure 11] FIG. 11 is a data structure diagram showing an example of the data structure of the learning data used by the learning device of this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of a voice recognition device, a voice recognition method, a learning device, a learning method, and a recording medium will be described. Hereinafter, an embodiment of a voice recognition device and a voice recognition method (and an embodiment of a recording medium on which a computer program for causing a computer to execute the voice recognition method) will be described using a voice recognition device 1, and then an embodiment of a learning device and a learning method (and an embodiment of a recording medium on which a computer program for causing a computer to execute the learning method) will be described using a learning device 2.

[0015] (1) Speech recognition device 1 of this embodiment First, a voice recognition device 1 of the present embodiment will be described. The voice recognition device 1 is capable of performing a voice recognition process for identifying a character sequence and a phoneme sequence corresponding to a voice sequence indicated by voice data, based on voice data. The voice sequence may mean a time series of a voice uttered by a speaker (i.e., a time change of a voice, and an observation result obtained by continuously or discontinuously observing the time change of the voice). The character sequence may mean a time series of characters corresponding to a voice uttered by a speaker (i.e., a time change of a character corresponding to a voice, and a series of characters in which a plurality of characters are connected). The phoneme sequence may mean a time series of phonemes corresponding to a voice uttered by a speaker (i.e., a time change of a phoneme corresponding to a voice, and a series of phoneme groups in which a plurality of phonemes are connected).

[0016] The configuration and operation of the voice recognition device 1 capable of performing such voice recognition processing will be described below.

[0017] (1-1) Configuration of speech recognition device 1 First, the configuration of a voice recognition device 1 of this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the voice recognition device 1 of this embodiment.

[0018] As shown in Fig. 1, the speech recognition device 1 includes a calculation device 11 and a storage device 12. The speech recognition device 1 may further include a communication device 13, an input device 14, and an output device 15. However, the speech recognition device 1 does not necessarily have to include the communication device 13. The speech recognition device 1 does not necessarily have to include the input device 14. The speech recognition device 1 does not necessarily have to include the output device 15. The calculation device 11, the storage device 12, the communication device 13, the input device 14, and the output device 15 may be connected via a data bus 16.

[0019] The arithmetic device 11 may include, for example, a CPU (Central Processing Unit). The arithmetic device 11 may include, for example, a GPU (Graphics Proecssing Unit) in addition to or instead of the CPU. The arithmetic device 11 may include, for example, an FPGA (Field Programmable Gate Array) in addition to or instead of at least one of the CPU and the GPU. The arithmetic device 21 reads a computer program. For example, the arithmetic device 11 may read a computer program stored in the storage device 12. For example, the arithmetic device 11 may read a computer program stored in a computer-readable and non-transitory recording medium using a recording medium reading device (for example, an input device 14 described later) included in the voice recognition device 1. The arithmetic device 11 may obtain (i.e., read) a computer program from a device (for example, a server) not shown that is arranged outside the voice recognition device 1 via the communication device 13. That is, the arithmetic device 11 may download a computer program. The arithmetic device 11 executes the read computer program. As a result, a logical functional block for executing the operation (for example, the above-mentioned voice recognition process) that the voice recognition device 1 should perform is realized in the arithmetic device 11. In other words, the arithmetic device 11 can function as a controller for realizing a logical functional block for executing the process that the voice recognition device 1 should perform.

[0020] 1 shows an example of logical functional blocks implemented in the arithmetic device 11 for executing speech recognition processing. As shown in Fig. 1, implemented in the arithmetic device 11 are a probability output unit 111, which is a specific example of "output means", and a probability update unit 112, which is a specific example of "update means".

[0021] The probability output unit 111 is capable of outputting (in other words, capable of calculating) a character probability CP based on the voice data. The character probability CP indicates the probability of a character sequence (in other words, a word sequence) corresponding to a voice sequence indicated by the voice data. More specifically, the character probability CP indicates the posterior probability P(W|X) that, when a feature of a voice sequence indicated by the voice data is X, a character sequence corresponding to the voice sequence is a character sequence W. A character sequence is a time sequence indicating the character representation of a voice sequence. For this reason, a character sequence may be referred to as a representation sequence. A character sequence may also be a series of words in which a plurality of words are linked together. In this case, a character sequence may be referred to as a word sequence.

[0022] If the audio data represents a Japanese speech sequence, the character sequence may include Kanji characters. That is, the character sequence may be a time series including Kanji characters. If the audio data represents a Japanese speech sequence, the character sequence may include Hiragana characters. That is, the character sequence may be a time series including Hiragana characters. If the audio data represents a Japanese speech sequence, the character sequence may include Katakana characters. That is, the character sequence may be a time series including Katakana characters. The character sequence may include numbers.

[0023] Note that kanji is an example of a logographic character. Therefore, the character sequence may include a logographic character. In other words, the character sequence may be a time series including a logographic character. The character sequence may include a logographic character not only when the audio data represents a Japanese audio sequence, but also when the audio data represents a audio sequence of a language other than Japanese. Furthermore, each of hiragana and katakana is an example of a phonogram. Therefore, the character sequence may include a logographic character. In other words, the character sequence may be a time series including a phonogram. The character sequence may include a logographic character not only when the audio data represents a Japanese audio sequence, but also when the audio data represents a audio sequence of a language other than Japanese.

[0024] An example of the character probability CP is shown in FIG. 2. As shown in FIG. 2, the probability output unit 111 may output a character probability CP including the probability that the character corresponding to the voice at a certain time is a specific character candidate. In the example shown in FIG. 2, the probability output unit 111 outputs (i) the probability that the character corresponding to the voice at time t is the first character candidate (in the example shown in FIG. 2, the first Chinese character "亜" having the meaning of "the second"), (ii) the probability that the character corresponding to the voice at time t is a second character candidate different from the first character candidate (in the example shown in FIG. 2, the second Chinese character "阿" added at the beginning affectionately when calling someone), (iii) the probability that the character corresponding to the voice at time t is a third character candidate different from the first to second character candidates (in the example shown in FIG. 2, the third Chinese character "愛" meaning "a heart of loving each other" and "a feeling of admiring the other party"), (iv) the probability that the character corresponding to the voice at time t is a fourth character candidate different from the first to third character candidates (in the example shown in FIG. 2, the fourth Chinese character "哀" meaning "pity"), and (v) the probability that the character corresponding to the voice at time t is a fifth character candidate different from the first to fourth character candidates (in the example shown in FIG. 2, the fifth Chinese character "藍" meaning "a kind of annual herb of the dock family"), ···, and outputs a character probability CP including them.

[0025] Furthermore, since the voice data is time-series data indicating a voice sequence, the probability output unit 111 may output a character probability CP including a probability that a character corresponding to a voice at each of a plurality of different times is a certain character candidate. In other words, the probability output unit 111 may output a character probability CP including a time series of probabilities that a character corresponding to a voice at a certain time is a certain character candidate. In the example shown in FIG. 2, the probability output unit 111 outputs (i) a time series of probabilities that a character corresponding to a voice is a first character candidate (for example, (i-1) the probability that a character corresponding to a voice at time t is the first character candidate, (i-2) the probability that a character corresponding to a voice at time t+1 following time t is the first character candidate, (i-3) the probability that a character corresponding to a voice at time t+2 following time t+1 is the first character candidate, (i-4) the probability that a character corresponding to a voice at time t+3 following time t+2 is the first character candidate, (i-5) the probability that a character corresponding to speech at time t+4 following time t+3 is the first character candidate, (i-6) the probability that a character corresponding to speech at time t+5 following time t+4 is the first character candidate, and (i-7) the probability that a character corresponding to speech at time t+6 following time t+5 is the first character candidate), (ii) a time series of the probability that a character corresponding to speech is the second character candidate (for example, (ii-1) (ii-1) the probability that the character corresponding to the speech at time t+1 following time t is the second character candidate, (ii-2) the probability that the character corresponding to the speech at time t+1 following time t is the second character candidate, (ii-3) the probability that the character corresponding to the speech at time t+2 following time t+1 is the second character candidate, (ii-4) the probability that the character corresponding to the speech at time t+3 following time t+2 is the second character candidate, (ii-5) the probability that the character corresponding to the speech at time t+4 following time t+3 is the second character candidate, (ii-6) the probability that the character corresponding to the speech at time t+5 following time t+4 is the second character candidate, and (ii-7) the probability that the character corresponding to the speech at time t+6 following time t+5 is the second character candidate), (iii) a time series of the probability that the character corresponding to the speech is the third character candidate (for example, (iii-1) the probability that the character corresponding to the speech at time t is the third character candidate, (iii-2) the probability that the character corresponding to the speech at time t+1 following time t is theThe probability that the character corresponding to the speech at time t+1 following time t is the third character candidate, (iii-3) the probability that the character corresponding to the speech at time t+2 following time t+1 is the third character candidate, (iii-4) the probability that the character corresponding to the speech at time t+3 following time t+2 is the third character candidate, (iii-5) the probability that the character corresponding to the speech at time t+4 following time t+3 is the third character candidate, (iii-6) the probability that the character corresponding to the speech at time t+5 following time t+4 is the third character candidate, and (iii-7) the probability that the character corresponding to the speech at time t+6 following time t+5 is the third character candidate), (iv) the time series of the probability that the character corresponding to the speech is the fourth character candidate (for example, (iv-1) the probability that the character corresponding to the speech at time t is the fourth character candidate, (iv-2) the probability that the character corresponding to the speech at time t+1 following time t is the fourth character candidate, (iv-3) the probability that the character corresponding to the speech at time t+2 following time t+1 is the fourth character candidate, (iv-4) the probability that the character corresponding to the speech at time t+3 following time t+2 is the fourth character candidate, (iv-5) the probability that the character corresponding to the speech at time t+4 following time t+3 is the fourth character candidate, (iv-6) the probability that the character corresponding to the speech at time t+5 following time t+4 is the fourth character candidate, and (iv-7) the probability that the character corresponding to the speech at time t+6 following time t+5 is the fourth character candidate), and (v) the time series of the probability that the character corresponding to the speech is the fifth character candidate (for example, (v-1) the probability that the character corresponding to the speech at time t is the fifth character candidate (in the example shown in FIG. 2, the fifth Chinese character "blue"), (v-2) the probability that the character corresponding to the speech at time t+1 following time t is the fifth character candidate, (v-3) the probability that the character corresponding to the speech at time t+2 following time t+1 is the fifth character candidate, (v-4) the probability that the character corresponding to the speech at time t+3 following time t+2 is the fifth character candidate, (v-5) the probability that the character corresponding to the speech at time t+4 following time t+3 is the fifth character candidate, (v-6) the probability that the character corresponding to the speech at time t+5 following time t+4 is the fifth character candidate, and (v-7) the probability that the character corresponding to the speech at time t+6 following time t+5 is the fifth character candidate),The character probability CP containing ... is output.

[0026] In the example shown in Fig. 2, in order to emphasize ease of viewing the drawing, the probability that a character corresponding to a voice at a certain time is a certain character candidate is expressed by the presence or absence of hatching of a cell indicating the probability and the density of the hatching. Specifically, in the example shown in Fig. 2, the probability is expressed by the presence or absence of hatching of a cell and the density of the hatching such that the denser the hatching of a cell, the higher the probability indicated by that cell (i.e., the lighter the hatching of a cell, the lower the probability indicated by that cell).

[0027] The speech recognition device 1 (particularly, the arithmetic unit 11) may identify a single character sequence that is most likely to be a character sequence corresponding to a voice sequence indicated by the voice data, based on the character probability CP output by the probability output unit 111. In the following description, the single character sequence that is most likely is referred to as a "maximum likelihood character sequence." In this case, the arithmetic unit 11 may include a character sequence identification unit (not shown) for identifying the maximum likelihood character sequence. The maximum likelihood character sequence identified by the character sequence identification unit may be output from the arithmetic unit 11 as a result of the speech recognition process.

[0028] For example, the voice recognition device 1 (particularly, the calculation device 11, the character sequence identification unit) may identify the character sequence with the highest character probability CP (i.e., the character sequence corresponding to the most likely path connecting the character candidates with the highest character probability CP in chronological order) as the most likely character sequence corresponding to the voice sequence indicated by the voice data. For example, in the example shown in FIG. 2, the character probability CP indicates that the character corresponding to the voice at each of the times t+1 to t+4 is most likely to be the third character candidate (the third Chinese character "love" in the example shown in FIG. 2). In this case, the voice recognition device 1 (particularly, the calculation device 11) may select the third character candidate as the most likely character (i.e., the most likely character) corresponding to the voice at each of the times t+1 to t+4. Thereafter, the voice recognition device 1 (particularly, the calculation device 11) may select the most likely character corresponding to the voice at each time by repeating the same operation at each time. As a result, the speech recognition device 1 (particularly, the arithmetic unit 11) may identify a character sequence in which the most likely characters selected at each time are arranged in chronological order as the most likely character sequence corresponding to the speech sequence indicated by the speech data. In the example shown in Fig. 2, the speech recognition device 1 (particularly, the arithmetic unit 11) identifies the character sequence "The prefectural capital of Aichi Prefecture is Nagoya City" as the most likely character sequence corresponding to the speech sequence indicated by the speech data. In this manner, the speech recognition device 1 (particularly, the arithmetic unit 11) can identify a character sequence corresponding to the speech sequence indicated by the speech data.

[0029] The probability output unit 111 can further output (in other words, can calculate) a phoneme probability PP in addition to the character probability CP based on the voice data. The phoneme probability PP indicates the probability of a phoneme sequence corresponding to a voice sequence indicated by the voice data. More specifically, the phoneme probability PP indicates the posterior probability P(S|X) that when a feature value of a voice sequence indicated by the voice data is X, a phoneme sequence corresponding to the voice sequence is a phoneme sequence S. A phoneme sequence is time-series data indicating the reading (i.e., phonemes) of a character sequence corresponding to a voice sequence. For this reason, a phoneme sequence may also be called a reading sequence or a phoneme sequence.

[0030] When the speech data indicates a Japanese speech, the phoneme sequence may include Japanese phonemes. For example, the phoneme sequence may include Japanese phonemes written using hiragana or katakana. That is, the phoneme sequence may include Japanese phonemes written using syllabaries called hiragana or katakana. Alternatively, the phoneme sequence may include Japanese phonemes written using an alphabet. That is, the phoneme sequence may include Japanese phonemes written using a phonemic alphabet called the alphabet. The Japanese phonemes written using the alphabet may include vowel phonemes including "a", "i", "u", "e", and "o". The Japanese phonemes written using the alphabet may include consonant phonemes including "k", "s", "t", "n", "h", "m", "y", "r", "g", "z", "d", "b", and "p". The Japanese phonemes written using the alphabet may include semi-vowel phonemes including "j" and "w". Japanese phonemes written using the alphabet may include special mora phonemes including "N", "Q" and "H".

[0031] An example of the phoneme probability PP is shown in Fig. 3. As shown in Fig. 3, the probability output unit 111 may output the phoneme probability PP including the probability that a phoneme corresponding to a speech at a certain time is a specific phoneme candidate. In the example shown in Fig. 3, the probability output unit 111 outputs (i) the probability that a phoneme corresponding to a speech at a time t is a first phoneme candidate (in the example shown in Fig. 3, a first phoneme "a" (in alphabetic notation, a first phoneme "a")), (ii) the probability that a phoneme corresponding to a speech at a time t is a second phoneme candidate different from the first phoneme candidate (in the example shown in Fig. 3, a second phoneme "i" (in alphabetic notation, a second phoneme "i")), (iii) the probability that a phoneme corresponding to a speech at a time t is a third phoneme candidate different from the first to second phoneme candidates (in the example shown in Fig. 3, a second phoneme "u" (in alphabetic notation, a second phoneme "u")), (iv) the probability that a phoneme corresponding to a speech at a time t is a third phoneme candidate different from the first to second phoneme candidates (in the example shown in Fig. 3, a The computer outputs phoneme probabilities PP including (i) the probability that the phoneme corresponding to the speech at time t is a third phoneme (the third phoneme “u” in alphabetic notation)), (ii) the probability that the phoneme corresponding to the speech at time t is a fourth phoneme candidate different from the first to third phoneme candidates (in the example shown in FIG. 3, the fourth phoneme “e” (the fourth phoneme “e” in alphabetic notation)), and (iii) the probability that the phoneme corresponding to the speech at time t is a fifth phoneme candidate different from the first to fourth phoneme candidates (in the example shown in FIG. 3, the fifth phoneme “o” (the fifth phoneme “o” in alphabetic notation)).

[0032] Furthermore, since the voice data is time-series data indicating a voice sequence, the probability output unit 111 may output phoneme probabilities PP including probabilities that a phoneme corresponding to a voice at each of a plurality of different times is a specific phoneme candidate. In other words, the probability output unit 111 may output character probabilities CP including a time series of probabilities that a phoneme corresponding to a voice at a certain time is a specific phoneme candidate. In the example shown in FIG. 3, the probability output unit 111 outputs (i) a time series of probabilities that a phoneme corresponding to a voice is a first character candidate (for example, (i-1) the probability that a phoneme corresponding to a voice at time t is the first phoneme candidate, (i-2) the probability that a phoneme corresponding to a voice at time t+1 following time t is the first phoneme candidate, (i-3) the probability that a phoneme corresponding to a voice at time t+2 following time t+1 is the first phoneme candidate, (i-4) the probability that a phoneme corresponding to a voice at time t+3 following time t+2 is the first phoneme candidate, (i-5) the probability that a phoneme corresponding to speech at time t+4 following time t+3 is the first phoneme candidate, (i-6) the probability that a phoneme corresponding to speech at time t+5 following time t+4 is the first phoneme candidate, and (i-7) the probability that a phoneme corresponding to speech at time t+6 following time t+5 is the first phoneme candidate), (ii) a time series of probabilities that a phoneme corresponding to speech is the second phoneme candidate (e.g., (ii-1) the probability that a phoneme corresponding to speech at time t (ii-1) the probability that a phoneme corresponding to speech at time t+1 following time t is the second phoneme candidate; (ii-2) the probability that a phoneme corresponding to speech at time t+1 following time t is the second phoneme candidate; (ii-3) the probability that a phoneme corresponding to speech at time t+2 following time t+1 is the second phoneme candidate; (ii-4) the probability that a phoneme corresponding to speech at time t+3 following time t+2 is the second phoneme candidate; (ii-5) the probability that a phoneme corresponding to speech at time t+4 following time t+3 is the second phoneme candidate. (ii-6) the probability that a phoneme corresponding to speech at time t+5 following time t+4 is the second phoneme candidate, and (ii-7) the probability that a phoneme corresponding to speech at time t+6 following time t+5 is the second phoneme candidate), (iii) a time series of the probability that a phoneme corresponding to speech is the third phoneme candidate (e.g., (iii-1) the probability that a phoneme corresponding to speech at time t is the third phoneme candidate, (iii-2) the probability that a phoneme corresponding to speech at time t+1 following time t is the third phoneme candidate,(iii-1) the probability that the phoneme corresponding to the speech at time t+2 following time t+1 is the third phoneme candidate, (iii-2) the probability that the phoneme corresponding to the speech at time t+3 following time t+2 is the third phoneme candidate, (iii-3) the probability that the phoneme corresponding to the speech at time t+4 following time t+3 is the third phoneme candidate, (iii-4) the probability that the phoneme corresponding to the speech at time t+3 following time t+2 is the third phoneme candidate, (iii-5) the probability that the phoneme corresponding to the speech at time t+4 following time t+3 is the third phoneme candidate, (iii-6) the probability that the phoneme corresponding to the speech at time t+5 following time t+4 is the third phoneme candidate, and (iii-7) the probability that the phoneme corresponding to the speech at time t+6 following time t+5 is the third phoneme candidate. (iv) a time series of probabilities that a phoneme corresponding to speech is the fourth phoneme candidate (e.g., (iv-1) the probability that a phoneme corresponding to speech at time t is the fourth phoneme candidate, (iv-2) the probability that a phoneme corresponding to speech at time t+1 following time t is the fourth phoneme candidate, (iv-3) the probability that a phoneme corresponding to speech at time t+2 following time t+1 is the fourth phoneme candidate, (iv-4) the probability that a phoneme corresponding to speech at time t+3 following time t+2 is the fourth phoneme candidate, (iv-5) the probability that a phoneme corresponding to speech at time t+4 following time t+3 is the fourth phoneme candidate, (iv-6) the probability that a phoneme corresponding to speech at time t+5 following time t+6 is the fourth phoneme candidate, (iv-7) the probability that a phoneme corresponding to speech at time t+6 following time t+7 is the fourth phoneme candidate, (iv-8) the probability that a phoneme corresponding to speech at time t+8 following time t+9 is the fourth phoneme candidate, (iv-9) the probability that a phoneme corresponding to speech at time t+9 following time t+1 is the fourth phoneme candidate, (iv-10) (iv-5) the probability that the phoneme corresponding to the voice at time t+5 following time t+4 is the fourth phoneme candidate, and (iv-6) the probability that the phoneme corresponding to the voice at time t+5 following time t+4 is the fourth phoneme candidate, and (iv-7) the probability that the phoneme corresponding to the voice at time t+6 following time t+5 is the fourth phoneme candidate), as well as (v) a time series of the probability that the phoneme corresponding to the voice is the fifth phoneme candidate (for example, (v-1) the probability that the phoneme corresponding to the voice at time t is the fifth phoneme candidate, (v-2) the probability that the phoneme corresponding to the voice at time t+1 following time t is the fifth phoneme candidate, (v-3) the probability that the phoneme corresponding to the voice at time t is the fifth phoneme candidate, The computer outputs phoneme probabilities PP including (v-1) the probability that the phoneme corresponding to the speech at time t+2 following time t+1 is the fifth phoneme candidate, (v-4) the probability that the phoneme corresponding to the speech at time t+3 following time t+2 is the fifth phoneme candidate, (v-5) the probability that the phoneme corresponding to the speech at time t+4 following time t+3 is the fifth phoneme candidate, (v-6) the probability that the phoneme corresponding to the speech at time t+5 following time t+4 is the fifth phoneme candidate, and (v-7) the probability that the phoneme corresponding to the speech at time t+6 following time t+5 is the fifth phoneme candidate), etc.

[0033] In the example shown in Fig. 3, in order to make the drawing easier to read, the probability that a phoneme corresponding to a voice at a certain time is a specific phoneme candidate is expressed by the presence or absence of hatching of a cell indicating the probability and the density of the hatching. Specifically, in the example shown in Fig. 3, the probability is expressed by the presence or absence of hatching of a cell and the density of the hatching such that the denser the hatching of a cell, the higher the probability indicated by that cell (i.e., the lighter the hatching of a cell, the lower the probability indicated by that cell).

[0034] The speech recognition device 1 (particularly, the arithmetic unit 11) may identify one phoneme sequence that is most likely to be a phoneme sequence corresponding to a speech sequence indicated by the speech data, based on the phoneme probability PP output by the probability output unit 111. In the following description, the one phoneme sequence that is most likely is referred to as a "maximum likely phoneme sequence." In this case, the arithmetic unit 11 may include a phoneme sequence identification unit (not shown) for identifying the maximum likely phoneme sequence. The maximum likely phoneme sequence identified by the phoneme sequence identification unit may be output from the arithmetic unit 11 as a result of the speech recognition process.

[0035] For example, the speech recognition device 1 (particularly, the arithmetic device 11, the phoneme sequence identification unit) may identify the phoneme sequence with the highest phoneme probability PP (i.e., the phoneme sequence corresponding to the most likely path connecting the phoneme candidates with the highest phoneme probability PP in chronological order) as the most likely phoneme sequence corresponding to the speech sequence indicated by the speech data. For example, in the example shown in FIG. 3, the phoneme probability PP indicates that the phoneme corresponding to the speech at each of the times t+1 to t+2 is most likely to be the first phoneme candidate (in the example shown in FIG. 3, the first phoneme "a" (the first phoneme "a" in alphabetic notation)). In this case, the speech recognition device 1 may select the first phoneme candidate as the most likely phoneme (i.e., the most likely phoneme) corresponding to the speech at each of the times t+1 to t+2. Furthermore, in the example shown in FIG. 3, the phoneme probability PP indicates that the phoneme corresponding to each of the speech from time t+3 to time t+4 is most likely to be the second phoneme candidate (the second phoneme "i" in the example shown in FIG. 3 (the first phoneme "i" in the alphabetical notation)). In this case, the speech recognition device 1 may select the second phoneme candidate as the most likely phoneme corresponding to each of the speech from time t+3 to time t+4. Thereafter, the speech recognition device 1 may select the most likely phoneme corresponding to the speech at each time by repeating the same operation at each time. As a result, the speech recognition device 1 may specify a phoneme sequence in which the most likely phonemes selected at each time are arranged in chronological order as the most likely phoneme sequence corresponding to the speech indicated by the speech data. 3, the speech recognition device 1 identifies the phoneme sequence "Aichi-ken-cho-sho-za-ichi-hana-yashi-desu (written in alphabetical order as ai-chi-ke-n-no-ke-n-cho-syo-za-i-chi-ha-na-go-ya-shi-de-su)" as the most likely phoneme sequence corresponding to the speech sequence indicated by the speech data. In this manner, the speech recognition device 1 can identify the phoneme sequence corresponding to the speech sequence indicated by the speech data.

[0036] In this embodiment, the probability output unit 111 outputs each of the character probability CP and the phoneme probability PP using a neural network NN. For this reason, the arithmetic device 11 may be equipped with a neural network NN. The neural network NN can output each of the character probability CP and the phoneme probability PP when voice data (for example, voice data subjected to Fourier transformation) is input. For this reason, the voice recognition device 1 of this embodiment is an end-to-end type voice recognition device.

[0037] The neural network NN may be a neural network using CTC (Connectionist Temporal Classification). The neural network using CTC may be a recurrent neural network (RNN: Recurrent Neural Network) that uses a plurality of LSTMs (Long Short Term Memories) with subwords including phonemes and characters as output units to contract the output sequences of the plurality of LSTMs. Alternatively, the neural network NN may be an Encoder-Attention-Decoder type neural network. The Encoder-Attention-Decoder type neural network is a neural network that encodes an input sequence (e.g., a voice sequence) using LSTM, and then decodes the encoded input sequence into a subword sequence (e.g., a character sequence and a phoneme sequence). However, the neural network NN may be different from the neural network using CTC and the neural network using the attention mechanism. For example, the neural network NN may be a convolutional neural network (CNN: Convolutional Neural Network). For example, the neural network NN may be a neural network that utilizes a self-attention mechanism.

[0038] The neural network NN may include a feature generator 1111, a character probability output unit 1112, and a phoneme probability output unit 1113. That is, the neural network NN may include a first network part NN1 capable of functioning as the feature generator 1111, a second network part NN2 capable of functioning as the character probability output unit 1112, and a third network part NN3 capable of functioning as the phoneme probability output unit 1113. The feature generator 1111 is capable of generating features of a voice sequence represented by the voice data based on the voice data. The character probability output unit 1112 is capable of outputting (in other words, capable of calculating) a character probability CP based on the features generated by the feature generator 1111. The phoneme probability output unit 1113 is capable of outputting (in other words, capable of calculating) a phoneme probability PP based on the features generated by the feature generator 1111.

[0039] The parameters of the neural network NN may be learned (i.e., set or determined) by a learning device 2 described later. For example, the learning device 2 may learn the parameters of the neural network NN using learning data 221 (see Figs. 10 and 11 described later) including learning voice data, a correct answer label of a character sequence corresponding to a voice sequence indicated by the learning voice data, and a correct answer label of a phoneme sequence corresponding to a voice sequence indicated by the learning voice data. The parameters of the neural network NN may include at least one of a weight by which an input value input to each node included in the neural network NN is multiplied, and a bias to be added at each node to the input value multiplied by the weight.

[0040] Note that, instead of a single neural network NN including the feature generation unit 1111, the character probability output unit 1112, and the phoneme probability output unit 1113, the probability output unit 111 may output the character probability CP and the phoneme probability PP using a neural network capable of functioning as at least one of the feature generation unit 1111, the character probability output unit 1112, and the phoneme probability output unit 1113, and a neural network capable of functioning as at least another of the feature generation unit 1111, the character probability output unit 1112, and the phoneme probability output unit 1113. In other words, the calculation device 11 may be separately equipped with a neural network capable of functioning as at least one of the feature generation unit 1111, the character probability output unit 1112, and the phoneme probability output unit 1113, and a neural network capable of functioning as at least another of the feature generation unit 1111, the character probability output unit 1112, and the phoneme probability output unit 1113. For example, the probability output unit 111 may output each of the character probability CP and the phoneme probability PP using a neural network capable of functioning as the feature generation unit 1111 and the character probability output unit 1112, and a neural network capable of functioning as the phoneme probability output unit 1113. For example, the probability output unit 111 may output each of the character probability CP and the phoneme probability PP using a neural network capable of functioning as the feature generation unit 1111, a neural network capable of functioning as the character probability output unit 1112, and a neural network capable of functioning as the phoneme probability output unit 1113.

[0041] The probability update unit 112 updates the character probability CP output by the probability output unit 111 (particularly, the character probability output unit 1112). For example, the probability update unit 112 may update the character probability CP by updating the probability that a character corresponding to a voice at a certain time is a specific character candidate. Note that "updating the probability" here may also mean "changing (in other words, adjusting) the probability."

[0042] In this embodiment, the probability update unit 112 updates the character probability CP based on the phoneme probability PP output by the probability output unit 111 (particularly, the phoneme probability output unit 1113) and the dictionary data 121. Note that the operation of updating the character probability CP based on the phoneme probability PP and the dictionary data 121 will be described in detail later with reference to Fig. 5 etc., and therefore will not be described here.

[0043] When the probability update unit 112 updates the character probability CP, it is preferable that the speech recognition device 1 (particularly, the calculation device 11) identifies the most likely character sequence based on the character probability CP updated by the probability update unit 112, instead of the character probability CP output by the probability output unit 111.

[0044] The arithmetic device 11 may further perform other processing using the result of the voice recognition processing (for example, at least one of the above-mentioned most likely character sequence and the most likely phoneme sequence). For example, the arithmetic device 11 may perform processing of translating the voice represented by the voice data into voice or characters in another language using the result of the voice recognition processing. For example, the arithmetic device 11 may perform processing of converting the voice represented by the voice data into text (so-called transcription processing) using the result of the voice recognition processing. For example, the arithmetic device 11 may perform processing of identifying a request from a speaker of the voice by performing natural language processing using the result of the voice recognition processing and responding to the request. As an example, when the request from the speaker of the voice is a request to know the weather forecast for a certain area, the arithmetic device 11 may perform processing of notifying the speaker of the weather forecast for the area.

[0045] The storage device 12 can store desired data. For example, the storage device 12 may temporarily store a computer program executed by the arithmetic device 11. The storage device 12 may temporarily store data that is temporarily used by the arithmetic device 11 when the arithmetic device 11 is executing a computer program. The storage device 12 may store data that the voice recognition device 1 stores for a long period of time. The storage device 12 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 12 may include a non-temporary recording medium.

[0046] In this embodiment, the storage device 12 stores the dictionary data 121. As described above, the dictionary data 121 is used by the probability update unit 112 to update the character probability CP. An example of the data structure of the dictionary data 121 is shown in FIG. 4. As shown in FIG. 4, the dictionary data includes at least one dictionary record 1211. In the dictionary record 1211, a character (or a character sequence) and a phoneme of the character (i.e., the reading of the character) are registered. In other words, in the dictionary record 1211, a phoneme (or a phoneme sequence) and a character corresponding to the phoneme (i.e., a character read in the reading indicated by the phoneme) are registered. For this reason, the character and the phoneme registered in the dictionary record 1211 are referred to as a "registered character" and a "registered phoneme", respectively. In this case, it can be said that the dictionary data 121 includes the dictionary record 1211 in which the registered character and the registered phoneme are associated with each other. As described in this paragraph, a registered character in this embodiment may mean not only a single character but also a character sequence including multiple characters. Similarly, a registered phoneme in this embodiment may mean not only a single phoneme but also a phoneme sequence including multiple phonemes.

[0047] In the example shown in FIG. 4, the dictionary data 121 includes: (i) a first dictionary record 1211 in which a first registered character "三密" and a first registered phoneme indicating that the reading of the first registered character is "さんみつ" are registered; (ii) a second dictionary record 1211 in which a second registered character "置き配" and a second registered phoneme indicating that the reading of the second registered character is "おきはい" are registered; and (iii) a third dictionary record 1211 in which a third registered character "脱ハンコ" and a third registered phoneme indicating that the reading of the third registered character is "だつはんこ" are registered. In other words, in the example shown in FIG. 4, the dictionary data 121 includes: (i) a first dictionary record 1211 in which a first registered phoneme "さんみつ" and a first registered character "三密" read in the reading indicated by the first registered phoneme are registered; (ii) a second dictionary record 1211 in which a second registered phoneme "おきはい" and a second registered character "置き配" read in the reading indicated by the second registered phoneme are registered; and (iii) a third dictionary record 1211 in which a third registered phoneme "だつはんこ" and a third registered character "脱ハンコ" read in the reading indicated by the third registered phoneme are registered.

[0048] The dictionary data 121 may include a dictionary record 1211 in which a character (including a character sequence) not included as a correct label in the training data 221 used to train the parameters of the neural network NN and the phoneme (including a phoneme sequence) corresponding to the character are registered as a registered character and a registered phoneme, respectively. That is, the dictionary data 121 may include a dictionary record 1211 in which a character sequence unknown to the neural network NN and a phoneme sequence corresponding to the character sequence are registered as a registered character and a registered phoneme, respectively.

[0049] The registered characters and registered phonemes may be manually registered by a user of the voice recognition device 1. That is, the user of the voice recognition device 1 may manually add the dictionary record 1211 to the dictionary data 121. Alternatively, the registered characters and registered phonemes may be automatically registered by a dictionary registration device capable of registering the registered characters and registered phonemes in the dictionary data 121. That is, the dictionary registration device may automatically add the dictionary record 1211 to the dictionary data 121.

[0050] The dictionary data 121 does not necessarily have to be stored in the storage device 12. For example, the dictionary data 121 may be recorded in a recording medium that can be read using a recording medium reading device (not shown) provided in the voice recognition device 1. The dictionary data 121 may be recorded in a device (e.g., a server) external to the voice recognition device 1.

[0051] The communication device 13 can communicate with a device outside the voice recognition device 1 via a communication network (not shown). For example, the communication device 13 may be able to communicate with an external device that stores a computer program executed by the arithmetic device 11. Specifically, the communication device 13 may be able to receive a computer program executed by the arithmetic device 11 from an external device. In this case, the arithmetic device 11 may execute the computer program received by the communication device 13. For example, the communication device 13 may be able to communicate with an external device that stores voice data. Specifically, the communication device 13 may be able to receive voice data from an external device. In this case, the arithmetic device 11 (particularly, the probability output unit 111) may output a character probability CP and a phoneme probability PP based on the voice data received by the communication device 13. For example, the communication device 13 may be able to communicate with an external device that stores dictionary data 121. Specifically, the communication device 13 may be able to receive dictionary data 121 from an external device. In this case, the calculation device 11 (particularly, the probability update unit 112) may update the character probability CP based on the dictionary data 121 received by the communication device 13.

[0052] The input device 14 is a device that accepts information input to the voice recognition device 1 from outside the voice recognition device 1. For example, the input device 14 may include an operation device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the voice recognition device 1. For example, the input device 14 may include a recording medium reading device that can read information recorded as data on a recording medium that can be attached externally to the voice recognition device 1.

[0053] The output device 15 is a device that outputs information to the outside of the voice recognition device 1. For example, the output device 15 may output information as an image. That is, the output device 15 may include a display device (so-called a display) capable of displaying an image showing the information to be output. For example, the output device 15 may output information as sound. That is, the output device 15 may include an audio device (so-called a speaker) capable of outputting sound. For example, the output device 15 may output information on paper. That is, the output device 15 may include a printing device (so-called a printer) capable of printing desired information on paper.

[0054] (1-2) Speech recognition processing by a speech recognition device Next, the flow of the voice recognition process performed by the voice recognition device 1 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the voice recognition process performed by the voice recognition device 1.

[0055] As shown in FIG. 5, the probability output unit 111 (particularly, the feature generating unit 1111) acquires voice data (step S11). For example, when the voice data is stored in the storage device 12, the probability output unit 111 may acquire the voice data from the storage device 12. For example, when the voice data is recorded on a recording medium that can be attached externally to the voice recognition device 1, the probability output unit 111 may acquire the voice data from the recording medium using a recording medium reading device (for example, the input device 14) provided in the voice recognition device 1. For example, when the voice data is recorded in a device (for example, a server) external to the voice recognition device 1, the probability output unit 111 may acquire the voice data from the external device using the communication device 13. For example, the probability output unit 111 may acquire voice data indicating the voice recorded by a recording device (that is, a microphone) capable of recording voice, using the input device 14.

[0056] Thereafter, the probability output unit 111 outputs a character probability CP based on the voice data acquired in step S11 (step S12). Specifically, the feature generation unit 1111 included in the probability output unit 111 generates features of a voice sequence indicated by the voice data based on the voice data acquired in step S11. Thereafter, the character probability output unit 1112 included in the probability output unit 111 outputs the character probability CP based on the features generated by the feature generation unit 1111.

[0057] In parallel with or before or after the process of step S12, the probability output unit 111 outputs the phoneme probability PP based on the speech data acquired in step S11 (step S13). Specifically, the feature generation unit 1111 included in the probability output unit 111 generates features of a speech sequence indicated by the speech data based on the speech data acquired in step S11. Thereafter, the phoneme probability output unit 1113 included in the probability output unit 111 outputs the phoneme probability PP based on the features generated by the feature generation unit 1111.

[0058] The phoneme probability output unit 1113 may output the phoneme probability PP using the feature used by the character probability output unit 1112 to output the character probability CP. That is, the feature generation unit 1111 may generate a common feature used to output the character probability CP and to output the phoneme probability PP. Alternatively, the phoneme probability output unit 1113 may output the phoneme probability PP using a feature different from the feature used by the character probability output unit 1112 to output the character probability CP. That is, the feature generation unit 1111 may generate the feature used to output the character probability CP and the feature used to output the phoneme probability PP separately.

[0059] Thereafter, the probability update unit 112 updates the character probability CP outputted in step S12 based on the phoneme probability PP outputted in step S13 and the dictionary data 121 (step S14).

[0060] For this reason, first, the probability update unit 112 acquires the character probability CP from the probability output unit 111 (particularly, the character probability output unit 1112). Furthermore, the probability update unit 112 acquires the phoneme probability PP from the probability output unit 111 (particularly, the phoneme probability output unit 1113). Furthermore, the probability update unit 112 acquires the dictionary data 121 from the storage device 12. Note that, when the dictionary data 121 is recorded in a recording medium that can be attached externally to the voice recognition device 1, the probability update unit 112 may acquire the dictionary data 121 from the recording medium using a recording medium reading device (e.g., the input device 14) provided in the voice recognition device 1. When the dictionary data 121 is recorded in a device (e.g., a server) external to the voice recognition device 1, the probability update unit 112 may acquire the dictionary data 121 from the external device using the communication device 13.

[0061] Then, the probability update unit 112 identifies one phoneme sequence that is most likely to correspond to the speech sequence indicated by the speech data (i.e., the most likely phoneme sequence) based on the phoneme probability PP. Note that the method for identifying the most likely phoneme sequence has already been described, and therefore a detailed description thereof will be omitted here.

[0062] Thereafter, the probability updating unit 112 determines whether or not the most likely phoneme sequence includes a registered phoneme registered in the dictionary data 121. If it is determined that the most likely phoneme sequence does not include a registered phoneme, the probability updating unit 112 does not need to update the character probability CP. In this case, the calculation device 11 identifies the most likely character sequence using the character probability CP output by the probability output unit 111. On the other hand, if it is determined that the most likely phoneme sequence includes a registered phoneme, the probability updating unit 112 updates the character probability CP. In this case, the calculation device 11 identifies the most likely character sequence using the character probability CP updated by the probability updating unit 112.

[0063] To update the character probability CP, the probability updating unit 112 may identify a time at which a registered phoneme appears in the most likely phoneme sequence. Thereafter, the probability updating unit 112 updates the character probability CP so that the probability of a registered character at the identified time becomes higher compared to before updating the character probability CP. More specifically, the probability updating unit 112 updates the character probability CP so that the posterior probability P(W|X) that a character sequence corresponding to a voice sequence at the identified time is a character sequence W including a registered character becomes higher compared to before updating the character probability CP. In other words, the probability updating unit 112 updates the character probability CP so that the probability that a registered character is included in a character sequence corresponding to a voice sequence at the identified time becomes higher compared to before updating the character probability CP.

[0064] Hereinafter, a specific example of the process of updating the character probability CP will be described with reference to FIGS.

[0065] 6 shows the most likely phonemes (i.e., phonemes with the highest phoneme probability PP) from time t to time t+8. In this case, as shown in FIG 6, the probability update unit 112 specifies the phoneme sequence "okihaiwo" as the most likely phoneme sequence.

[0066] As shown in Fig. 6, the probability updating unit 112 may select the same phoneme as the most likely phoneme at two consecutive times. In particular, when the neural network NN used by the probability output unit 111 is a neural network using CTC, the probability updating unit 112 may select the same phoneme as the most likely phoneme at two consecutive times. Not only in a case where the probability updating unit 112 specifies the most likely phoneme sequence, but also in any case where the arithmetic device 11 specifies the most likely phoneme sequence, the arithmetic device 11 may select the same phoneme as the most likely phoneme at two consecutive times. In this case, the probability updating unit 112 (the arithmetic device 11) may ignore one of the two most likely phonemes selected at two consecutive times when specifying the most likely phoneme sequence. For example, in the example shown in Figure 6, the most likely phoneme "o" is selected at each of time t and time t+1, but the probability update unit 112 (calculation device 11) may select the phoneme "o" instead of the phoneme "oo" as the phoneme at time t and time t+1 when identifying the most likely phoneme sequence.

[0067] Furthermore, as shown in Fig. 6, the probability updating unit 112 may set a blank symbol indicating that a corresponding phoneme does not exist at a certain time. In the example shown in Fig. 6, the probability updating unit 112 sets a blank symbol represented by the symbol "_" at time t+3. Note that the blank symbol may be ignored when selecting the most likely phoneme sequence.

[0068] After that, the probability updating unit 112 judges whether or not the most likely phoneme sequence "okihaiwo" contains a registered phoneme registered in the dictionary data 121 shown in Fig. 4. In the example of the dictionary data 121 shown in Fig. 4, the registered phoneme "sanmitsu", the registered phoneme "okihai", and the registered phoneme "datsuhanko" are registered in the dictionary data 121. In this case, the probability updating unit 112 judges whether or not at least one of the registered phoneme "sanmitsu", the registered phoneme "okihai", and the registered phoneme "datsuhanko" is included in the most likely phoneme sequence.

[0069] As a result, the probability updating unit 112 determines that the most likely phoneme sequence "okihaiwo" contains the registered phoneme "okihai". Therefore, in this case, the probability updating unit 112 updates the character probability CP. Specifically, the probability updating unit 112 specifies that the time at which the registered phoneme appears in the most likely phoneme sequence is from time t to time t+6. Thereafter, the probability updating unit 112 updates the character probability CP so that the probability of the registered character from the specified time t to t+6 becomes higher compared to before updating the character probability CP.

[0070] For example, Fig. 7 shows the character probability CP before the probability update unit 112 updates it. In the example shown in Fig. 7, before the probability update unit 112 updates the character probability CP, the calculation device 11 identifies an incorrect character sequence "Okihai wo" (i.e., an unnatural character sequence) as the most likely character sequence based on the character probability CP, rather than a correct character sequence "Okihai wo" (i.e., a natural character sequence). One of the reasons for identifying an incorrect character sequence in this way is that the learning data 221 used to learn the parameters of the neural network NN does not include a correct character sequence. In the example shown in Fig. 7, one of the reasons for identifying an incorrect character sequence is that the learning data 221 does not include the correct character sequence "Okihai".

[0071] In this case, the probability update unit 112 updates the character probability CP so that the probability of the character candidates included in the registered characters (i.e., the character candidate "oki", the character candidate "ki" and the character candidate "tai") is increased from time t to t+6 when the registered phoneme is included in the most likely phoneme sequence. Specifically, the probability update unit 112 may specify a path of the character candidates (probability path) such that the most likely character sequence is a character sequence including the registered character, based on the character probability CP. When there are multiple paths of the character candidates such that the most likely character sequence is a character sequence including the registered character, the probability update unit 112 may specify the most likely path from among the multiple paths. In the example shown in FIG. 7, the probability update unit 112 may specify a path of the character candidates such that the character candidate "oki" is selected from time t to time t+1, the character candidate "ki" is selected at time t+2, and the character candidate "tai" is selected from time t+5 to time t+6. Thereafter, the probability update unit 112 may update the character probability CP so that the probability corresponding to the specified path is increased compared to before updating the character probability CP. In the example shown in FIG. 7, the probability update unit 112 may update the character probability CP so that, compared to before updating the character probability CP, the probability that the character corresponding to the voice from time t to time t+1 is the character candidate "oki" becomes higher, the probability that the character corresponding to the voice at time t+2 is the character candidate "ki" becomes higher, and the probability that the character corresponding to the voice from time t+5 to time t+6 is the character candidate "hai" becomes higher. For example, the probability update unit 112 may update the character probability CP so that the character probability CP shown in FIG. 7 changes to the character probability CP shown in FIG. 8. As a result, after the probability update unit 112 updates the character probability CP, the calculation device 11 is more likely to identify the correct character sequence "okihai wo" (i.e., a natural character sequence) as the most likely character sequence, rather than the incorrect character sequence "okihai wo" (i.e., an unnatural character sequence). In other words, the calculation device 11 is more likely to identify the correct character sequence (i.e., a natural character sequence) as the most likely character sequence.

[0072] The probability update unit 112 may update the character probability CP so that the probability of the character candidate included in the registered characters is increased by a desired amount. In the example shown in Fig. 7, the probability update unit 112 may update the character probability CP so that, compared to before updating the character probability CP, the probability that the character corresponding to the voice from time t to time t+1 is the character candidate "ki" is increased by a first desired amount, the probability that the character corresponding to the voice at time t+2 is the character candidate "ki" is increased by a second desired amount that is the same as or different from the first desired amount, and the probability that the character corresponding to the voice from time t+5 to time t+6 is the character candidate "hai" is increased by a third desired amount that is the same as or different from at least one of the first desired amount and the second desired amount.

[0073] As an example, the probability update unit 112 may update the character probability CP so that the probability of a character candidate included in a registered character is increased by a predetermined amount determined according to the probability of a phoneme candidate corresponding to a registered phoneme (specifically, a registered phoneme included in the most likely phoneme sequence). Specifically, the probability update unit 112 may calculate an average value of the probabilities of the phoneme candidates corresponding to the registered phonemes. In the example shown in Fig. 6, the probability update unit 112 may calculate the average of (i) the probability that the phoneme corresponding to the voice at time t is the phoneme candidate "o" corresponding to the registered phoneme, (ii) the probability that the phoneme corresponding to the voice at time t+1 is the phoneme candidate "o" corresponding to the registered phoneme, (iii) the probability that the phoneme corresponding to the voice at time t+2 is the phoneme candidate "ki" corresponding to the registered phoneme, (iv) the probability that the phoneme corresponding to the voice at time t+4 is the phoneme candidate "ha" corresponding to the registered phoneme, the probability that the phoneme corresponding to the voice at time t+5 is the phoneme candidate "ha" corresponding to the registered phoneme, and the probability that the phoneme corresponding to the voice at time t+6 is the phoneme candidate "i" corresponding to the registered phoneme. After that, the probability update unit 112 may update the character probability CP so that the probability of the character candidate included in the registered character is increased by a desired amount determined according to the average of the calculated probabilities. For example, the probability update unit 112 may update the character probability CP so that the probability of a character candidate being included in the registered characters is increased by a desired amount equivalent to a constant multiple of the average value of the calculated probabilities.

[0074] (1-3) Technical Effects of Voice Recognition Device 1 As described above, the voice recognition device 1 of the present embodiment updates the character probability CP based on the phoneme probability PP and the dictionary data 121. Therefore, the registered characters registered in the dictionary data 121 are reflected in the character probability CP. As a result, the voice recognition device 1 is more likely to output a character probability CP capable of identifying a most likely character sequence including a registered character, compared to a case where the character probability CP is not updated based on the dictionary data 121. Therefore, the voice recognition device 1 is more likely to be able to output a character probability CP capable of identifying a correct character sequence (i.e., a natural character sequence) as the most likely character sequence, compared to a case where the character probability CP is not updated based on the dictionary data 121. In other words, the voice recognition device 1 is less likely to output a character probability CP that may identify an incorrect character sequence (i.e., an unnatural character sequence) as the most likely character sequence, compared to a case where the character probability CP is not updated based on the dictionary data 121. As a result, the voice recognition device 1 is more likely to be able to identify a correct character sequence (i.e., a natural character sequence) as the most likely character sequence, compared to a case where the character probability CP is not updated based on the dictionary data 121.

[0075] In particular, since the voice recognition device 1 updates the character probability CP based on the dictionary data 121, even if the learning data 221 for learning the parameters of the neural network NN does not include a character sequence including a registered character, it is highly likely that the voice recognition device 1 can output a character probability CP that can identify a correct character sequence (i.e., a natural character sequence) as the most likely character sequence. In other words, the voice recognition device 1 is highly likely to output a character probability CP that can identify a character sequence unknown to the neural network NN (i.e., unlearned) as the most likely character sequence. If the character probability CP is not updated based on the dictionary data 121, in order for the voice recognition device 1 to output a character probability CP that can identify a character sequence not included in the learning data 221 as the most likely character sequence, the voice recognition device 1 needs to learn the parameters of the neural network NN using the learning data 221 that includes a character sequence unknown to the neural network NN (i.e., unlearned) as a correct label. However, because the cost of learning the parameters of the neural network NN is high, it is not necessarily easy to relearn the parameters of the neural network NN. However, in this embodiment, the speech recognition device 1 can output a character probability CP capable of identifying an unknown (i.e., unlearned) character sequence to the neural network NN as the most likely character sequence, without the need to re-learn parameters of the neural network NN. In other words, the speech recognition device 1 can identify an unknown (i.e., unlearned) character sequence to the neural network NN as the most likely character sequence.

[0076] When a registered phoneme is included in the most likely phoneme sequence, the speech recognition device 1 updates the character probability CP so as to increase the probability of a character candidate constituting a registered character corresponding to the registered phoneme. Therefore, the speech recognition device 1 is more likely to be able to output a character probability CP capable of identifying a character sequence including a registered character as the most likely character sequence. In other words, the speech recognition device 1 is more likely to be able to identify a character sequence including a registered character as the most likely character sequence.

[0077] The speech recognition device 1 performs speech recognition processing using a neural network NN including a first network part NN1 capable of functioning as a feature generating unit 1111, a second network part NN2 capable of functioning as a character probability output unit 1112, and a third network part NN3 capable of functioning as a phoneme probability output unit 1113. Therefore, when introducing a neural network NN, if there is an existing neural network that includes the first network part NN1 and the second network part NN2 but does not include the third network part NN3, it is possible to construct a neural network NN by adding the third network part NN3 to the existing neural network.

[0078] (1-4) Modification of the voice recognition device 1 In the above description, the probability updating unit 112 determines whether or not the most likely phoneme sequence includes a registered phoneme registered in the dictionary data 121 in order to update the character probability CP. However, the probability updating unit 112 may further specify, in addition to the most likely phoneme sequence, at least one phoneme sequence that is second most likely to be a phoneme sequence corresponding to the speech sequence indicated by the speech data, based on the phoneme probability PP. That is, the probability updating unit 112 may specify multiple phoneme sequences that are likely to be a phoneme sequence corresponding to the speech sequence indicated by the speech data, based on the phoneme probability PP. For example, the probability updating unit 112 may specify multiple phoneme sequences using a beam search method. When multiple phoneme sequences are specified in this manner, the probability updating unit 112 may determine whether or not a registered phoneme is included in each of the multiple phoneme sequences. In this case, when it is determined that at least one of the multiple phoneme sequences contains a registered phoneme, the probability update unit 112 may identify a time at which the registered phoneme appears in at least one phoneme sequence determined to contain the registered phoneme, and update the character probability CP so that the probability of the registered character at the identified time becomes higher. As a result, the character probability CP is more likely to be updated than when it is determined whether or not a registered phoneme is included in a single most likely phoneme sequence. In other words, the registered character registered in the dictionary data 121 is more likely to be reflected in the character probability CP. As a result, the calculation device 11 is more likely to be able to output a natural most likely character sequence.

[0079] In the above description, the voice recognition device 1 performs voice recognition processing using voice data indicating a Japanese phonetic sequence. However, the voice recognition device 1 may perform voice recognition processing using voice data indicating a phonetic sequence in a language other than Japanese. Even in this case, the voice recognition device 1 may output the character probability CP and the phoneme probability PP based on the voice data, and update the character probability CP based on the phoneme probability PP and the dictionary data 121. As a result, even when the voice recognition device 1 performs voice recognition processing using voice data indicating a phonetic sequence in a language other than Japanese, it can obtain the same effect as that which can be obtained when the voice recognition device 1 performs voice recognition processing using voice data indicating a Japanese phonetic sequence.

[0080] As an example, the voice recognition device 1 may perform voice recognition processing using voice data indicating a voice sequence of a language using an alphabet (e.g., at least one of English, German, French, Spanish, Italian, Greek, and Vietnamese). In this case, the character probability CP may indicate the probability of a character sequence corresponding to an alphabet sequence (so-called spelling). More specifically, when a feature value of a voice sequence indicated by voice data is X, the character probability CP may indicate a posterior probability P(W|X) that a character sequence corresponding to the voice sequence is a character sequence W corresponding to a certain alphabet sequence. On the other hand, the phoneme probability PP may indicate the probability of a phoneme sequence corresponding to a phonetic symbol sequence. More specifically, when a feature value of a voice sequence indicated by voice data is X, the phoneme probability PP may indicate a posterior probability P(S|X) that a phoneme sequence corresponding to the voice sequence is a phoneme sequence S corresponding to a certain phonetic symbol sequence.

[0081] As another example, the voice recognition device 1 may perform the voice recognition process using voice data indicating a Chinese voice sequence. In this case, the character probability CP may indicate the probability of a character sequence corresponding to a sequence of Chinese characters. More specifically, the character probability CP may indicate a posterior probability P(W|X) that, when a feature value of a voice sequence indicated by voice data is X, a character sequence corresponding to the voice sequence is a character sequence W corresponding to a certain Chinese character sequence. Meanwhile, the phoneme probability PP may indicate the probability of a phoneme sequence corresponding to a Pinyin sequence. More specifically, the phoneme probability PP may indicate a posterior probability P(S|X) that, when a feature value of a voice sequence indicated by voice data is X, a phoneme sequence corresponding to the voice sequence is a phoneme sequence S corresponding to a certain Pinyin sequence.

[0082] In the above description, the probability output unit 111 included in the speech recognition device 1 outputs the character probability CP and the phoneme probability PP using a neural network NN including a feature generating unit 1111, a character probability output unit 1112, and a phoneme probability output unit 1113. However, as shown in Fig. 9, the probability output unit 111 may output the character probability CP and the phoneme probability PP without using a neural network NN including the feature generating unit 1111, the character probability output unit 1112, and the phoneme probability output unit 1113. In other words, the probability output unit 111 may output the character probability CP and the phoneme probability PP using any neural network capable of outputting the character probability CP and the phoneme probability PP based on speech data.

[0083] (2) Learning device 2 of this embodiment Next, the learning device 2 of this embodiment will be described. The learning device 2 performs a learning process to learn parameters of a neural network NN used by the speech recognition device 1 to output the character probability CP and the phoneme probability PP. The speech recognition device 1 outputs the character probability CP and the phoneme probability PP using the neural network NN to which the parameters learned by the learning device 2 are applied.

[0084] The configuration of such a learning device 2 will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the learning device 2 of this embodiment.

[0085] 10, the learning device 2 includes a calculation device 21 and a storage device 22. Furthermore, the learning device 2 may include a communication device 23, an input device 24, and an output device 25. However, the learning device 2 does not have to include the communication device 23. The learning device 2 does not have to include the input device 24. The learning device 2 does not have to include the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0086] The arithmetic device 21 may include, for example, a CPU. The arithmetic device 21 may include, for example, a GPU in addition to or instead of the CPU. The arithmetic device 21 may include, for example, an FPGA in addition to or instead of at least one of the CPU and the GPU. The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable and non-transient recording medium using a recording medium reading device (for example, an input device 24 described later) provided in the learning device 2. The arithmetic device 21 may obtain (i.e., read) a computer program from a device (for example, a server) not shown that is disposed outside the learning device 2 via the communication device 23. That is, the arithmetic device 21 may download a computer program. The arithmetic device 21 executes the read computer program. As a result, a logical function block for executing an operation to be performed by the learning device 2 (for example, the above-mentioned learning process) is realized in the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the processing that the learning device 2 should perform.

[0087] Fig. 10 shows an example of logical functional blocks realized in the arithmetic device 21 to execute the learning process. As shown in Fig. 10, a learning data acquisition unit 211, which is a specific example of "acquisition means", and a learning unit 212, which is a specific example of "learning means", are realized in the arithmetic device 21.

[0088] The learning data acquisition unit 211 acquires learning data 221 used to learn parameters of the neural network NN. For example, when the learning data 221 is stored in the storage device 22 as shown in FIG. 10, the learning data acquisition unit 211 may acquire the learning data 221 from the storage device 22. For example, when the learning data 221 is recorded in a recording medium that can be attached externally to the learning device 2, the learning data acquisition unit 211 may acquire the learning data 221 from the recording medium using a recording medium reading device (for example, the input device 24) provided in the learning device 2. For example, when the learning data 221 is recorded in a device (for example, a server) external to the learning device 2, the learning data acquisition unit 211 may acquire the learning data 221 from the external device using the communication device 23.

[0089] An example of the data structure of the training data 221 is shown in Fig. 11. As shown in Fig. 11, the training data 221 includes at least one training record 2211. The training record 2211 includes training speech data, a correct answer label of a character sequence corresponding to a speech sequence indicated by the training speech data, and a correct answer label of a phoneme sequence corresponding to a speech sequence indicated by the training speech data.

[0090] The learning unit 212 learns parameters of the neural network NN using the learning data 221 acquired by the learning data acquisition unit 211. As a result, the learning unit 212 can build a neural network NN that can output appropriate character probabilities CP and appropriate phoneme probabilities PP when voice data is input.

[0091] Specifically, the learning unit 212 inputs the training voice data included in the training data 221 to the neural network NN (or a training neural network simulating a neural network NN, the same applies below). As a result, the neural network NN outputs character probabilities CP, which are the probabilities of a character sequence corresponding to a voice sequence indicated by the training voice data, and phoneme probabilities PP, which are the probabilities of a phoneme sequence corresponding to a voice sequence indicated by the training voice data. As described above, the most likely character sequence is identified from the character probabilities CP, and the most likely phoneme sequence is identified from the phoneme probabilities PP, so that the neural network NN may be regarded as essentially outputting the most likely character sequence and the most likely phoneme sequence.

[0092] Thereafter, the learning unit 212 adjusts the parameters of the neural network NN using a loss function based on a character sequence error, which is the error between the most likely character sequence output by the neural network NN and the correct label of a character sequence included in the training data 221, and a phoneme sequence error, which is the error between the most likely phoneme sequence output by the neural network NN and the correct label of a phoneme sequence included in the training data 221. For example, when a loss function that decreases as the character sequence error and the phoneme sequence error become smaller is used, the learning unit 212 may adjust the parameters of the neural network NN so that the loss function becomes smaller (preferably, minimized).

[0093] The learning unit 212 may adjust the parameters of the neural network NN using an existing algorithm for learning the parameters of the neural network NN. For example, the learning unit 212 may adjust the parameters of the neural network NN using the backpropagation method.

[0094] As described above, the neural network NN may include the first network part NN1 capable of functioning as the feature generating unit 1111, the second network part NN2 capable of functioning as the character probability output unit 1112, and the third network part NN3 capable of functioning as the phoneme probability output unit 1113. In this case, the learning unit 212 may learn at least one parameter of the first network part NN1 to the third network part NN3 after learning at least one parameter of the first network part NN1 to the third network part NN3 while the learned parameter is fixed. For example, the learning unit 212 may learn the parameters of the first network part NN1 and the second network part NN2 after learning the parameters of the third network part NN3 while the learned parameters are fixed. Specifically, the learning unit 212 may learn the parameters of the first network part NN1 and the second network part NN2 using the learning voice data and the correct answer label of the character sequence in the learning data 221. Thereafter, the learning unit 212 may learn parameters of the third network part NN3 using the learning voice data and the correct label of the phoneme sequence in the learning data 221 while fixing the parameters of the first network part NN1 and the second network part NN2. In this case, when introducing the neural network NN, if there is an existing neural network that includes the first network part NN1 and the second network part NN2 but does not include the third network part NN3, the learning device 2 can learn the parameters of the existing neural network and the third network part NN3 separately. After learning the parameters of the existing neural network, the learning device 2 can selectively learn the parameters of the third network part NN3 in a state where the third network part NN3 is added to the existing trained neural network.

[0095] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the learning device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM, a ROM, a hard disk device, a magneto-optical disk device, an SSD, and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0096] The communication device 23 can communicate with a device external to the learning device 2 via a communication network (not shown). For example, the communication device 23 may be capable of communicating with an external device that stores a computer program executed by the arithmetic device 21. Specifically, the communication device 23 may be capable of receiving a computer program executed by the arithmetic device 21 from an external device. In this case, the arithmetic device 21 may execute the computer program received by the communication device 23. For example, the communication device 23 may be capable of communicating with an external device that stores the learning data 221. Specifically, the communication device 23 may be capable of receiving the learning data 221 from the external device.

[0097] The input device 24 is a device that accepts information input to the learning device 2 from outside the learning device 2. For example, the input device 24 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the learning device 2. For example, the input device 24 may include a recording medium reading device that can read information recorded as data on a recording medium that can be attached externally to the learning device 2.

[0098] Output device 25 is a device that outputs information to the outside of learning device 2. For example, output device 25 may output information as an image. That is, output device 25 may include a display device (so-called a display) capable of displaying an image showing the information to be output. For example, output device 25 may output information as sound. That is, output device 25 may include an audio device (so-called a speaker) capable of outputting sound. For example, output device 25 may output information on paper. That is, output device 25 may include a printing device (so-called a printer) capable of printing desired information on paper.

[0099] The voice recognition device 1 may also function as the learning device 2. For example, the arithmetic device 11 of the voice recognition device 1 may include a learning data acquisition unit 211 and a learning unit 212. In this case, the voice recognition device 1 may learn parameters of the neural network NN.

[0100] (3) Additional notes The following supplementary notes are further disclosed regarding the above-described embodiment. [Appendix 1] an output means for outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network that outputs the first probability and the second probability; an update means for updating the first probability based on dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and the second probability; A voice recognition device comprising: [Appendix 2] When the phoneme sequence includes the registered phoneme, the update means updates the first probability so that the probability that the registered character is included in the character sequence becomes higher than before updating the first probability. 2. A speech recognition device according to claim 1. [Appendix 3] The neural network comprises: a first network portion configured to output features of the speech sequence when the speech data is input; a second network portion that outputs the first probability when the feature is input; a third network portion that outputs the second probability when the feature is input; 3. The speech recognition device according to claim 1 or 2, [Appendix 4] an acquiring means for acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; a learning means for learning parameters of a neural network that outputs, when second voice data is input, a first probability which is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability which is a probability of a second phoneme sequence corresponding to the second voice sequence, using the learning data; A learning device comprising: [Appendix 5] The neural network comprises: a first model that outputs a feature quantity of the second speech sequence when the second speech data is input; a second model that outputs the first probability when the feature is input; a third model that outputs the second probability when the feature is input; Including, The learning means learns parameters of the first and second models using the first speech data and the correct label of the first character sequence in the training data, and thereafter learns parameters of the third model using the first speech data and the correct label of the first phoneme sequence in the training data. 5. A learning device as described in appendix 4. [Appendix 6] outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network which outputs the first probability and the second probability; The first probability is updated based on dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and the second probability. Speech recognition methods. [Appendix 7] acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; Using the learning data, parameters of a neural network are learned that outputs, when second voice data is input, a first probability that is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability that is a probability of a second phoneme sequence corresponding to the second voice sequence. How to learn. [Appendix 8] On the computer, outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network which outputs the first probability and the second probability; The first probability is updated based on dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and the second probability. A recording medium on which a computer program for executing a speech recognition method is recorded. [Appendix 9] On the computer, acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; Using the learning data, parameters of a neural network are learned that outputs, when second voice data is input, a first probability that is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability that is a probability of a second phoneme sequence corresponding to the second voice sequence. A recording medium on which a computer program for executing a learning method is recorded. [Appendix 10] On the computer, outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network which outputs the first probability and the second probability; The first probability is updated based on dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and the second probability. A computer program product that causes a speech recognition method to be performed. [Appendix 11] On the computer, acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; Using the learning data, parameters of a neural network are learned that outputs, when second voice data is input, a first probability that is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability that is a probability of a second phoneme sequence corresponding to the second voice sequence. A computer program that implements the learning method.

[0101] At least a part of the constituent elements of each of the above-mentioned embodiments can be appropriately combined with at least a part of the other constituent elements of each of the above-mentioned embodiments. Some of the constituent elements of each of the above-mentioned embodiments may not be used. In addition, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure above are incorporated by reference as part of the description of this disclosure.

[0102] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. The voice recognition device, the voice recognition method, the learning device, the learning method, and the recording medium that involve such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0103] 1. Voice recognition device 11 Arithmetic unit 111 Probability Output Unit 1111 Feature Generation Unit 1112 Character Probability Output Unit 1113 Phoneme Probability Output Unit 12 Storage device 121 Dictionary Data 1211 Dictionary Records 2 Learning device 21 Arithmetic unit 211 Learning Data Acquisition Unit 212 Learning Department 22 Storage device 221 Training Data NN Neural Network NN1 First network part NN2 Second network part NN3 Third network part CP Character Probability PP Phoneme Probability

Claims

1. an output means for outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network; and an updating means for identifying one or more phoneme sequences likely to be the phoneme sequence corresponding to the voice sequence indicated by the voice data based on the second probability, determining whether or not each of the one or more phoneme sequences includes a registered phoneme included in dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and, when it is determined that at least one of the one or more phoneme sequences includes the registered phoneme, identifying the registered character associated with the registered phoneme determined to be included in the dictionary data, and updating the probability that the identified registered character corresponds to the voice data so as to be higher, thereby updating the first probability; A voice recognition device comprising:

2. The neural network comprises: a first network portion configured to output features of the speech sequence when the speech data is input; a second network portion that outputs the first probability when the feature is input; a third network portion that outputs the second probability when the feature is input; 2. The speech recognition device of claim 1, comprising:

3. an acquiring means for acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; a learning means for learning parameters of a neural network that outputs, when second voice data is input, a first probability which is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability which is a probability of a second phoneme sequence corresponding to the second voice sequence, using the learning data; A learning device comprising:

4. The neural network comprises: a first model that outputs a feature quantity of the speech sequence when the second speech data is input; a second model that outputs the first probability when the feature is input; a third model that outputs the second probability when the feature is input; Including, The learning means learns parameters of the first and second models using the first voice data and the correct label of the first character sequence in the learning data, and thereafter learns parameters of the third model using the first voice data and the correct label of the first phoneme sequence in the learning data. The learning device according to claim 3 .

5. outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network which outputs the first probability and the second probability; based on the second probability, identifying one or more phoneme sequences likely to be the phoneme sequence corresponding to the voice sequence indicated by the voice data, determining whether or not each of the one or more phoneme sequences includes a registered phoneme included in dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and when it is determined that at least one of the one or more phoneme sequences includes the registered phoneme, identifying the registered character associated with the registered phoneme determined to be included in the dictionary data, and updating the probability that the identified registered character corresponds to the voice data so as to be higher, thereby updating the first probability. Speech recognition methods.

6. An acquisition means for acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; a learning means for learning parameters of a neural network that outputs, when second voice data is input, a first probability which is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability which is a probability of a second phoneme sequence corresponding to the second voice sequence, using the learning data; A learning method performed by a learning device comprising: acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; Using the learning data, parameters of a neural network are learned that outputs, when second voice data is input, a first probability that is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability that is a probability of a second phoneme sequence corresponding to the second voice sequence. How to learn.

7. On the computer, outputting, when voice data is input, a first probability which is a probability of a character sequence corresponding to a voice sequence indicated by the voice data, and a second probability which is a probability of a phoneme sequence corresponding to the voice sequence, using a neural network which outputs the first probability and the second probability; based on the second probability, identifying one or more phoneme sequences likely to be the phoneme sequence corresponding to the voice sequence indicated by the voice data, determining whether or not each of the one or more phoneme sequences includes a registered phoneme included in dictionary data in which a registered character is associated with a registered phoneme that is a phoneme of the registered character, and when it is determined that at least one of the one or more phoneme sequences includes the registered phoneme, identifying the registered character associated with the registered phoneme determined to be included in the dictionary data, and updating the probability that the identified registered character corresponds to the voice data so as to be higher, thereby updating the first probability. A computer program product that causes a speech recognition method to be performed.

8. On the computer, acquiring training data including first speech data for training, a correct answer label of a first character sequence corresponding to a first speech sequence indicated by the first speech data, and a correct answer label of a first phoneme sequence corresponding to the first speech sequence; Using the learning data, parameters of a neural network are learned that outputs, when second voice data is input, a first probability that is a probability of a second character sequence corresponding to a second voice sequence indicated by the second voice data, and a second probability that is a probability of a second phoneme sequence corresponding to the second voice sequence. A computer program that implements the learning method.

Images