Speech recognition device, machine learning method, and program

Abstract

The present disclosure is a speech recognition device comprising: a symbolic-distributed representation series conversion unit that receives input of a symbol feature amount series including a symbol representing redundancy, converts said symbol feature amount series into feature amounts of corresponding continuous values, thereafter converts the feature amounts into an intermediate character feature amount series by using a first neural network, and outputs the intermediate character feature amount series; a label estimation unit that receives input of the intermediate character feature amount series and then performs label estimation by using a second neural network so as to output an output probability distribution that is a two-dimensional matrix; an accuracy rate calculation unit that receives input of the output probability distribution and a correct answer symbol series, and outputs an accuracy rate using an argument that takes the maximum value of each prescribed vector of the matrix as the output of a third neural network; and a symbol sampling unit that receives input of the output probability distribution, the correct answer symbol series, a threshold, and the accuracy rate, outputs a new correct answer symbol series, and causes the first through third neural networks to perform machine learning by means of an RNNT, using the new correct answer symbol series as the original correct answer symbol series.

Description

[Technical Field]

[0001] The present disclosure relates to a speech recognition technology using a neural transducer type speech recognition model that directly outputs any character string (for example, phonemes, characters, subwords, or words) represented from input speech. [Background technology]

[0002] Recent speech recognition methods using neural networks are capable of directly outputting word sequences from speech (acoustic) features. Non-Patent Document 1 discloses a method for training a neural network for speech recognition using a recurrent neural network transducer (RNNT) (see the section "Recurrent Neural Network Transducer" in Non-Patent Document 1). By introducing a "blank" symbol (referred to as "null output" in Non-Patent Document 1) representing redundancy, RNNT model training makes it possible to dynamically learn correspondences between speech and output sequences from training data, provided only that character strings (e.g., phonemes, characters, subwords, and word sequences (≠ frame-by-frame)) corresponding to the speech content are available. In other words, training is possible using features and labels with an incompatible relationship between input length T and output length U (generally, T>>U). Furthermore, because it can operate frame-by-frame during inference, it has attracted considerable attention as a technology capable of streaming operation.

[0003] An end-to-end speech recognition device (see Non-Patent Document 1) that outputs any character sequence directly from these acoustic features is shown as a conventional technology in Fig. 9. As shown in Fig. 9, the conventional speech recognition device includes an input unit 130, a speech denuded representation sequence converter 131, a symbol denuded representation sequence converter 132, a label estimator 133, and an RNNT loss calculator 134.

[0004] The speech densified representation sequence converter 131 receives an acoustic feature sequence X from the input unit 130 and outputs an output intermediate acoustic feature sequence H. The symbol densified representation sequence converter 132 receives a symbol sequence Y 1:U (length U) is input and an intermediate character feature sequence C (length U) is output.

[0005] The label estimation unit 133 receives the intermediate acoustic feature sequence H and the intermediate character feature sequence C and calculates an output probability distribution

[0006]

number

[0007] The RNNT loss calculation unit 134 calculates the output probability distribution

[0008]

number

[0009]

number

[0010] Generally, RNNTs are trained using RNNT loss, assuming that the tensor is a three-dimensional tensor (T × U × K: time length × number of tokens included in the correct label × number of token entries). However, during inference, there is no expansion operation, so the output is a two-dimensional matrix.

[0011] However, in the technology of Non-Patent Document 1, when training the RNNT, the correct label is input to the module that processes linguistic information, but when inferring, the output results of the RNNT itself are input recursively, which creates a gap with the training time and causes a problem of deterioration in recognition performance.

[0012] In response to this, Non-Patent Document 2 proposes a method (Scheduled Sampling) that eliminates the gap between training and inference in another speech recognition model, and reports that it has improved recognition performance. Scheduled Sampling, as described in Non-Patent Document 2, is a method used for a speech recognition model called Encoder-Decoder, and is effective for machine learning models whose output during training is two-dimensional (U×K: number of tokens included in correct utterance × number of targets (≒ vocabulary size)). [Prior art documents] [Non-patent literature]

[0013] [Non-Patent Document 1] Alex Graves, "Sequence transduction with recurrent neural networks," in Proc. of ICML, 2012. [Non-patent document 2] S. Bengio et. al., "Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks," in Advances in NeurIPS, 2015. Summary of the Invention [Problem to be solved by the invention]

[0014] However, as described above, the RNNT of Non-Patent Document 1 has a three-dimensional output during learning, making it difficult to apply to the technique of Non-Patent Document 2, which is effective for machine learning models with a two-dimensional output during learning.

[0015] The present invention has been made in view of the above points, and aims to reduce errors in speech recognition by narrowing the gap between training and inference. [Means for solving the problem]

[0016] In order to solve the above problem, the invention of claim 1 is a speech recognition device for training a neural network for speech recognition, comprising: a symbol distributed representation sequence conversion unit that receives an input of a symbol feature sequence including symbols representing redundancy, converts the input into corresponding continuous-value features, and then converts the converted input into an intermediate character feature sequence using a first neural network, and outputs the intermediate character feature sequence; a label estimation unit that receives an input of the intermediate character feature sequence, performs label estimation using a second neural network, and outputs an output probability distribution, which is a two-dimensional matrix; an accuracy rate calculation unit that receives an input of the output probability distribution and a correct symbol sequence, and outputs an accuracy rate as an output of a third neural network, an argument that takes the maximum value in each predetermined-th vector of the matrix; and a symbol sampling unit that receives an input of the output probability distribution, the correct symbol sequence, a threshold, and the accuracy rate, outputs a new correct symbol sequence, and performs machine learning of the first to third neural networks using an RNNT (Recurrent Neural Network Transducer) with the new correct symbol sequence as the original correct symbol sequence. [Effects of the Invention]

[0017] As described above, according to the present invention, by narrowing the gap between learning and inference, it is possible to suppress recognition errors. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of a communication system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram illustrating an electrical hardware configuration of the voice recognition device. [Figure 3] FIG. 2 is a diagram illustrating an electrical hardware configuration of a communication terminal. [Figure 4] FIG. 2 is a functional configuration diagram of the speech recognition device according to the first embodiment in a learning phase. [Figure 5] 4 is a flowchart showing processing or operations in a learning phase of the voice recognition device according to the first embodiment. [Figure 6] FIG. 10 is a functional configuration diagram of a speech recognition device according to a second embodiment in a learning phase. [Figure 7] 10 is a flowchart showing processing or operations in a learning phase of a voice recognition device according to a second embodiment. [Figure 8] 10 is a flowchart showing processing or operations in a learning phase of a voice recognition device according to a second embodiment. [Figure 9] FIG. 1 is a functional configuration diagram of an end-to-end speech recognition device according to the prior art that outputs an arbitrary character sequence directly from speech features. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0020] [System configuration of the embodiment] First, an outline of the configuration of a communication system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of a communication system according to an embodiment of the present invention.

[0021] 1, a communication system 1 of this embodiment is constructed by a voice recognition device 3 and a communication terminal 5. The communication terminal 5 is managed and used by a user.

[0022] Furthermore, the voice recognition device 3 and the communication terminal 5 can communicate with each other via a communication network 100 such as the Internet. The communication network 100 may be connected wirelessly or by wire.

[0023] The speech recognition device 3 is composed of one or more computers. When the speech recognition device 3 is composed of multiple computers, it may be referred to as a "speech recognition device" or a "speech recognition system." The speech recognition device 3 is a device that performs machine learning to create a neural transducer-type speech recognition model that directly outputs any character string (e.g., phonemes, characters, subwords, words) represented by input speech.

[0024] The communication terminal 5 is a computer, and a notebook computer is shown as an example in Fig. 1. In Fig. 1, a user operates the communication terminal 5. Note that the speech recognition device 3 may perform processing independently without using the communication terminal 5.

[0025] [Hardware configuration] <Hardware configuration of the speech recognition device> Next, the electrical hardware configuration of the voice recognition device 3 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of the voice recognition device.

[0026] As shown in FIG. 2, the speech recognition device 3 is a computer and includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, an SSD (Solid State Drive) 304, an external device connection I / F (Interface) 305, a network I / F 306, a media I / F 309, and a bus line 310.

[0027] Of these, the CPU 301 controls the overall operation of the speech recognition device 3. The ROM 302 stores programs such as an IPL (Initial Program Loader) used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301.

[0028] The SSD 304 reads or writes various data under the control of the CPU 301. Note that instead of the SDD 304, a HDD (Hard Disk Drive) may be used.

[0029] The external device connection I / F 305 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0030] The network I / F 306 is an interface for performing data communication via the communication network 100 .

[0031] The media I / F 309 controls reading and writing (storing) of data from and to a recording medium 309m such as a flash memory, etc. The recording medium 309m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0032] The bus line 310 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 301 shown in FIG.

[0033] <Hardware configuration of communication terminal> Next, the electrical hardware configuration of the communication terminal 5 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the electrical hardware configuration of the communication terminal.

[0034] As shown in FIG. 3, the communication terminal 5 is a computer and includes a CPU 501, a ROM 502, a RAM 503, an SSD 504, an external device connection I / F (Interface) 505, a network I / F 506, a display 507, a pointing device 508, a media I / F 509, and a bus line 510.

[0035] Of these, the CPU 501 controls the overall operation of the communication terminal 5. The ROM 502 stores programs such as IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501.

[0036] The SSD 504 reads or writes various data under the control of the CPU 501. Note that instead of the SSD 504, an HDD (Hard Disk Drive) may be used.

[0037] The external device connection I / F 505 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB memory, and a printer.

[0038] The network I / F 506 is an interface for performing data communication via the communication network 100 .

[0039] The display 507 is a type of display means such as a liquid crystal display or organic EL (Electro Luminescence) display that displays various images.

[0040] The pointing device 508 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. If user Y uses a keyboard, the function of the pointing device 508 may be turned off.

[0041] The media I / F 509 controls reading and writing (storing) of data from and to a recording medium 509m such as a flash memory, etc. The recording medium 509m includes DVDs, Blu-ray Discs (registered trademarks), etc.

[0042] The bus line 510 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 501 shown in FIG.

[0043] First embodiment [Functional configuration of the speech recognition device according to the first embodiment] Next, the functional configuration of the speech recognition device 3a according to the first embodiment in the learning phase will be described with reference to Fig. 4. Fig. 4 is a functional configuration diagram of the speech recognition device according to the first embodiment in the learning phase. The speech recognition device 3a is an example of the speech recognition device 3 shown in Fig. 1. The speech recognition device 3a obtains a two-dimensional output from the RNNT in order to apply the technology disclosed in Non-Patent Document 2.

[0044] 4, the speech recognition device 3a includes an input unit 30, a symbol distributed representation sequence conversion unit 32, a label estimation unit 33a, an accuracy rate calculation unit 41, and a symbol sampling unit 42. The symbol distributed representation sequence conversion unit 32 includes a first neural network, the label estimation unit 33a includes a second neural network, and the accuracy rate calculation unit 41 includes a third neural network.

[0045] Among these, the input unit 30 receives input information. In this embodiment, the input information is a symbol feature sequence Y 1:U The symbol is "blank" (referred to as "null output" in Non-Patent Document 1) which indicates redundancy. As an input method, the user may input directly to the speech recognition device 3, or the user may input indirectly from a communication terminal.

[0046] The symbol distributed representation sequence converter 32 converts the symbol feature sequence Y 1:U(length U) and outputs an intermediate character feature sequence C (length U). Therefore, the symbol distributed representation sequence converter 32 converts the input symbol feature sequence Y 1:U The symbol distributed representation sequence converter 32 converts the input information (length U) into a one-hot vector, and then converts it into an intermediate character feature sequence using a multi-stage first neural network.

[0047] The label estimation unit 33a receives the intermediate character feature sequence C and calculates the output probability distribution

[0048]

number

[0049]

number

[0050] The accuracy rate calculation unit 41 calculates the output probability distribution

[0051]

number

[0052]

number

[0053] The symbol sampling unit 42 calculates the output probability distribution

[0054]

number

[0055] (per token) If the random number ρ generated from the continuous uniform distribution is below the threshold λ, the symbol sampling unit 42 converts the u-th correct label into the actual u-th output of the learning model as shown in (Equation 2).

[0056]

number

[0057]

number

[0058] (For each utterance) When processing for each utterance (level), if the random number ρ generated from the continuous uniform distribution falls below the value obtained by multiplying the accuracy rate Acc obtained from the accuracy rate calculation unit 201 by the threshold λ, the symbol sampling unit 42 converts the correct label into the actual output of the machine learning model (third neural network) as shown in (Equation 3).

[0059]

number

[0060]

number

[0061]

number

[0062] (Common processing for both token-based and utterance-based processing) The symbol sampling unit 42 samples the obtained new correct symbol sequence Y' 1:U is used as the original correct symbol sequence, and machine learning is performed by updating the parameters of the first to third neural networks using the technique of Non-Patent Document 1. In Fig. 4, the arrow from the symbol sampling unit 42 to the symbol feature sequence indicates recursive processing in the case of the token level.

[0063] [Processing or Operation of the First Embodiment] Next, the processing or operation in the learning phase of the speech recognition device according to the first embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing or operation in the learning phase of the speech recognition device according to the first embodiment.

[0064] S11: The input unit 30 receives input information (symbol feature sequence).

[0065] S12: The symbol distributed representation sequence converter 32 converts the input symbols into corresponding continuous value feature quantities.

[0066] S13: The label estimation unit 33a performs label estimation based on the intermediate character feature sequence C (length U) using a neural network.

[0067] S14: The accuracy rate calculation unit 41 calculates the matrix

[0068]

number

[0069] S15: The symbol sampling unit 42 calculates the output probability distribution

[0070]

number

[0071] [Major Effects of the First Embodiment] As described above, according to the first embodiment, in order to apply Scheduled Sampling disclosed in Non-Patent Document 2 to RNNT, the same output format (two-dimensional matrix: U×K) as that of the Encoder-Decoder is obtained during training. This reduces the gap between training and inference, thereby achieving the effect of suppressing recognition errors. It is possible.

[0072] Second embodiment [Functional configuration of the speech recognition device according to the second embodiment] Next, the functional configuration of a voice recognition device 3b according to the second embodiment will be described with reference to Fig. 6. Fig. 6 is a functional configuration diagram of the voice recognition device according to the second embodiment in the learning phase.

[0073] 6, the speech recognition device 3a includes an input unit 30, a speech distributed representation sequence conversion unit 31, a symbol distributed representation sequence conversion unit 32, a label estimation unit 33b, an accuracy rate calculation unit 41, a symbol sampling unit 42, and an alignment extraction unit 43. The symbol distributed representation sequence conversion unit 32 includes a first neural network, the label estimation unit 33a includes a second neural network, the accuracy rate calculation unit 41 includes a third neural network, and the speech distributed representation sequence conversion unit 31 includes a fourth neural network. Functional components having the same functions as those in the first embodiment are denoted by the same reference numerals, and a description thereof will be omitted. The input information in this embodiment is a speech feature sequence X and a symbol feature sequence Y. 1:U The speech feature sequence X may be referred to as an acoustic feature sequence X. Furthermore, an intermediate speech feature sequence H, which will be described later, may be referred to as an intermediate acoustic feature sequence H.

[0074] Of these, the speech densified representation sequence conversion unit 31 receives an input of a speech (acoustic) feature sequence X and outputs an intermediate speech (acoustic) feature sequence H. Therefore, the speech densified representation sequence conversion unit 31 plays the role of an encoder that converts the input acoustic feature sequence X into an intermediate acoustic feature sequence H using a multi-stage fourth neural network. Note that the neural network held by the symbol densified representation sequence conversion unit 32 in the second embodiment is the first neural network.

[0075] The label estimation unit 33b receives the intermediate acoustic feature sequence H and the intermediate character feature sequence C and calculates the output probability distribution

[0076]

number

[0077]

number

[0078] The alignment extraction unit 43 calculates the output probability distribution

[0079]

number

[0080]

number

[0081]

number

[0082]

number

[0083]

number

[0084]

number

[0085] [Processing or Operation of the Second Embodiment] Next, the processing or operation in the learning phase of the speech recognition device according to the second embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 and Fig. 8 are flowcharts showing the processing or operation in the learning phase of the speech recognition device according to the second embodiment.

[0086] S21: The input unit 30 receives input information (a speech feature sequence and a symbol feature sequence).

[0087] S22: The speech distributed representation sequence converter 31 converts the input acoustic features into intermediate acoustic features using a multi-stage neural network.

[0088] S23: The symbol distributed representation sequence converter 32 converts the input symbols into corresponding continuous value feature quantities.

[0089] S24: The label estimation unit 33b performs label estimation based on the intermediate acoustic feature sequence H and the intermediate character feature sequence C (length U) using a neural network.

[0090] S25: The alignment extraction unit 43 uses a forward-backward algorithm similar to the gradient calculation disclosed in Non-Patent Document 1 to calculate the three-dimensional tensor

[0091]

number

[0092]

number

[0093] S26: The alignment extraction unit 43 extracts a vector according to the obtained time sequence and the corresponding token.

[0094]

number

[0095]

number

[0096] S27: The accuracy rate calculation unit 41 calculates the matrix

[0097]

number

[0098] S28: The symbol sampling unit 42 calculates the output probability distribution

[0099]

number

[0100] [Major Effects of the Second Embodiment] As described above, according to the second embodiment, a two-dimensional output (two-dimensional matrix: U×K) is obtained from a three-dimensional output (three-dimensional tensor: T×U×K) during learning by using the output of the RNNT itself during learning. This reduces the gap between learning and inference, thereby achieving the effect of suppressing recognition errors.

[0101] 〔supplement〕 The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations). (1) The speech recognition device 3 can be realized by a computer and a program, but this program can also be recorded on a (non-transitory) recording medium or provided via the communication network 100. (2) In the above embodiment, a notebook computer is shown as an example of a communication terminal 5, but this is not limited to this and may be, for example, a desktop computer, a tablet terminal, a smartphone, a smartwatch, a car navigation device, a refrigerator, a microwave oven, etc. (3) Each of the CPUs 301 and 501 serving as a processor may be a single processor or may be a multiple processor. [Explanation of symbols]

[0102] 1. Communication Systems 3. Voice recognition device 5. Communication terminals 30 Input section 31 Speech Distributed Representation Serial Conversion Unit 32 Symbol distributed representation sequence converter 33a Label estimation unit 33b Label estimation unit 41 Accuracy rate calculation section 42 Symbol sampling section 43 Alignment Extraction Unit

Claims

1. A speech recognition device that trains a neural network for speech recognition, a symbol distributed representation sequence conversion unit that receives a symbol feature sequence including a symbol representing redundancy, converts the received symbol feature sequence into a corresponding continuous value feature, and then converts the intermediate character feature sequence into an intermediate character feature sequence using a first neural network, and outputs the intermediate character feature sequence; a label estimation unit that receives the intermediate character feature sequence, performs label estimation using a second neural network, and outputs an output probability distribution that is a two-dimensional matrix; a correct answer rate calculation unit that receives the output probability distribution and the correct answer symbol sequence, and outputs a correct answer rate as an output of a third neural network, the argument that takes the maximum value in each predetermined-th vector of the matrix; a symbol sampling unit that receives the output probability distribution, the correct symbol sequence, a threshold, and the accuracy rate, outputs a new correct symbol sequence, and performs machine learning of the first to third neural networks using an RNNT (Recurrent Neural Network Transducer) with the new correct symbol sequence as the original correct symbol sequence; A speech recognition device having:

2. 2. The speech recognition device according to claim 1, wherein, when a random number generated from a continuous uniform distribution falls below the threshold, the symbol sampling unit inputs the new correct symbol sequence to the symbol distributed representation sequence conversion unit and the label estimation unit for each character, and recursively performs sampling determination the number of times of correct tokens to obtain the new correct symbol sequence.

3. 2. The speech recognition device according to claim 1, wherein, when processing is performed for each utterance, if the random number generated from the continuous uniform distribution falls below a value obtained by multiplying the accuracy rate obtained from the accuracy rate calculation unit by the threshold, the symbol sampling unit simultaneously replaces correct labels with actual outputs of the third neural network, and sets the obtained values ​​as the new correct symbol sequence.

4. A machine learning method executed by a speech recognition device for training a neural network for speech recognition, comprising: The speech recognition device a symbol distributed representation sequence conversion process that inputs a symbol feature sequence including a symbol representing redundancy, converts it into corresponding continuous value features, and then converts it into an intermediate character feature sequence using a first neural network, and outputs the intermediate character feature sequence; a label estimation process for inputting the intermediate character feature sequence and performing label estimation using a second neural network to output an output probability distribution that is a two-dimensional matrix; a correct answer rate calculation process in which the output probability distribution and the correct answer symbol sequence are input, and an argument having a maximum value in each predetermined-th vector of the matrix is ​​output as an output of a third neural network to output a correct answer rate; a symbol sampling process in which the output probability distribution, the correct symbol sequence, a threshold, and the accuracy rate are input, a new correct symbol sequence is output, and machine learning of the first to third neural networks is performed by an RNNT (Recurrent Neural Network Transducer) using the new correct symbol sequence as the original correct symbol sequence; Machine learning methods that perform

5. A program causing a computer to execute the method according to claim 4.

6. A speech recognition device that trains a neural network for speech recognition, an audio distributed representation sequence conversion unit that receives an acoustic feature sequence, converts the acoustic feature sequence into an intermediate acoustic feature sequence using a fourth neural network, and outputs the intermediate acoustic feature sequence; a symbol distributed representation sequence conversion unit that receives a symbol feature sequence including a symbol representing redundancy, converts the received symbol feature sequence into a corresponding continuous value feature, and then converts the intermediate character feature sequence into an intermediate character feature sequence using a first neural network, and outputs the intermediate character feature sequence; a label estimation unit that receives the intermediate acoustic feature sequence and the intermediate character feature sequence, performs label estimation using a second neural network, and outputs an output probability distribution that is a three-dimensional tensor; an alignment extraction unit that receives the output probability distribution and a correct symbol sequence, extracts only the correct symbol sequence using a forward-backward algorithm, and outputs a predetermined output probability distribution that is a two-dimensional matrix; an accuracy rate calculation unit that receives the predetermined output probability distribution and the correct symbol sequence, and outputs an accuracy rate as an output of a third neural network, the argument that takes the maximum value in each predetermined-th vector of the matrix; a symbol sampling unit that receives the predetermined output probability distribution, the correct symbol sequence, a threshold value, and the accuracy rate, outputs a new correct symbol sequence, and performs machine learning of the first to fourth neural networks using an RNNT (Recurrent Neural Network Transducer) with the new correct symbol sequence as the original correct symbol sequence; A speech recognition device having:

7. A machine learning method executed by a speech recognition device for training a neural network for speech recognition, comprising: The speech recognition device an audio distributed representation sequence conversion process for inputting an acoustic feature sequence, converting the acoustic feature sequence into an intermediate acoustic feature sequence using a fourth neural network, and outputting the intermediate acoustic feature sequence; a symbol distributed representation sequence conversion process that inputs a symbol feature sequence including a symbol representing redundancy, converts it into corresponding continuous value features, and then converts it into an intermediate character feature sequence using a first neural network, and outputs the intermediate character feature sequence; a label estimation process for inputting the intermediate acoustic feature sequence and the intermediate character feature sequence, and performing label estimation using a second neural network to output an output probability distribution that is a three-dimensional tensor; an alignment extraction process that inputs the output probability distribution and a correct symbol sequence, and extracts only the correct symbol sequence using a forward-backward algorithm, thereby outputting a predetermined output probability distribution that is a two-dimensional matrix; an accuracy rate calculation process in which the predetermined output probability distribution and the correct symbol sequence are input, and an argument having a maximum value in each predetermined-th vector of the matrix is ​​output as an output of a third neural network to output an accuracy rate; a symbol sampling process in which the predetermined output probability distribution, the correct symbol sequence, a threshold value, and the accuracy rate are input, a new correct symbol sequence is output, and machine learning of the first to fourth neural networks is performed by an RNNT (Recurrent Neural Network Transducer) using the new correct symbol sequence as the original correct symbol sequence; Machine learning methods that perform

8. A program causing a computer to execute the method according to claim 7.

Images