Speech recognition model training device, speech recognition model training method, and program

Abstract

A speech recognition model training device 1 comprises: a first speech conversion unit 11 that converts an auxiliary feature quantity XA into an auxiliary intermediate feature quantity HA using a first multilayer neural network; a second speech conversion unit 12 that receives, as inputs, and converts the auxiliary intermediate feature quantity HA and a mixed sound feature quantity XM into a target speaker intermediate feature quantity HS using a second multilayer neural network; a symbol conversion unit 13 that converts a symbol feature quantity c into an intermediate character feature quantity C using a third multilayer neural network; an estimation unit 14 that receives, as inputs, the target speaker intermediate feature quantity HS and the intermediate character feature quantity C and calculates an output probability distribution Y using a neural network; a loss calculation unit 15 that receive, as inputs, a correct answer symbol CT and the output probability distribution Y, and calculates a loss LRNN-T; and an updating unit 16 that updates the model parameters of the first speech conversion unit 11, the second speech conversion unit 12, the symbol conversion unit 13, and the estimation unit 14 using the loss LRNN-T.

Description

[Technical Field]

[0001] The present disclosure relates to a learning device, a speech recognition model learning method, and a program for a speech recognition model that directly outputs any character string (phoneme, character, subword, word) that represents the content of an utterance by a target speaker from among the speeches of multiple people. [Background technology]

[0002] Recent neural network-based speech recognition systems have become capable of directly outputting word sequences from speech features. In training a Recurrent Neural Network Transducer (RNN-T) model, the introduction of a "blank" symbol representing redundancy allows dynamic learning of correspondences between speech and output sequences from training data, provided that phonemes, characters, subwords, and word sequences (≠ frame-by-frame) corresponding to the speech content are available. In other words, training is now possible using features and labels with an incompatible relationship between input length T and output length U (generally T >> U) (see, for example, Non-Patent Document 1). Because word sequence inference processing can be performed frame-by-frame, this technology has attracted attention as a technology capable of speech recognition while speaking (enabling real-time speech recognition).

[0003] Furthermore, when mixed speech containing speech from multiple speakers is input, there is a technology that extracts the target speaker's speech from the mixed speech using the speech of the target speaker that has been registered in advance as a clue (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Alex Graves, “Sequence Transduction with Recurrent Neural Networks,” in Proc. of International Conference on Machine Learning (ICML), 2012. [Non-patent document 2] K. Zmolikova et. al., “SpeakerBeam: Speaker Aware Neural Network for Target Speaker Extraction in Speech Mixtures,” IEEE Journal of Selected Topics in Signal Processing, vol. 13, no. 4, pp. 800-814, 2019. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology for extracting the target speaker's voice from the above-mentioned mixed voice requires a lot of computational effort. Therefore, if the target speaker extraction technology is directly applied to the above-mentioned RNN-T speech recognition technology, a response delay occurs in the speech recognition processing step, and the advantage of real-time processing, which is a characteristic of RNN-T, is lost.

[0006] Therefore, the present disclosure has been made to solve the above-mentioned problems, and aims to provide a technology that enables real-time recognition of the target speaker's voice from a mixed voice containing utterances from multiple speakers while maintaining the amount of delay at the same level as conventional voice recognition systems, by providing a function within a voice recognition model that converts a distributed representation series of voice corresponding to target speaker extraction. [Means for solving the problem]

[0007] In order to solve the above problem, a speech recognition model training device according to one aspect of the present disclosure includes a first speech conversion unit that converts auxiliary features, which are a feature sequence of speech of a target speaker, into auxiliary intermediate features using a first multilayer neural network; a second speech conversion unit that converts the auxiliary intermediate features and mixed sound features, which are a feature sequence of speech of multiple speakers, into target speaker intermediate features, which are a sequence of intermediate features of the target speaker, using a second multilayer neural network; and a third speech conversion unit that converts symbol features, which are a symbol sequence of the target speaker, into corresponding continuous value features. an estimation unit that uses a neural network to receive the target speaker intermediate features and the intermediate character features and calculates an output probability distribution of a two-dimensional matrix corresponding to label estimation; a loss calculation unit that receives as input a correct symbol that is a symbol sequence of the target speaker that corresponds to correct answer data and the output probability distribution Y and calculates a loss corresponding to an error in the output probability distribution; and an update unit that uses the loss to update model parameters of the first speech conversion unit, the second speech conversion unit, the symbol conversion unit, and the estimation unit. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to recognize the voice of a target speaker in real time from a mixed voice containing speech from multiple speakers. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining the prior art 1. [Figure 2] FIG. 2 is a diagram for explaining the prior art 2. [Figure 3] FIG. 3 is a diagram showing an example of the functional configuration of the speech recognition model training device according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a processing flow of the speech recognition model training method according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of the functional configuration of a speech recognition model training device according to a modified example of the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a processing flow of a speech recognition model training method according to a modified example of the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the functional configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Character notation> The symbol "^" (superscript hat) used in text should normally be written directly above the character immediately following it, but due to limitations in text notation, it is written immediately before the character in question. In mathematical formulas, these symbols are written in their proper position, that is, directly above the character. For example, "^S" is expressed in a mathematical formula as follows:

number

number

[0011] An embodiment of the present disclosure provides a technology that enables real-time recognition of a target speaker's speech from a mixed speech containing speech from multiple speakers by providing a speech recognition model with a function for converting a distributed representation sequence of speech corresponding to target speaker extraction. Before describing the details of the embodiment of the present disclosure, we will first explain a neural network training method for speech recognition and a target speaker speech extraction method in the prior art.

[0012] (Prior Art Neural Network Training Methods for Speech Recognition) As a method for training an acoustic model using a general neural network training method, the "Recurrent Neural Network Transducer" described in Non-Patent Document 1 is known (hereinafter, this method will also be referred to as "Prior Art 1"). Figure 1 shows a functional configuration diagram of a speech recognition model training device that uses this method.

[0013] Acoustic features X, which are a sequence of features of speech, are converted into a distributed representation sequence by a speech conversion unit 101 having a multi-layer neural network function, and become intermediate features H, which are a sequence of acoustic features used for estimation in speech recognition. Furthermore, symbol features c, which are a sequence of symbols corresponding to acoustic features X and have a length U, are converted into a distributed representation sequence by a symbol conversion unit 102 having a multi-layer neural network function, and become intermediate character features C, which are a sequence of corresponding continuous-value features.

[0014] The intermediate feature amount H and the intermediate character feature amount C are input to a label estimation unit 103 having a neural network function, and an output probability distribution Y corresponding to label estimation, which is speech recognition, is calculated.

[0015] The calculated output probability distribution Y is a sequence of correct symbols C of length U or T. T At the same time, the loss L is input to the loss calculation unit 104 and calculated using a predetermined calculation formula. RNN-T The calculated loss L RNN-T is used to update the model parameters of the speech conversion unit 101, the symbol conversion unit 102, and the estimation unit 103. By repeating the above-described updating of the model parameters, learning is performed to enable more accurate speech recognition.

[0016] (Prior art target speaker voice extraction method) As a method for extracting the voice of a target speaker from a mixed sound that is the voices of multiple speakers, "SpeakerBeam" described in Non-Patent Document 2 is known (hereinafter, this method will also be referred to as "Prior Art 2"). Figure 2 shows a functional configuration diagram of a target speaker voice extraction device that uses this method.

[0017] Auxiliary speech A, which is a speech waveform of a pre-recorded target speaker's speech and is used as a clue for extracting the target speaker, is input to an auxiliary feature extraction unit 201 having a multi-layer neural network function and converted into auxiliary intermediate features A', which are acoustic features used to extract the target speaker.

[0018] The mixed speech M, which is a speech waveform composed of speeches from multiple people, and the auxiliary intermediate features A' are input to a target speaker extraction unit 202 having a multi-layer neural network function. The target speaker extraction unit 202 extracts a target speaker speech ^S, which is the speech of the target speaker, from the mixed speech M using the auxiliary intermediate features A' as a clue.

[0019] The extracted target speaker's speech ^S is input to the loss calculation unit 203 together with the target speaker's speech S, which is the correct speech waveform of the target speaker, and the loss L is calculated using these from a predetermined calculation formula. TSE The calculated loss L TSE is used to update the model parameters of the auxiliary feature extraction unit 201 and the target speaker extraction unit 202. By repeating the above-mentioned updating of the model parameters, learning is performed to more accurately extract the target speaker's voice from the mixed voice.

[0020] First Embodiment Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0021] As shown in Fig. 3, the speech recognition model training device 1 includes a first speech conversion unit 11, a second speech conversion unit 12, a symbol conversion unit 13, an estimation unit 14, a loss calculation unit 15, and an update unit 16. The speech recognition model training device 1 as a whole configures a multi-stage and multi-layer neural network. The speech recognition model training device 1 performs the speech recognition model training method of this embodiment by implementing the processing flow shown in Fig. 4.

[0022] (First voice conversion unit 11) The first speech conversion unit 11 is a target speaker information extraction type speech distributed representation sequence conversion unit. That is, the first speech conversion unit 11 uses a multi-layer neural network (first multi-layer neural network) to convert auxiliary features X A The auxiliary intermediate features H A (Step S11). Here, the auxiliary feature X A is a sequence of acoustic features extracted from a pre-recorded utterance of a target speaker, and is a sequence of acoustic features of speech used as a clue for extracting the target speaker (this speech is also referred to as "target speaker information"). That is, unlike the auxiliary feature extraction unit 201 that inputs a speech waveform in Prior Art 2, the first speech conversion unit 11 plays the role of an encoder that inputs the sequence of acoustic features of the target speaker extracted for speech recognition into a multi-layer neural network and converts it into intermediate acoustic features of the target speaker information.

[0023] The first speech conversion unit 11 performs the conversion using a mathematical formula equivalent to the following formula:

number

number

[0024] (Second speech conversion unit 12) The second speech conversion unit 12 is a target speaker speech extraction type speech distributed representation series conversion unit. That is, the second speech conversion unit 12 uses a multi-layer neural network (second multi-layer neural network) to convert auxiliary intermediate features H A and the mixed sound feature X, which is a feature sequence of mixed sound that is a mixture of the voices of multiple speakers. M and are used as inputs to generate the target speaker intermediate features H S (Step S12).

[0025] Unlike the target speaker extraction unit 202 that receives the speech waveform, the second speech conversion unit 12 receives the mixed sound feature X , which is a sequence of acoustic features of mixed speech including multiple speakers extracted for speech recognition. M is converted into the target speaker intermediate feature H S Convert to.

[0026] In this embodiment, the target speaker intermediate feature H S It is assumed that only speech information of the target speaker is included in the input signal. Therefore, as a subsequent process, a speech recognition training function for estimating the symbol sequence of the target speaker can be provided, similar to the processes of the symbol conversion unit 102, the estimation unit 103, and the loss calculation unit 104 described in Prior Art 1.

[0027] The second speech conversion unit 12 performs the conversion using a mathematical formula equivalent to the following formula:

number

[0028] (Symbol conversion unit 13) The symbol converter 13 uses a multi-layer neural network (third multi-layer neural network) to convert the symbol feature c of length U, which is a symbol sequence of the target speaker, into intermediate character feature C, which is a sequence of corresponding continuous value features (step S13). That is, the symbol converter 13 plays the role of an encoder, converting the input into a one-hot vector once, and then converting it into the intermediate character feature C using the multi-layer neural network. The symbol converter 13 has a function equivalent to that of the symbol converter 102 in Prior Art 1.

[0029] (Estimation part 14) The estimation unit 14 uses a neural network to estimate the target speaker intermediate feature H S and the intermediate character feature amount C are input, and an output probability distribution Y of a two-dimensional matrix corresponding to label estimation is calculated (step S14). The estimation unit 14 corresponds to a function equivalent to the estimation unit 103 of the prior art 1.

[0030] The output probability distribution Y is calculated using a formula equivalent to the following formula:

number

[0031] Furthermore, in the above equation, since the lengths of t and u are different, there is also the dimension of the number of elements in the neural network in addition to t and u, making it three-dimensional. Specifically, when adding, W1H copies the same value in the U dimension and expands it into a three-dimensional tensor. W2C copies the same value in the T dimension and expands it into a three-dimensional tensor. Since three-dimensional tensors are added together, the output is also a three-dimensional tensor.

[0032] Generally, RNN-T is trained using RNN-T loss, assuming that the output will be a three-dimensional tensor. However, during inference, which is the processing of the estimation unit 14, the output will be a two-dimensional matrix because there is no expansion operation.

[0033] (Loss calculation section 15) The loss calculation unit 15 calculates the correct symbol C, which is a symbol sequence of the target speaker (of length U or T) that corresponds to the correct data. T and the output probability distribution Y, which is a three-dimensional tensor, are input, and the loss L, which corresponds to the error of the output probability distribution Y, is calculated. RNN-T (Step S15). The loss calculation unit 15 has a function equivalent to the processing function of loss calculation carried out by the loss calculation unit 104 in the prior art 1.

[0034] loss L RNN-T The calculation of is performed by creating a tensor with the symbol sequence length U on the vertical axis, the input sequence length T on the horizontal axis, and the depth as the number of classes, i.e., the number of symbol entries K, and then calculating the path of the optimal transition probability on the U×T plane based on a forward-backward algorithm. Details of the calculation are described in, for example, Chapter 2 "2. Recurrent Neural Network Transducer" of the above-mentioned Non-Patent Document 1.

[0035] (Updated part 16) The update unit 16 updates the loss L RNN-T The update unit 16 updates the model parameters of the first speech conversion unit 11, the second speech conversion unit 12, the symbol conversion unit 13, and the estimation unit 14 using the above (step S16). The update unit 16 corresponds to a function similar to the model parameter update function carried out by the loss calculation unit 104 of prior art 1.

[0036] The voice recognition model learning device 1 repeats the above-described updating of the model parameters to learn so as to be able to correctly recognize voices.

[0037] The effects of the speech recognition model training device 1 of this embodiment are expected to be the same as those described in Non-Patent Documents 1 and 2 above. That is, the amount of computational processing is considered to be equivalent to that of conventional speech recognition devices, such as those described in Non-Patent Document 1. Furthermore, the recognition performance of speech recognition is considered to be equivalent to the result of combining Conventional Technique 1 and Conventional Technique 2, for example. Therefore, compared to simply extracting target speech using Conventional Technique 2 and then performing speech recognition processing using Conventional Technique 1, it is possible to achieve speech recognition of the target speaker while dramatically reducing the amount of computation.

[0038] Therefore, according to the present disclosure, it is possible to recognize the voice of a target speaker in real time from mixed voices containing speeches from multiple speakers.

[0039] <Modification of the first embodiment> In the first embodiment, the mixed sound feature X M However, it is possible that the acoustic features of the target speaker are not included in the actual sound mixture. Therefore, if we can create a situation equivalent to when the acoustic features of the target speaker are not included in the sound mixture, and train the model to output a symbol indicating that the target speaker is not included in that situation, we can create a learning model that operates more robustly.

[0040] To incorporate the above functions, the above-described speech recognition model training device 1 may be configured as a speech recognition model training device 1' in Fig. 5. The speech recognition model training device 1' differs from the speech recognition model training device 1 in Fig. 3 in that an inversion unit 17 is newly provided. Accordingly, the flow diagram in Fig. 4 is changed as shown in Fig. 6. That is, step S17 is added before step S11, step S11 changes to step S11', step S12 changes to step S12', step S14 changes to step S14', and step S15 changes to step S15'.

[0041] As shown in FIGS. 5 and 6, the inverting unit 17 generates the auxiliary feature X A and the inversion coefficient λ are used as inputs to obtain the second auxiliary feature X A2 (=λX A The reversing unit 17 generates the correct symbol C T and the inversion coefficient λ as inputs to obtain the second correct symbol C T2 (=λC T The inverting unit 17 generates the second auxiliary feature X A2 to the first speech conversion unit 11, and outputs the second correct symbol C T2 to the loss calculation unit 15. The inversion coefficient λ is a preset coefficient that satisfies the condition 0≦λ≦1. When the inversion coefficient λ=0, the inversion unit 17 outputs the input auxiliary feature X A , correct symbol C T When the inversion coefficient λ≠0, the inversion unit 17 outputs the auxiliary feature X A The inversion unit 17 converts and outputs the correct symbol C T is converted and output (step S17).

[0042] In step S11, the first speech conversion unit 11 converts the sequence used for conversion into auxiliary features X A to the second auxiliary feature X A2 (λX A) and performs the conversion process of the first speech conversion unit 11 (step S11'). In addition, the loss calculation unit 15 changes the sequence used for calculation in step S15 to the correct symbol C T From the second correct symbol C T2 (λC T ), and the calculation process of the loss calculation unit 15 is performed (step S15').

[0043] When the inversion coefficient λ is not 0 (λ≠0), the second speech conversion unit 12 converts the mixed sound feature X M The second auxiliary feature X A2 In some cases, it may not be possible to find a target speaker. In such cases, a notification to that effect is output to the estimation unit 14 (step S12'). In this case, the estimation unit 14 outputs a unified symbol (for example, ~C) indicating a non-target speaker as the result of the output probability distribution Y (step S14').

[0044] Note that when the inversion coefficient λ is set to a value close to 0 (zero), such as 0.01, the inversion unit 17 may be configured to output the auxiliary feature X without conversion. A The same content as the second auxiliary feature X A2 to the first speech conversion unit 11, and the correct symbol C T The same content as the second correct answer symbol C T2 In this case, similar to the device for recognizing only the target speaker's voice, the loss calculation unit 15 receives the original correct symbol C T The second correct symbol C is the same as T2 As a result, the parameters are updated by the processing of the update unit 16.

[0045] This modification makes it possible to realize a framework that does not explicitly recognize speech other than that of the target speaker. Even in this modification, it is possible to recognize the speech of the target speaker in real time from a mixed speech containing speech from multiple speakers. Furthermore, in this modification, it is possible to train a part of the training data in the case where the inversion coefficient λ≠0. By performing such training, it becomes possible to train a more robust model than in the first embodiment.

[0046] The various processes in the first embodiment and the modified example of the first embodiment described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capacity of the device executing the processes or as needed. For example, step S12 and step S13 may be executed in parallel, or the process of step S13 may be executed before step S12. Needless to say, other appropriate modifications are possible within the scope of the present invention.

[0047] [Programs, recording media] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 7, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

[0048] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0049] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0050] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the received program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0051] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

Claims

1. a first speech conversion unit that converts auxiliary features, which are a feature sequence of the speech of a target speaker, into auxiliary intermediate features using a first multilayer neural network; a second speech conversion unit that converts the auxiliary intermediate features and mixed sound features, which are a feature sequence of speeches of multiple speakers, into target speaker intermediate features, which are a sequence of intermediate features of the target speaker, using a second multilayer neural network; a symbol conversion unit that converts the symbol features, which are symbol sequences of the target speaker, into corresponding intermediate character features, which are continuous value features, using a third multilayer neural network; an estimation unit that uses a neural network to calculate an output probability distribution of a two-dimensional matrix corresponding to label estimation, using the target speaker intermediate features and the intermediate character features as inputs; a loss calculation unit that receives as input a correct symbol, which is a symbol sequence of the target speaker corresponding to correct data, and the output probability distribution Y, and calculates a loss corresponding to an error in the output probability distribution; an update unit that updates model parameters of the first speech conversion unit, the second speech conversion unit, the symbol conversion unit, and the estimation unit using the loss; A speech recognition model learning device having the above configuration.

2. H ｔａｒｇｅｔ’ is a sequence of auxiliary intermediate features of length T that are the source of the auxiliary intermediate features, and f Ｓｐｋ－Ｅｎｃ’ (·) is the first multilayer neural network, and f ＦＥ (·) is the feature extraction function, and A ｃｌｕｅ is the speech waveform of the auxiliary speech that is the basis of the auxiliary feature, and θ Ｓｐｋ－Ｅｎｃ’ is an updatable parameter in the first speech conversion unit, and h ｔａｒｇｅｔ’ is the auxiliary intermediate feature, and h ｔ ｔａｒｇｅｔ’ 2. The speech recognition model training device according to claim 1, wherein, when x is the auxiliary intermediate feature at time t, the first speech conversion unit performs conversion using the following equation: [Equation 7] [Equation 8]

3. h ｔ ＡＳＲ’ is the target speaker intermediate feature, and f ＡＳＲ－Ｅｎｃ’ is the second multilayer neural network, and f ＦＥ (·) is the feature extraction function, and x ｔ’ is the speech waveform of the mixed speech that is the source of the mixed speech feature at time t', and h ｔａｒｇｅｔ’ is the auxiliary intermediate feature, and θ ＡＳＲ－Ｅｎｃ’ 3. The speech recognition model training device according to claim 2, wherein, when is an updatable parameter in the second speech conversion unit, the second speech conversion unit performs conversion using the following equation: [Equation 9]

4. 2. The speech recognition model training device according to claim 1, wherein the symbol converter converts the input data into a one-hot vector, and then converts the converted data into the intermediate character feature by the third multilayer neural network.

5. h t is the auxiliary feature at time t, and c ｕ is the u-th symbol feature, and W １ is the input h ｔ The hidden layer weights for ２ is the input c ｕ Let be the hidden layer weight for , b be the bias, and W ３ is the input tanh(W １ h ｔ +W ２ c ｕ +b), Softmax is the activation function, and y ｔ，ｕ 2. The speech recognition model training device according to claim 1, wherein, where is an output probability distribution, the estimation unit performs label estimation using the following equation: [Equation 10]

6. the speech recognition model training device further comprises an inversion unit, The inverting unit generates a second auxiliary feature using the auxiliary feature and an inversion coefficient, and generates a second correct symbol using the correct symbol and the inversion coefficient. the first speech conversion unit changes a sequence used for conversion from the auxiliary feature to a second auxiliary feature; the loss calculation unit changes the sequence used for calculation from the correct symbol to a second correct symbol; when the second speech conversion unit cannot find the second auxiliary feature among the mixed sound features, it outputs a message to that effect; When the estimation unit receives the input to that effect, it outputs a symbol indicating a non-target speaker as a result of the output probability distribution Y. The speech recognition model training device according to claim 1 .

7. A computer comprising: Auxiliary features, which are a sequence of acoustic features of the speech of a target speaker, are input and converted into auxiliary intermediate features using a first multilayer neural network; the auxiliary intermediate features and mixed sound features, which are a feature sequence of speeches of multiple speakers, are input, and converted into target speaker intermediate features, which are a sequence of intermediate features of the target speaker, using a second multilayer neural network; a third multilayer neural network is used to convert the symbol features, which are symbol sequences of the target speaker, into corresponding intermediate character features, which are continuous value features; Using the target speaker intermediate features and the intermediate character features as input, a neural network is used to calculate an output probability distribution of a two-dimensional matrix corresponding to label estimation; a correct symbol sequence of the target speaker corresponding to correct data and the output probability distribution are input, and a loss corresponding to an error in the output probability distribution is calculated; using the loss to update the first multilayer neural network, the second multilayer neural network, the third multilayer neural network, and model parameters used by the neural networks; Speech recognition model training method.

8. 7. A program for causing a computer to function as the speech recognition model training device according to claim 1.

Images