Speech recognition training device, speech recognition device, speech recognition training method, speech recognition method, and program

Abstract

This speech recognition training device comprises a model parameter optimization unit configured to train a machine learning model so that an output from the machine learning model to which a first speech comprising superimposition of at least two utterances has been input approximates data which is given as a correct answer to the first speech and which includes a text indicating contents of each utterance, a time section in the first speech for each utterance, and identification information of a speaker of each utterance. In this way, the accuracy of determining whether the speakers are the same or different with respect to speech in which a plurality of utterances are superimposed is improved.

Description

Speech recognition training device, speech recognition device, speech recognition training method, speech recognition method and program

[0001] The present invention relates to a speech recognition training device, a speech recognition device, a speech recognition training method, a speech recognition method, and a program.

[0002] Multi-speaker speech recognition is a technology that transcribes the content of each utterance from speech, including overlapping speech, produced by multiple speakers. Non-Patent Document 1 introduces a special token [sep] that indicates speaker changes, and outputs the utterances in order of earliest start time. For example, if one speaker says "Hello, it's nice weather, isn't it?" and another speaker interrupts while saying "Hello," adding "It's been a while," the output of this system will be "Hello [sep] It's been a while [sep] It's nice weather, isn't it?"

[0003] A technology for simultaneously estimating speech duration and speech content has been proposed for multi-speaker speech recognition. In Non-Patent Document 2, the start and end times of an utterance are assigned to each utterance. For example, in the sentence "[0] [5] Hello [sep] [3] [6] It's been a while [sep] [7] [9] It's nice weather, isn't it?", the start and end times of each utterance are represented by numbers in [ ] for a predetermined speech time unit (e.g., 0.5 seconds). The first number in [ ] for each utterance indicates the start time, and the second number in [ ] indicates the end time. Since "Hello" is assigned [0] [5], it can be seen that it was spoken from 0 (= 0 × 0.5) seconds to 2.5 (= 5 × 0.5) seconds.

[0004] N. Kanda, Y. Gaur, X. Wang, Z. Meng, and T. Yoshioka, "Serialized output training for end-to-end overlapped speech recognition," in Proc. Interspeech, 2020, pp. 2797-2801N. Makishima, K. Suzuki, S, Suzuki, A. Ando, ​​R. Masumura, "Joint Autoregressive Modeling of End-to-End Multi-Talker Overlapped Speech Recognition and Utterance-level Timestamp Prediction", in Proc. Interspeech, 2023

[0005] The technology in Non-Patent Document 2 can estimate the content and duration of speech at the utterance level (for each utterance) for speech that includes overlaps, but cannot identify whether the speaker of each utterance is the same or different. For example, from the output result "[0] [5] Hello [sep] [3] [6] It's been a while [sep] [7] [9] It's nice weather, isn't it?", it is unclear whether "Hello" and "It's nice weather, isn't it?" are spoken by the same person or different people. Therefore, it is necessary to use a different model to detect non-overlapping speech segments from the speech time and estimate speaker vectors from those segments for speech estimation. However, even with this different model, there is a problem in that the estimation accuracy of speaker differences decreases when non-overlapping speech is short or does not exist.

[0006] The present invention has been made in view of the above points, and has as its object to improve the accuracy of distinguishing between different speakers in a voice in which a plurality of utterances are superimposed.

[0007] In order to solve the above problem, the speech recognition training device has a model parameter optimization unit configured to train the machine learning model so that the output from the machine learning model to which a first speech in which two or more utterances are superimposed approaches data including text indicating the content of each utterance, a time interval in the first speech for each utterance, and identification information of the speaker of each utterance, which is given as a correct answer for the first speech, and a time interval in the first speech for each utterance.

[0008] It is possible to improve the accuracy of distinguishing between different speakers when a plurality of utterances are superimposed on one another.

[0009] FIG. 1 is a diagram illustrating an example of a hardware configuration of a speech recognition device 10 according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a functional configuration of the speech recognition device 10 according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a configuration of a model parameter learning unit 13. FIG. 4 is a diagram illustrating an example of a configuration of an utterance information estimation unit 14. FIG. 5 is a diagram illustrating an example of a configuration of a decoder unit 142.

[0010] An embodiment of the present invention will now be described with reference to the accompanying drawings. Fig. 1 is a diagram showing an example of the hardware configuration of a speech recognition device 10 according to an embodiment of the present invention. The speech recognition device 10 shown in Fig. 1 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, and an interface device 105, all of which are interconnected via a bus B.

[0011] A program that realizes processing in the speech recognition device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0012] When an instruction to start the program is received, the memory device 103 reads and stores the program from the auxiliary storage device 102. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the speech recognition device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0013] 2 is a diagram showing an example of the functional configuration of a speech recognition device 10 according to an embodiment of the present invention. In FIG. 2, the speech recognition device 10 includes a speaker ID table acquisition unit 11, a training speaker ID label estimation unit 12, a model parameter learning unit 13, an utterance information estimation unit 14, and a speaker vector comparison unit 15. Each of these units is realized by a process executed by a processor 104 of one or more programs installed in the speech recognition device 10.

[0014] 2, the speaker ID table acquisition unit 11, the training speaker ID label estimation unit 12, and the model parameter training unit 13 are used only during training, while the speaker vector comparison unit 15 is used only during inference.

[0015] [Speaker ID Table Acquisition Unit 11] The speaker ID table acquisition unit 11 acquires a single speaker voice V １ , ..., V ｜Ｄ｜ is input, a speaker ID table T is obtained from the input single speaker speech, and the speaker ID table T is output. ｉ (i=1,...,|D|) is the speech of one utterance by one speaker. ｉ The speakers of |D| V may be different. ｉ Among these, there are multiple utterances by the same person. For example, if 100 utterances are collected from 100 people, then |D| = 10,000. However, the number of utterances from each person does not have to be the same. Furthermore, the content of each utterance may be unrelated to each other.

[0016] The speaker ID table is a table that associates a speaker ID number with a speaker vector that represents the speaker characteristics. １, ..., V ｜Ｄ｜ The speaker ID table acquisition unit 11 assigns an identification number to each speaker identified by the speaker ID table acquisition unit 11 based on the above. Any method can be used to acquire the speaker ID table. For example, each V is acquired by a model (hereinafter referred to as "model Z") obtained by removing the identification layer from a pre-trained speaker identification model. ｉ A speaker vector representing the speaker's identity is estimated from V ｉ For each speaker vector, the cluster number of the cluster after applying clustering such as the k-means method is set as the speaker ID number, and the center (centre of gravity) of the cluster is registered in the speaker ID table as the speaker vector corresponding to the speaker ID number. ｉ are classified by clustering, the number of speaker ID numbers in the speaker ID table does not necessarily match the number of actual speakers. Also, utterances by the same speaker are not necessarily classified into the same cluster.

[0017] [Training Speaker ID Label Estimation Unit 12] The training speaker ID label estimation unit 12 receives the single-speaker voices X1, ..., X|D| and the speaker ID table T acquired by the speaker ID table acquisition unit 11, and estimates the speaker ID label S １ , ..., S ｜Ｄ｜ That is, the training speaker id label estimation unit 12 outputs each single speaker speech X ｉ For (i=1, . . . , |D|), refer to the speaker ID table T to find the speaker ID label S ｉ (i=1,...,|D|) is estimated. ｉ (i=1,...,|D|) is the speech of one utterance by one speaker. ｉ The speakers of |D| V may be different. ｉ Among the utterances, there are multiple utterances by the same person. For example, if 100 utterances are collected from 100 people, |D| = 10,000. However, the number of utterances from each person does not have to be the same. Furthermore, the contents of each utterance may be unrelated to each other. ｉ is the utterance used to obtain the speaker id table, while X ｉThe difference between the two is that the utterances are used for speech recognition training and the content of the utterances is known. ｉ The speaker and X in ｉ The speaker may be different from the speaker in

[0018] each X ｉ For ｉ Any method can be used to estimate X. For example, ｉ If a speaker vector that completely matches speaker vector B is registered in the speaker ID table, the speaker ID number associated with the speaker vector in the speaker ID table is X. ｉ Speaker ID label S ｉ If there is no speaker vector in the speaker id table that perfectly matches the speaker vector B, a set of speaker vectors whose average for each dimension is closest to the speaker vector B is found from the speaker id table, and a set of speaker id numbers corresponding to the set is denoted as X ｉ Speaker id label S for ｉ That is, the learning speaker id label estimation unit 12 estimates X ｉ The speaker id label S is estimated for ｉ is a set of one or more speaker id numbers.

[0019] The set of speaker vectors closest to speaker vector B may be obtained by repeatedly finding one speaker vector closest to speaker vector B, averaging it with the found speaker vector, and finding a speaker vector that is even closer to speaker vector B. By representing a speaker with one speaker ID label using a set of multiple speaker ID numbers, it is possible to obtain equal or greater expressive power with a set of ID labels with a smaller number of classes than when an ID label with a larger number of classes is used as the speaker ID label. For example, using two labels with 1,000 classes can obtain equal or greater expressive power with a smaller number of classes than using one label with 10,000 classes.

[0020] [Model Parameter Learning Unit 13] Fig. 3 is a diagram showing an example of the configuration of the model parameter learning unit 13. As shown in Fig. 3, the model parameter learning unit 13 includes a learning data generating unit 131 and a model parameter optimizing unit 132.

[0021] {Learning Data Generation Unit 131} The learning data generation unit 131 generates a single-speaker voice X1, ..., X|D|, a single-speaker utterance content Y1, ..., Y|D|, and a speaker ID label S output from the learning speaker ID label estimation unit 12. １ , ..., S ｜Ｄ｜ and generate training data for the speech recognition model in this embodiment. ｉ (i=1,...,|D|) is a single speaker voice X ｉ It is a text indicating the content of the utterance of (i=1, . . . , |D|) and is created in advance.

[0022] The training data for the speech recognition model is a multi-speaker superimposed speech X^ １ , ..., X^ ｜Ｄ｜ and multi-speaker utterance label Y^ １ , ..., Y^ ｜Ｄ｜ It is a pair with X^ ｉ (i=1, ..., |D|) is a voice in which utterances by two or more (multiple) speakers are superimposed. ｉ (i=1,...,|D|) is X^ ｉ is the correct answer as the output when the above is input to the speech recognition model. Note that X^ and Y^ correspond to the symbols X and Y with ^ above them in the figure.

[0023] The learning data generation unit 131 generates each X ｉ to another X ｊ (i ≠ j) are superimposed, so that each X^ ｉ Generate X ｊ is X ｉ It may or may not be the next utterance of X. ｉ From the beginning of ｊ may be superimposed, or X ｉ X from the middle of ｊ At least X ｉ Part of Xｊ That is, X^ ｉ is a sound that includes a mixed (superimposed) part. ｉ One X in ｊ An example will be described in which each X ｉ 2 or more X ｊ , X ｋ may be superimposed.

[0024] The learning data generation unit 131 also ｉ X in ｉ and X ｊ the start and end times of each utterance (i.e., the time interval of the utterance), and the speaker id label S ｉ and based on Sj, X^ ｉ Multi-speaker utterance label Y^ for ｉ For example, X ｉ The content of the utterance is Y ｉ and X^ after superposition ｉ X in ｉ The speech duration of is 0 to 5 seconds, and the speaker id label S ｉ Let X be <1> <2> <3>. ｊ The content of the utterance is Y ｊ and X^ after superposition ｉ The speech duration in is 3 to 7 seconds, and the speaker ID labels Sj are <4> <5> <6>. Furthermore, the speech duration is discretized every 0.1 seconds. In this case, the speech duration after superposition is ｉ In this case, Y ｉ The speech start time of Y is [0], the speech end time is

[50] , ｊ The start time of the utterance can be expressed by the token

[30] and the end time of the utterance by the token

[70] . ｉ The structure of "X ｉ Start time of X ｉ End time of X ｉ The content of the utterance (= Y ｉ ), X ｉ Speaker ID label (=S ｉ ), [sep], X ｊ The content of the utterance (= Y ｊ ), X ｊIn this case, the multi-speaker utterance label Y^ ｉ is "[0]

[50] Y ｉ <1><2><3>[sep]

[30]

[70] Y ｊ <4> <5> <6>". In this way, an utterance label is information (correct label) that includes, for each utterance contained in the audio, the start time of the utterance, the end time of the utterance, the content of the utterance, and the speaker ID label. In addition to this, the order of utterance labels for one utterance can be any order such as: Speaker ID label, content of the utterance, start time of the utterance, end time of the utterance; Utterance start time, end time of the utterance, speaker ID label, content of the utterance.

[0025] Note that the multi-speaker utterance label Y^ １ , ..., Y^ ｜Ｄ｜ may be created manually.

[0026] {Model Parameter Optimization Unit 132} The model parameter optimization unit 132 optimizes the learning data (X^ １ , ..., X^ ｜Ｄ｜ , Y^ １ , ..., Y^ ｜Ｄ｜ ) to learn the model parameters θ of the utterance information estimation unit 14 as a speech recognition model. ｉ is input to the utterance information estimation unit 14, and the inference content z (utterance start time, utterance end time, utterance content, and speaker ID label) output by the utterance information estimation unit 14 is calculated. ｉ The correct answer to the multi-speaker utterance label Y^ ｉ Compare with and z is Y^ ｉ In this embodiment, the utterance information estimation unit 14 differs from the conventional model in that it outputs the inference content z including the speaker ID label, but it is common to the conventional model in that the inference content z including the speaker ID label is also a token sequence. ｉ The method of calculating the loss and the method of updating θ based on the loss (such as backpropagation) can be the same as those in the prior art (for example, Non-Patent Document 2).

[0027] [Utterance Information Estimation Unit 14] As described above, the utterance information estimation unit 14 is a machine learning model (voice recognition model) that performs voice recognition in this embodiment, and is configured by, for example, a neural network.

[0028] 4 is a diagram showing an example of the configuration of the utterance information estimation unit 14. As shown in FIG. 4, the utterance information estimation unit 14 includes an encoder unit 141 and a decoder unit 142.

[0029] {Encoder Unit 141} The encoder unit 141 receives an observed signal x and a model parameter θ, and outputs a latent representation h, which is a vector including features important for estimating an utterance label for x. ｉ and is a speech signal relating to each utterance in a plurality of utterances (e.g., a sequence of utterances) during inference. The encoder unit 141 may have the same configuration as that of conventional technology (multiple speaker speech recognition or single speaker speech recognition). Any model can be used for the encoder unit 141, and for example, a Transformer can be used.

[0030] {Decoder Unit 142} Fig. 5 is a diagram showing an example of the configuration of the decoder unit 142. As shown in Fig. 5, the decoder unit 142 includes a feature transform unit 21, a sequence output unit 22, a mask unit 23, and an averaging unit 24.

[0031] <Feature Transformation Unit 21> The feature transformation unit 21 transforms a latent representation h, which is an output from the encoder, a model parameter θ, and a token z from the sequence unit up to one step before. ｔ The encoder 141 receives the output of a series of tokens (the estimation result of the token sequence constituting the inference content) and outputs a vector q for each token at each step based on h and the output up to the previous step as θ. Since the encoder 141 is trained so that the latent representation h also includes features related to the speaker ID label, the feature converter 21 is trained so that q in the step of outputting the speaker ID label also includes information corresponding to the speaker ID label. The feature converter 21 may have the same configuration as conventional technology (multiple speaker speech recognition or single speaker speech recognition).

[0032] <Sequence Output Unit 22> The sequence output unit 22 receives the output q from the feature conversion unit 21 and the model parameter θ, and outputs the t-th token z constituting the inference content z (utterance start time, utterance end time, utterance content, speaker ID label). ｔ are output sequentially (for each step (token)). Here, one token of the speech start time and speech end time is a character string indicating one time, such as [0] or

[50] . Therefore, the speech start time and speech end time are made up of two tokens. One token of the speech content is one character. Therefore, the speech content is made up of tokens equal to the number of characters in the speech content. One token making up the speaker ID label is a character string in which each speaker ID number making up the speaker ID label is enclosed in <>, such as <1>. A speaker ID label made up of three speaker ID numbers is made up of three tokens. Token z at step t ｔ is one of these tokens.

[0033] The order of the utterance start time, utterance end time, utterance content, and speaker ID label in the inference content z is the order of these in the utterance label during learning of the utterance information estimation unit 14 .

[0034] The sequence output layer has output units equal to (number of character tokens + number of time tokens + number of speaker ID label tokens + number of special symbols). Here, the special symbols are [sep], which indicates a speaker switch, and [eos], which indicates the end of an utterance. The number of speaker ID label tokens can be set arbitrarily. For example, if the speaker embeddings representing speaker characteristics are classified into 1,000 clusters for all speakers, and speaker characteristics are represented by the cluster IDs, the number of speaker ID label tokens is 1,000.

[0035] <Masking Unit 23> The masking unit 23 receives the vector q output for each step from the feature transforming unit 21 and the inference content z up to that step from the sequence outputting unit 22, and outputs a speaker vector set a for the observed signal x. Specifically, the masking unit 23 obtains, as the speaker vector, a vector obtained by masking the q for each step that does not correspond to a speaker ID label. Such a mask can be obtained from the output z of the sequence outputting unit 22. That is, the masking unit 23 determines from z (a sequence of indexes representing tokens) whether each index corresponds to a speaker ID label, and masks it as 1 if it corresponds, or as 0 if it does not correspond.

[0036] Specifically, when q is a 100x512 vector (100 is the step length, 512 is the dimension of the vector) and z is a 100x1 token sequence, the masking unit 23 passes the portion of z, q, that corresponds to the speaker ID label, as a vector constituting a. For example, when one speaker ID label is expressed by three speaker ID numbers, a becomes 3x512. In other words, the masking unit 23 outputs multiple speaker vectors for one utterance according to the learning.

[0037] Therefore, a includes the speaker vectors of each utterance included in observed signal x, and for example, when observed signal x in which utterance 1 and utterance 2 are superimposed is input, a includes the speaker characteristics (speaker vectors) for each utterance. Therefore, in this case, when one speaker characteristic is expressed by three speaker vectors, three speaker vectors are obtained for utterance 1 and three speaker vectors are obtained for utterance 2, for a total of six speaker vectors. Note that mask unit 23 is not a training target.

[0038] <Averaging unit 24> The averaging unit 24 inputs the speaker vector set a, calculates the average (average for each dimension) of the speaker vectors included in the speaker vector set a for each utterance (for each observed signal x), and outputs the calculation result as a speaker vector o. The speaker vector o is an estimated value of the speaker vector for the observed signal x. Note that when speaker characteristics are expressed by one speaker vector, a becomes the speaker vector o.

[0039] For example, when an observed signal x in which utterance 1 and utterance 2 are superimposed is input and one speaker characteristic is expressed by three speaker vectors, a total of six speaker vectors are obtained as a, but the averaging unit 24 takes the average of three speaker vectors for each utterance and outputs a total of two speaker vectors as o. Note that when one speaker vector is used for each utterance, the input and output of the averaging unit 24 are equal.

[0040] The averaging unit 24 calculates the z ｔ is a terminal symbol [eos] or a [sep] indicating a speaker change, can be detected as a break in the utterance. Therefore, the averaging unit 24 can execute processing at this timing. Furthermore, the operation timing of the averaging unit 24 and the masking unit 23 may be the same.

[0041] Note that the inference content z output from the decoder unit 142 also includes a speaker ID label that identifies the speaker. By also being able to output the speaker vector o, the granularity (resolution) of speaker identification can be made finer, and it is also possible to identify untrained speakers. In other words, a speaker ID label is a combination of speaker ID numbers included in the speaker ID table T, and the number of speaker characteristics that can be expressed is finite. On the other hand, a speaker vector is a vector in which the values ​​of each dimension are real numbers, so it is possible to express a virtually infinite number of speaker characteristics. For this reason, this embodiment is configured to also output the speaker vector o.

[0042] [Speaker Vector Comparison Unit 15] The speaker vector comparison unit 15 (Figure 2) receives the speaker vector o output from the averaging unit 24 for each utterance as input, and identifies whether the speaker of each speaker vector o is similar or different, thereby identifying whether the speaker of the utterance corresponding to each speaker vector o is similar or different.

[0043] For example, the speaker vector comparison unit 15 applies clustering such as the k-means method to a set of speaker vectors o obtained for each utterance, and determines that utterances related to speaker vectors o that belong to the same cluster are utterances made by the same speaker. As a result, the user can grasp the differences between speakers for multiple utterances.

[0044] [During Inference] As described above, during inference, a speech signal relating to each utterance in a plurality of utterances (e.g., a sequence of utterances) is input as an observed signal x. The observed signal x may be a superposition of a plurality of utterances. The utterance information estimation unit 14 outputs an inference content z including an utterance time, utterance content, and speaker ID label for each utterance, and a speaker vector o for each utterance.

[0045] As described above, according to the present embodiment, unlike conventional techniques that combine multiple technologies such as speech separation, it is possible to perform multi-speaker speech recognition at the utterance level, speech time estimation, and speaker vector estimation for speaker estimation using a single model, thereby improving the accuracy of distinguishing between different speakers in speech in which multiple utterances are superimposed. Speaker vector estimation enables speaker identification, enabling speaker diarization and speaker verification. This makes it possible to output more detailed information about the situation at the time of utterance, which can be useful for communication purposes such as creating minutes that summarize the content of utterances while distinguishing between speakers, or visualizing the speech rate for each speaker.

[0046] The speech recognition device 10 during training is an example of a speech recognition training device. ｉ is an example of the first speech. The observed signal x at the time of inference is an example of the second speech. The speaker id label is an example of speaker identification information.

[0047] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0048] 10 Speech recognition device 11 Speaker ID table acquisition unit 12 Training speaker ID label estimation unit 13 Model parameter learning unit 14 Speech information estimation unit 15 Speaker vector comparison unit 21 Feature conversion unit 22 Sequence output unit 23 Mask unit 24 Averaging unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 Processor 105 Interface device 131 Training data generation unit 132 Model parameter optimization unit 141 Encoder unit 142 Decoder unit B Bus

Claims

1. A speech recognition training device comprising: a model parameter optimization unit configured to train a machine learning model to which a first speech in which two or more utterances are superimposed is input so that the output from the machine learning model approaches data including text indicating the content of each utterance, time intervals in the first speech for each utterance, and identification information of the speaker of each utterance, which are given as correct answers for the first speech.

2. A speech recognition device characterized by having an utterance information estimation unit configured to estimate, for a second speech in which two or more utterances are superimposed, text indicating the content of each utterance, time periods in the second speech for each utterance, and identification information of the speaker of each utterance, using a machine learning model trained so that the output when a first speech in which two or more utterances are superimposed approaches data including text indicating the content of each utterance, time periods in the first speech for each utterance, and identification information of the speaker of each utterance, which are given as a correct answer for the first speech.

3. The speech recognition device of claim 2, wherein the machine learning model sequentially outputs tokens for a second speech, the tokens constituting text indicating the content of each utterance, a time interval in the second speech for each utterance, and identification information of the speaker of each utterance, and the machine learning model includes: an encoder unit that inputs the second speech and outputs a latent representation; a feature conversion unit that inputs the latent representation and the sequentially output token sequence and outputs a vector for each token; a sequence output unit that sequentially outputs the tokens based on the vector; and a mask unit that outputs a speaker vector for identifying the speaker by masking parts of the vector that do not correspond to the speaker's identification information.

4. The speech recognition device according to claim 3, wherein the masking unit outputs a plurality of speaker vectors for one utterance, and an averaging unit outputs an average of the plurality of speaker vectors for each utterance.

5. A speech recognition training method characterized by a computer executing a model parameter optimization procedure for training the machine learning model to which a first speech in which two or more utterances are superimposed is input so that the output from the machine learning model approaches data including text indicating the content of each utterance, a time interval in the first speech for each utterance, and identification information of the speaker of each utterance, which are given as a correct answer for the first speech.

6. A speech recognition method characterized by a computer executing an utterance information estimation procedure for estimating, for a second speech in which two or more utterances are superimposed, text indicating the content of each utterance, time periods in the second speech for each utterance, and identification information of the speaker of each utterance, using a machine learning model trained to approximate data including text indicating the content of each utterance, time periods in the second speech for each utterance, and identification information of the speaker of each utterance, which are given as a correct answer for the first speech.

7. A program for causing a computer to execute a model parameter optimization procedure for training a machine learning model to which a first speech in which two or more utterances are superimposed is input so that the output from the machine learning model approaches data including text indicating the content of each utterance, a time interval in the first speech for each utterance, and identification information of the speaker of each utterance, which are given as a correct answer for the first speech.

8. A program for causing a computer to execute an utterance information estimation procedure for estimating, for a second speech in which two or more utterances are superimposed, text indicating the content of each utterance, time periods in the second speech for each utterance, and identification information of the speaker of each utterance, using a machine learning model trained so that the output, when a first speech in which two or more utterances are superimposed is similar to data including text indicating the content of each utterance, time periods in the first speech for each utterance, and identification information of the speaker of each utterance, which are given as a correct answer for the first speech.

Images