Speech activity detection device, speech activity detection method, and speech activity detection device program

Abstract

To provide an utterance section detection device capable of recognizing voice of each person from voice data in which voices of a plurality of speakers are mixed for an existing voice recognition system.SOLUTION: An utterance section detection device comprises: an utterance section determination part for receiving voice data in which voices of a plurality of speakers are mixed, determining an utterance section of each speaker from the voice data and extracting from the voice data, an intermediate speaker feature which can identify the speaker; and a speaker feature extraction part for extracting the speaker feature which can identify the speaker on the basis of the voice data and the intermediate speaker feature.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a speech activity detection device, a speech activity detection method, and a speech activity detection device program for determining an utterance activity for each speaker from an audio signal. [Background technology]

[0002] In general, in speech recognition, in order to achieve robust recognition performance against speaker differences, a method is adopted in which an acoustic model is adapted to a specific speaker using speaker features calculated for each speaker. However, when speech is composed of utterances from multiple speakers, it is problematic to determine which section of the input speech should be adapted to which speaker.

[0003] Speaker diarization technology has been developed as a technology for determining the speech intervals of each speaker. Speaker diarization is a technology for determining "when and who spoke" for a given audio recording. Speaker features calculated from the speech intervals of each speaker obtained by speaker diarization can be used to perform speaker adaptation of the acoustic model.

[0004] Target speaker speech recognition technology has been developed as a technology for recognizing only the speech of a specific speaker, and a representative patent document is Patent Document 1. In Patent Document 1, the speech section of the speaker is extracted from the input speech, and target speaker speech recognition is performed using the speaker features calculated from this speech section.

[0005] Meanwhile, technologies for generating speaker features simultaneously with speaker diarization have been developed, and a representative non-patent document is Non-Patent Document 1. In Non-Patent Document 1, speaker features are calculated for each speech segment obtained by speaker diarization. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-1988 (Patent Application No. 2019-116065) [Non-patent literature]

[0007] [Non-Patent Document 1] Keisuke Kinoshita,et al.,“Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds,”2021 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP).IEEE,2021(to appear). Summary of the Invention [Problem to be solved by the invention]

[0008] When extracting speaker features for each speaker from speech data that contains a mixture of multiple speakers' voices, there is a method of determining the speech periods of all speakers present in the input speech data and calculating speaker features from the speech periods determined for each speaker. However, this method cannot ensure a sufficient amount of speech for the target speaker to be recognized when there are few periods in which only the target speaker is speaking, or when there are only speech periods that overlap with other speakers.

[0009] On the other hand, if speaker features are learned independently of the speech recognition system, the learned speaker features cannot be used by the existing speech recognition system, and therefore it is necessary to reconstruct the speech recognition system from scratch.

[0010] An object of the present invention is to provide a voice activity detection device that can perform voice recognition for each individual speaker from voice data in which the voices of multiple speakers are mixed, in contrast to existing voice recognition systems. [Means for solving the problem]

[0011] One embodiment of the present invention provides a speech activity detection device having a processor and a memory, and includes a speech activity determination unit that receives voice data that is a mixture of voices from a plurality of speakers, determines the speech activity of each speaker from the voice data, and extracts intermediate speaker features from the voice data that can identify the speaker, and a speaker feature extraction unit that extracts speaker features that can identify the speaker based on the voice data and the intermediate speaker features, and the processor adapts the speaker features input to the voice recognition device to a specific speaker. Effect of the Invention

[0012] According to one aspect of the present invention, it is possible to perform individual voice recognition from voice data in which the voices of a plurality of speakers are mixed, in contrast to existing voice recognition systems. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram illustrating an example of a voice activity determination device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of speech segment determination and speaker feature extraction performed by the speech segment determination device according to the embodiment of the present invention. [Diagram 3] FIG. 2 illustrates an example of a speech segment determination unit according to the embodiment of the present invention. [Figure 4] 11 is a flowchart illustrating an example of a process performed by the speech segment determination unit according to the embodiment of the present invention. [Diagram 5] FIG. 2 illustrates an example of a speaker feature extraction unit according to the embodiment of the present invention. [Figure 6] 11 is a flowchart illustrating an example of a process performed by the speaker feature extraction unit according to the embodiment of the present invention. [Figure 7] FIG. 2 illustrates an example of learning of the voice activity detection device according to the embodiment of the present invention. [Figure 8] 3 is a diagram illustrating an example of processing performed by the voice utterance period determination device 100. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. EXAMPLES

[0015] FIG. 1 is a block diagram showing an example of a voice activity detection device 100 according to an embodiment of the present invention. The voice utterance period determination device 100 is configured by a computer including a processor 1, a memory 2, a storage device 3, an input device 4, an output device 5, and a network interface 6.

[0016] An utterance section determination unit 21 and a speaker feature extraction unit 22 are loaded as programs into the memory 2 and executed by the processor 1. Data and the like used by each program are stored in the storage device 3, and in this embodiment, determination voice data 31 is stored therein.

[0017] The input device 4 is composed of a mouse, a keyboard, a touch panel, etc. The output device 5 is composed of a display, etc. The network interface 6 is connected to a network (not shown).

[0018] The processor 1 operates as a functional unit that provides a specific function by processing according to the program of each functional unit. For example, the processor 1 functions as a speech segment determination unit 21 by processing according to a speech segment determination program. The same applies to other programs. Furthermore, the processor 1 also operates as a functional unit that provides each function of multiple processes executed by each program. A computer and a computer system are devices and systems that include these functional units.

[0019] Information such as programs and tables that realize each function of the voice utterance activity detection device 100 can be stored in a storage device such as the storage device 3, a non-volatile semiconductor memory, a hard disk drive, or an SSD (Solid State Drive), or in a computer-readable non-transitory data storage medium such as an IC card, an SD card, or a DVD.

[0020] FIG. 2 is a diagram showing an example of the process of the voice utterance period determination device 100. As shown in FIG. In this embodiment, an example is shown in which the determination use voice data 31 is pre-stored in the storage device 3. Here, the determination use voice data 31 is voice data in which voices of a plurality of speakers are mixed.

[0021] First, the speech section determination unit 201 receives the judgment speech data 31, and generates the speech section determination result 202 of the speaker included in the judgment speech data 31, and the intermediate speaker feature 203 capable of identifying the speaker in the judgment speech data 31. The speech section determination unit 201 generates the speech section determination result 202 and the intermediate speaker feature 203 according to the number of speakers included in the speech.

[0022] The number of speakers included in the determination-use voice data 31 may be input by a user of the voice utterance period determination device 100 via the input device 4, or may be automatically estimated. Next, the speaker feature extraction unit 204 generates speaker features 205 from the judgment speech data 31 and the intermediate speaker features 203 .

[0023] Here, the speech segment determination result 202 output by the speech segment determination section 201 includes a determination as to whether each speaker is present at each time. Finally, a speech recognizer 206 receives the speaker features 205 and performs speaker specific adaptation.

[0024] The speech recognition device 206 may use a known or well-known method capable of adapting to a specific speaker using speaker features that can identify the speaker, such as GMM-HMM (Gaussian Mixture Model, Hidden Markov Model) or DNN-HMM (Deep Neural Network).

[0025] FIG. 3 shows an example of components of the speech segment determination unit 21. The speech segment decision unit 21 includes a speech feature calculation unit 10 , an intermediate feature calculation unit 303 , a speech segment decision calculation unit 304 and an intermediate speaker feature extraction unit 305 .

[0026] First, the speech segment decision unit 21 receives the decision-use speech data 31 , and the speech feature calculation unit 10 calculates the speech feature 302 of the decision-use speech data 31 .

[0027] The intermediate representation calculation unit 303 , the speech segment determination calculation unit 304 and the intermediate speaker feature extraction unit 305 are configured, for example, by a neural network, and the intermediate representation calculation unit 303 has an input layer that receives the speech feature amount 302 .

[0028] For example, a Transformer or a Long Short-term memory (LSTM) can be used as a network configured by the intermediate representation calculation unit 303, the speech segment determination calculation unit 304, and the intermediate speaker feature extraction unit 305. Alternatively, a DNN may be used.

[0029] Next, the intermediate representation calculation unit 303 receives the speech features 302 and calculates an intermediate representation. Then, the intermediate speaker feature extraction unit 305 receives the intermediate representation and extracts the intermediate speaker features 203.

[0030] Finally, the speech segment determination calculation unit 304 outputs the speech segment determination result 202 for each time and each speaker included in the determination speech data 31 from the intermediate representation and intermediate speaker features 203, and calculates, for example, the existence probability for a one-dimensional variable controlled by a time index and a speaker index.

[0031] 4 is a flowchart showing an example of processing performed in the speech segment decision section 21. This processing is processing performed in the speech segment decision section 201 in FIG. First, the speech segment decision unit 21 receives the decision-use speech data 31 (step 501), and the speech feature calculation unit 10 calculates the speech feature 302 of the decision-use speech data 31 (step 502).

[0032] The speech feature 302 output by the speech feature calculation unit 10 is obtained, for example, by processing the input speech data for determination 31 with FFT (Fast Fourier Transform) and calculating MFCC (Mel Frequency Cepstral coefficient) based on the FFT processing result.

[0033] Then, the speech feature calculation unit 10 can calculate the difference between the MFCCs in units (frames) of a predetermined time interval (for example, 10 msec) to obtain the speech feature 302. Note that the speech feature 302 of the determination-use speech data 31 is not limited to the above, and any well-known or publicly known method may be applied.

[0034] Next, the intermediate representation calculation unit 303 receives the speech features 302 and calculates the intermediate representation, and the intermediate speaker feature extraction unit 305 receives the intermediate representation and extracts intermediate speaker features 203 according to the number of speakers included in the judgment speech data 31 (step 503). Finally, the speech segment decision calculation unit 304 calculates the speaker features 205 from the intermediate representation and the intermediate speaker features 203 (step 504).

[0035] FIG. 5 shows an example of components of the speaker feature extraction unit 22. The speaker feature extraction unit 22 includes a speech feature amount calculation unit 10 , an intermediate representation calculation unit 403 , and a speaker feature generation unit 404 .

[0036] First, the speaker feature extraction unit 22 receives the speech data for determination 31 , and the speech feature calculation unit 10 calculates the speech feature 402 of the speech data for determination 31 .

[0037] The intermediate representation calculation unit 403 and the speaker feature generation unit 404 are formed, for example, by a neural network, and the intermediate representation calculation unit 403 has an input layer that receives the speech features 402. Also, the speaker feature generation unit 404 has an input layer that receives the intermediate representation and intermediate speaker features 203. Note that a part or all of the network may be shared with the intermediate representation calculation unit 303 of the speech segment determination unit 21.

[0038] Next, the intermediate representation calculation unit 403 receives the speech features 402 and calculates an intermediate representation. Finally, the speaker feature generation unit 404 receives the intermediate representation and the intermediate speaker features 203 extracted from the speech segment decision unit 21 , and extracts the speaker features 205 .

[0039] 6 is a flowchart showing an example of processing performed in the speaker feature extraction unit 22. This processing is processing performed in the speaker feature extraction unit 204 in FIG. First, the speaker feature extraction unit 22 receives the determination-use speech data 31 (step 601), and the speech feature calculation unit 10 calculates the speech feature 402 of the determination-use speech data 31 (step 602).

[0040] Next, the intermediate representation calculation unit 403 receives the speech features 402 and calculates the intermediate representation. The speaker feature generation unit 404 receives the intermediate representation and the intermediate speaker features 203 extracted from the speech segment determination unit 21, and extracts speaker features 205 according to the number of speakers included in the determination speech data 31 (step 603).

[0041] FIG. 7 shows an example of learning of the voice activity detector. The parameters to be learned here are the intermediate representation calculation unit 303, the speech segment decision calculation unit 304, the intermediate speaker feature extraction unit 305, the intermediate representation calculation unit 403, and the speaker feature generation unit 404. Note that a common intermediate representation calculation unit 801 (see FIG. 8) may be used for the calculation of the intermediate representation.

[0042] First, speech segment determination 201 is executed on training speech data input 701, and speech segment determination result 202 and intermediate speaker features 203 are calculated. Then, speaker feature extraction 204 is executed using training speech data input 701 and intermediate speaker features 203 as inputs, and speaker features 205 are calculated.

[0043] For the speech segment determination result 202, a speech segment loss calculation 703 is performed for the training utterance label 702. For the intermediate speaker features 203, an intermediate speaker feature loss calculation 704 is performed for the training utterance label 702.

[0044] A publicly known or well-known method may be applied as the loss calculated in the speech section loss calculation 703 and the intermediate speaker feature loss calculation 704, for example, binary cross entropy is widely known. Note that the speech section loss is not limited to the above, and other publicly known or well-known methods used for classification problems may be applied.

[0045] For the speaker features 205, speaker feature loss calculation 705 is performed on the training speaker feature labels 706. For example, the training speaker feature labels 706 can be i-vectors. A publicly known or well-known method may be applied as a calculation method for the i-vectors, and is widely known, for example, as described in "Front-end factor analysis for speaker verification." (Dehak, Najim, et al. IEEE Transactions on Audio, Speech, and Language Processing 19.4 (2010): 788-798.). Note that the speaker features 205 are not limited to the above, and other publicly known or well-known methods used in speech recognition systems may be applied.

[0046] The loss calculated by the speaker feature loss calculation 705 may be a publicly known or well-known method, such as least square error, which is widely known. Note that the speaker feature loss is not limited to the above, and other publicly known or well-known methods used in regression problems may also be used. Finally, parameter update 707 is performed to obtain updated parameters.

[0047] The obtained updated parameters are used to replace the corresponding parameters of the intermediate representation calculation unit 303, the speech segment decision calculation unit 304, the intermediate speaker feature extraction unit 305, the intermediate representation calculation unit 403, and the speaker feature generation unit 404. This completes the learning.

[0048] FIG. 8 is a diagram showing an example of the process of the voice utterance period determination device 100. As shown in FIG. In this embodiment, an example is shown in which the determination voice data 31 is stored in advance in the storage device 3. Here, the determination voice data 31 is voice data in which voices of a plurality of speakers are mixed, and may include overlapping speech sections.

[0049] First, the intermediate representation calculation unit 801 receives the speech data for determination 31 and calculates an intermediate representation 802 that represents features effective for speech period determination. The intermediate representation 802 is used as an input to the intermediate speaker feature extraction unit 305, the speech period determination calculation unit 304, and the speaker feature generation unit 404.

[0050] Here, the intermediate representation calculation unit 801 is common to the intermediate speaker feature extraction unit 305, the speech segment determination calculation unit 304 and the speaker feature generation unit 404, but it is also possible to use separate units such as the intermediate representation calculation unit 303 (see FIG. 3) for the intermediate speaker feature extraction unit 305 and the speech segment determination calculation unit 304, and the intermediate representation calculation unit 403 (see FIG. 5) for the speaker feature generation unit 404.

[0051] Next, the intermediate speaker feature extraction unit 305 receives the intermediate representation 802 and generates intermediate speaker features 203 that can identify the speaker only for the input judgment speech data 31. Here, the intermediate speaker features 203 do not guarantee that the intermediate speaker features 203 of the same speaker generated from different judgment data containing the same speaker will be the same.

[0052] Next, the speech segment decision calculation unit 304 receives the intermediate representation 802 and the intermediate speaker features 203 , and generates the speech segment decision result 202 according to the number of speakers included in the decision-use speech data 31 .

[0053] Also, a speaker feature generator 404 receives the intermediate representation 802 and the intermediate speaker features 203, and generates speaker features 205 capable of uniquely identifying a speaker for any judgment speech data 31. Here, the speaker features 205 can be used to adapt a speech recognizer 206 to a particular speaker represented by the speaker features 205.

[0054] Through the above series of processes, the speech utterance period determination device 100 performs speech period determination using the speech period determination unit 201 while extracting intermediate speaker features 203 from the utterance included in the determination speech data 31. Furthermore, the speaker feature extraction unit 204 extracts speaker features based on the intermediate speaker features 203, and generates speaker features 205.

[0055] The generated speaker features 205 are then input to a speech recognition device 206 and used for adaptation to a specific speaker. This makes it possible to extract speaker features with high accuracy from sections where speech from multiple speakers overlap.

[0056] Furthermore, this embodiment may be operated online by dividing the determination speech data 31 into short time segments and then inputting the data to the speech utterance segment determination device 100. In this case, the speaker features 205 can be used to ensure the consistency of the speaker across the divided time segments.

[0057] According to the above embodiment, since the speaker features can be generated by utilizing the voice data of the overlapping sections, the accuracy of the speaker features can be improved. Also, since the voice utterance period determination model is trained to output the speaker features incorporated in the existing voice recognition model, the existing voice recognition model can be adapted to a specific speaker by using the speaker features output by the voice utterance period determination model. [Explanation of symbols]

[0058] 1 processor 2. Memory 3. Storage Device 4 Input Devices 5 Output Devices 6 Network Interface 10. Speech feature calculation unit 21 Speech segment determination unit 22 Speaker feature extraction unit 303 Intermediate feature calculation unit 304 Voice Activity Judgment Calculation Unit 305 Inter-speaker feature extraction unit 403 Intermediate representation calculation section 404 Speaker Feature Generation Unit

Claims

1. A voice activity detection device having a processor and a memory, a speech section determination unit that receives speech data in which speeches of a plurality of speakers are mixed, determines speech sections of each speaker from the speech data, and extracts intermediate speaker features from the speech data that enable identification of the speakers; a speaker feature extraction unit that extracts speaker features capable of identifying the speaker based on the speech data and the intermediate speaker features, The processor, Using the speaker features to adapt a speech recognizer to a particular speaker; The speech segment determination unit The speech feature calculation unit includes an intermediate representation calculation unit, a speech segment determination calculation unit, and an intermediate speaker feature extraction unit, The speech feature calculation unit Calculating a speech feature of the speech data; The intermediate representation calculation unit: receiving the speech features and computing an intermediate representation; The intermediate speaker feature extraction unit receiving the intermediate representation and extracting the intermediate speaker features according to the number of speakers included in the speech data; The speech segment determination calculation unit is outputting a speech segment determination result for each time and each speaker included in the speech data from the intermediate representation and the intermediate speaker features; The speaker feature extraction unit is The speech feature calculation unit, the intermediate representation calculation unit, and the speaker feature generation unit are included, The speaker feature generation unit includes: receiving the intermediate representation and the intermediate speaker features, and extracting speaker features according to the number of speakers included in the speech data; Regarding learning of parameters used in the intermediate representation calculation unit, the speech segment determination calculation unit, the intermediate speaker feature extraction unit, and the speaker feature generation unit, Executing the speech interval determination for the training speech data input, and calculating the speech interval determination result and the intermediate speaker features; Executing the extraction of the speaker features using the training speech data input and the intermediate speaker features as inputs, and calculating the speaker features; Execute a speech segment loss calculation for a learning utterance label with respect to the speech segment determination result; performing an intermediate speaker feature loss calculation for the training utterance labels on the intermediate speaker features; performing a speaker feature loss calculation on the speaker features against training speaker feature labels; a speech segment detection device which performs parameter updating to obtain updated parameters, and completes the learning by using the updated parameters to replace the corresponding parameters of the intermediate representation calculation unit, the speech segment judgment calculation unit, the intermediate speaker feature extraction unit, and the speaker feature generation unit.

2. 2. The speech activity detection device according to claim 1, The input device further includes:

4. The speech activity detection device according to claim 1, wherein the number of speakers included in the voice data is input from the input device.

3. 2. The speech activity detection device according to claim 1, The speech segment determination unit A speech activity detection device that determines whether each speaker is present at each time.

4. 2. The speech activity detection device according to claim 1, The processor, The speech segment determination unit and the speaker feature extraction unit are regarded as one neural network having the parameters, a speech section loss for a correct answer of the speech section determination result obtained by the speech section determination unit by executing the speech section loss calculation, an intermediate speaker feature loss for the intermediate speaker features obtained by the speech section determination unit by executing the intermediate speaker feature loss calculation, and a speaker feature loss for the speaker features obtained by the speaker feature extraction unit by executing the speaker feature loss calculation are each reduced.

5. 1. A method for voice activity detection having a processor and a memory, comprising: a speech interval determination step of receiving speech data in which speeches of a plurality of speakers are mixed, determining speech intervals of each speaker from the speech data, and extracting intermediate speaker features from the speech data that enable identification of the speakers; a speaker feature extraction step of extracting speaker features capable of identifying the speaker based on the speech data and the intermediate speaker features; an adapting step for adapting a speech recognition device to a particular speaker using said speaker features; having The speech segment determination step includes: The speech feature calculation step, the intermediate representation calculation step, the speech segment determination calculation step, and the intermediate speaker feature extraction step are included. The speech feature calculation step includes: Calculating a speech feature of the speech data; The intermediate representation calculation step includes: receiving the speech features and computing an intermediate representation; The intermediate speaker feature extraction step includes: receiving the intermediate representation and extracting the intermediate speaker features according to the number of speakers included in the speech data; The speech segment determination calculation step includes: outputting a speech segment determination result for each time and each speaker included in the speech data from the intermediate representation and the intermediate speaker features; The speaker feature extraction step includes: The speech feature calculation step, the intermediate representation calculation step, and the speaker feature generation step are included, The speaker feature generating step includes: receiving the intermediate representation and the intermediate speaker features, and extracting speaker features according to the number of speakers included in the speech data; Regarding learning of parameters used in the intermediate representation calculation step, the speech segment determination calculation step, the intermediate speaker feature extraction step, and the speaker feature generation step, Executing the speech interval determination for the training speech data input, and calculating the speech interval determination result and the intermediate speaker features; Executing the extraction of the speaker features using the training speech data input and the intermediate speaker features as inputs, and calculating the speaker features; Execute a speech segment loss calculation for a learning utterance label with respect to the speech segment determination result; performing an intermediate speaker feature loss calculation for the training utterance labels on the intermediate speaker features; performing a speaker feature loss calculation on the speaker features against training speaker feature labels; 2. A speech segment detection method comprising: performing parameter updating to obtain updated parameters; and using the updated parameters to replace the corresponding parameters in the intermediate representation calculation step, the speech segment determination calculation step, the intermediate speaker feature extraction step, and the speaker feature generation step, thereby completing the learning.

6. A speech activity detection program for causing a computer having a processor and a memory to execute speech activity detection, comprising: a speech segment determination program that receives speech data in which the speeches of a plurality of speakers are mixed, determines speech segments of each speaker from the speech data, and extracts intermediate speaker features from the speech data that enable identification of the speakers; a speaker feature extraction program for extracting speaker features capable of identifying the speaker based on the speech data and the intermediate speaker features, Using the speaker features to adapt a speech recognizer to a particular speaker; The speech segment determination program includes: causing the computer to function as a speech feature calculation unit, an intermediate representation calculation unit, a speech segment determination calculation unit, and an intermediate speaker feature extraction unit; The speech feature calculation unit Calculating a speech feature of the speech data; The intermediate representation calculation unit: receiving the speech features and computing an intermediate representation; The intermediate speaker feature extraction unit receiving the intermediate representation and extracting the intermediate speaker features according to the number of speakers included in the speech data; The speech segment determination calculation unit is outputting a speech segment determination result for each time and each speaker included in the speech data from the intermediate representation and the intermediate speaker features; The speaker feature extraction program is causing the computer to function as the speech feature calculation unit, the intermediate representation calculation unit, and the speaker feature generation unit; The speaker feature generation unit includes: receiving the intermediate representation and the intermediate speaker features, and extracting speaker features according to the number of speakers included in the speech data; Regarding learning of parameters used in the intermediate representation calculation unit, the speech segment determination calculation unit, the intermediate speaker feature extraction unit, and the speaker feature generation unit, Executing the speech interval determination for the training speech data input, and calculating the speech interval determination result and the intermediate speaker features; Executing the extraction of the speaker features using the training speech data input and the intermediate speaker features as inputs, and calculating the speaker features; Execute a speech segment loss calculation for a learning utterance label with respect to the speech segment determination result; performing an intermediate speaker feature loss calculation for the training utterance labels on the intermediate speaker features; performing a speaker feature loss calculation on the speaker features against training speaker feature labels; 2. A speech activity detection program comprising: a speech activity detection unit that performs parameter updating to obtain updated parameters; and a speech activity detection unit that uses the updated parameters to replace corresponding parameters of the intermediate representation calculation unit, the speech activity determination calculation unit, the intermediate speaker feature extraction unit, and the speaker feature generation unit, thereby completing the learning.

Images