Voice recognition device, voice recognition method, and program

Abstract

To recognize voice of a plurality of specified speakers and to control a device according to the specified speaker and recognized voice.SOLUTION: A voice recognition device includes: a storage part; a voice recognition part; an analysis part; a segmentation part; an embedding vector calculation part; a similarity calculation part; a determination part; and a device control part. The storage part stores a first speaker embedding vector of one or more prescribed registered speakers and individual setting of the registered speaker. The analysis part analyzes an acoustic signal and extracts a feature amount. The segmentation part segments a series of the feature amount included in an utterance section from a voice recognition result. The embedding vector calculation part calculates a second speaker embedding vector by using the series of the feature amount. The similarity calculation part calculates one or more similarities between the second speaker embedding vector and one or more first speaker embedding vectors. The device control part controls a device by individual setting which is read from the storage part based on the registered speaker determined from one or more similarities and the voice recognition result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a voice recognition device, a voice recognition method, and a program. Regarding. [Background technology]

[0002] There are known speech recognition technologies that recognize only the speech of a specific speaker. For example, as a method for recognizing only the speech of a speaker specified by given speaker information, there is a technology that learns to recognize only the speech of the speaker by combining input acoustic features with a speaker embedding vector. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-539674 [Non-patent literature]

[0004] [Non-Patent Document 1] Wan et al., Generalized End-to-End Loss for Speaker Verification, ICASSP 2018, pp.4879-4883, 2018 [Non-Patent Document 2] Synder et al., X-Vectors: Robust DNN Embeddings for Speaker Recognition, ICASSP 2018, pp.5329-5333, 2018 2018 [Non-Patent Document 3] Ioffe, Probabilistic linear discriminant analysis, ECCV, Part IV, LNCS 3954, pp.531-542, 2006 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional technology, it has been difficult to recognize the voices of multiple specific speakers and control devices in accordance with the identified speakers and the recognized voices. [Means for solving the problem]

[0006] A voice recognition device according to an embodiment includes a storage unit, a voice recognition unit, an analysis unit, a cutout unit, an embedding vector calculation unit, a similarity calculation unit, a determination unit, and a device control unit. The storage unit stores a first speaker embedding vector of one or more given registered speakers, and personal settings of the registered speakers used to control a device. The voice recognition unit recognizes voice from an acoustic signal and obtains a voice recognition result. The analysis unit analyzes the acoustic signal and extracts features representing characteristics of a waveform of the acoustic signal. The cutout unit cuts out a sequence of the features included in a speech section from the voice recognition result. The embedding vector calculation unit calculates a second speaker embedding vector using the sequence of the features. The similarity calculation unit calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors. The determination unit determines which of the one or more registered speakers an utterance belongs to based on the one or more similarities. The device control unit controls the device according to the personal settings read from the storage unit, based on the registered speaker determined from the one or more similarities and the voice recognition result. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional configuration of a voice recognition device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of a recognition unit according to the embodiment. [Figure 3A] FIG. 3A is a flowchart showing an example of an overall flow of speaker registration according to the embodiment. [Figure 3B] FIG. 3B is a flowchart showing an example of a detailed flow of step S2 in the embodiment. [Figure 4]FIG. 4 is a flowchart illustrating an example of a voice recognition flow according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a functional configuration of a recognition unit according to the first modification of the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of keyword information according to the first modification of the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a functional configuration of a voice recognition device according to a second modification of the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a hardware configuration of the voice recognition device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a voice recognition device, a voice recognition method, and a program will be described in detail with reference to the accompanying drawings.

[0009] The speech recognition device 100 of the embodiment calculates a speaker embedding vector for a speech section in which a keyword is detected by keyword spotting, using a speaker embedding model trained independently of the speech recognition model. The speech recognition device 100 of the embodiment performs speaker identification based on the similarity with each speaker registered in advance. Then, the speech recognition device 100 of the embodiment controls devices such as an audio device and an air conditioner based on the detected keyword and the result of the speaker identification. This makes it possible to perform speech recognition for multiple speakers and perform unique control according to the speaker.

[0010] [Example of functional configuration] 1 is a diagram showing an example of a functional configuration of a voice recognition device 100 according to an embodiment. The voice recognition device 100 according to the embodiment includes a microphone 1, a voice recognition unit 2, a classification unit 3, a determination unit 4, a display control unit 5, a display 6, and a device control unit 7. The voice recognition unit 2 includes a first analysis unit 21 and a detection unit 22. The voice recognition device 100 according to the embodiment also includes a voice recognition model storage unit 101, an embedded model storage unit 102, and a personal setting storage unit 103.

[0011] The microphone 1 picks up the voices of one or more speakers, and inputs the acoustic signals picked up at each time to the first analysis unit 11 and the classification unit 3 .

[0012] The first analysis unit 21 extracts features representing characteristics of the waveform of an audio signal from the audio signal input at each time from the microphone 1. The extracted features include, for example, MFCC (Mel-Frequency Cepstrum Coefficients) and Mel filter bank features.

[0013] The detection unit 22 detects a keyword from the sequence of feature quantities, and inputs keyword detection information including the keyword detection result, the keyword start time, and the keyword end time to the identification unit 3.

[0014] 2 is a diagram illustrating an example of a functional configuration of the classification unit 3 according to the embodiment. The classification unit 3 according to the embodiment includes a second analysis unit 31, a cutout unit 32, an embedding vector calculation unit 33, a registration unit 34, a similarity calculation unit 35, and an embedding vector storage unit 104.

[0015] The second analysis unit 31 has a feature extraction function similar to that of the above-described first analysis unit 21. Note that the feature extracted by the second analysis unit 31 may be the same as the feature extracted by the first analysis unit 21. In this case, the identification unit 3 may receive the feature from the first analysis unit 21.

[0016] The extraction unit 32 receives features from the second analysis unit 31 and keyword information from the detection unit 22, and then extracts a series of features included between the start time of the keyword and the end time of the keyword (keyword detection section).

[0017] The embedding vector calculation unit 33 reads the speaker embedding model from the embedding model storage unit 102, and calculates the speaker embedding vector based on the speaker embedding model. For example, i-vector, d-vector (Non-Patent Document 1), x-vector (Non-Patent Document 2), and their derived methods are used as a method of calculating the speaker embedding vector.

[0018] The registration unit 34 averages a predetermined number of pre-calculated speaker embedding vectors for the speaker to be speech recognized (to be registered), and stores the averaged speaker embedding vector in the embedding vector storage unit 104.

[0019] The similarity calculation unit 35 calculates the similarity between the speaker embedding vector calculated by the embedding vector calculation unit 33 and the speaker embedding vector of the registered speaker stored in the embedding vector storage unit 104. As a method for calculating the similarity, for example, cosine similarity, PLDA (Non-Patent Document 3), etc. are used.

[0020] Returning to Figure 1, the judgment unit 4 judges the speaker of the utterance included in the keyword detection section based on the similarity calculated by the similarity calculation unit 35, and inputs the speaker judgment result (identification result) to the display control unit 5 and the equipment control unit 7.

[0021] The display control unit 5 displays, on the display 6, information for identifying the speaker whose voice has been recognized, based on the determination result input from the determination unit 4.

[0022] Based on the judgment result input from the judgment unit 4, the equipment control unit 7 reads out the personal settings of the identified speaker from the personal setting storage unit 103, and controls equipment such as the air conditioner 111 and the audio device 112 based on the personal settings.

[0023] For example, the device control unit 7 controls the device according to a combination of the speaker identification result and the voice recognition result. Specifically, for example, when the audio device 112 is a car audio device, if the speaker says "Play my favorite song," the device control unit 7 plays the favorite song according to the speaker defined by personal settings. For example, for the air conditioner 111, if the speaker says "Preference settings," the device control unit 7 sets the preferred temperature and air volume according to the speaker defined by personal settings.

[0024] 3A is a flowchart showing an example of the overall flow of speaker registration according to the embodiment. First, the registration unit 34 initializes a variable k for identifying K speakers to be registered to 1 (step S1).

[0025] Next, the registration unit 34 executes speaker registration of the speaker k (step S2). A detailed flow of step S2 will be described later with reference to FIG. 3B.

[0026] Next, the registration unit 34 determines whether or not k=K (step S3). If k=K (Yes at step S3), the speaker registration process ends.

[0027] If k=K is not satisfied (No at step S3), the registration unit 34 increments the value of k (step S4), and the speaker registration process returns to step S2.

[0028] 3B is a flowchart showing an example of a detailed flow of step S2 in the embodiment. First, the registration unit 34 initializes a vector V and a variable n used in calculating the average of the speaker embedding vector to 0 (step S11).

[0029] Next, the speech recognition unit 2 recognizes the speech of the speaker k (step S12). Specifically, the first analysis unit 21 extracts features from the acoustic signal of the speaker k input from the microphone 1 at each time, and the detection unit 22 attempts to detect a keyword from the sequence of the features.

[0030] If no keyword is detected by the speech recognition in step S12 (step S13, No), the speaker registration process returns to step S12, and speech recognition of speaker k is continued.

[0031] If a keyword is detected by the speech recognition in step S12 (Yes in step S13), the cut-out unit 32 cuts out a feature amount included in the keyword detection section (step S14).

[0032] Next, the embedding vector calculation unit 33 calculates a speaker embedding vector v from the feature amount extracted in step S14 (step S15).

[0033] Next, the registration unit 34 adds the speaker embedding vector v calculated in step S15 to the vector V, and increments the variable n (step S16). Next, the registration unit 34 determines whether n=N (step S17). If n=N is not true (step S17, No), the speaker registration process returns to step S12, and speech recognition of speaker k is continued.

[0034] If n=N (step S17, Yes), the registration unit 34 calculates the average of the speaker embedding vector v of speaker k calculated in step S15 by dividing the vector V by N, and registers the vector V / N in the embedding vector storage unit 104 as the speaker embedding vector of speaker k (step S18).

[0035] As shown in Fig. 3B, the registration unit 34 calculates a speaker embedding vector (first speaker embedding vector) for each of N utterances (N ≥ 1) by the same speaker. Then, the registration unit 34 registers the statistics of each speaker embedding vector (average in the example of Fig. 3B) as the speaker embedding vector of the same speaker in the embedding vector storage unit 104. By inputting multiple utterances when registering a speaker, it is possible to reduce the variation in the speaker embedding vector for each utterance within a speaker.

[0036] 4 is a flowchart showing an example of a voice recognition flow according to an embodiment. First, the voice recognition unit 2 recognizes the voice input from the microphone 1 (step S21). Specifically, the first analysis unit 21 extracts features from the acoustic signal input from the microphone 1 at each time, and the detection unit 22 attempts to detect a keyword from the sequence of the features.

[0037] If no keyword is detected by the speech recognition in step S21 (step S22, No), the speaker registration process returns to step S21 and continues the speech recognition of the speaker k.

[0038] If a keyword is detected by the speech recognition in step S21 (Yes in step S22), the cut-out unit 32 cuts out a feature amount included in the keyword detection section (step S23).

[0039] Next, the embedding vector calculation unit 33 calculates the speaker embedding vector v from the feature extracted in step S23 (step S24). Next, the similarity calculation unit 35 calculates the similarity between the speaker embedding vector v calculated in step S24 and the speaker embedding vector of the registered speaker stored in advance in the embedding vector storage unit 104 by the registration unit 34 (step S25).

[0040] Next, the judgment unit 4 judges the speaker of the utterance included in the keyword detection section based on the similarity calculated in step S25, and the equipment control unit 7 controls the equipment according to the personal settings of the identified speaker based on the speaker judgment result (identification result) (step S26).

[0041] Next, the display control unit 5 displays information for identifying the speaker whose voice was recognized in step S21 on the display 6 (step S27).

[0042] As described above, in the voice recognition device 100 of the embodiment, the embedding model storage unit 102 and the personal setting storage unit 103 (an example of a storage unit) store the first speaker embedding vector of one or more given registered speakers and the personal settings of the registered speakers used to control the device. The voice recognition unit 2 recognizes voice from the acoustic signal and obtains the voice recognition result. The second analysis unit 31 analyzes the acoustic signal and extracts features representing the characteristics of the waveform of the acoustic signal. The cut-out unit 32 cuts out a series of features included in the speech section from the voice recognition result. The embedding vector calculation unit 33 calculates a second speaker embedding vector using the series of features. The similarity calculation unit 35 calculates one or more similarities between the second speaker embedding vector and one or more first speaker embedding vectors. The determination unit 4 determines which of the one or more registered speakers the utterance is from based on the one or more similarities. Then, the device control unit 7 controls the device according to the personal settings read out from the personal setting storage unit 103, based on the registered speaker determined from one or more similarities and the voice recognition result.

[0043] According to the configuration of the speech recognition device 100 of the embodiment, speech recognition for specific multiple speakers is performed using a speaker embedding model (first speaker embedding vector of one or more given registered speakers) that is independent of the speech recognition model used for speech recognition. This makes it possible to recognize the speech of specific multiple speakers and control equipment according to the identified speakers and the recognized speech.

[0044] In addition, because conventional systems only recognize the voice of a given speaker, there was a problem that it was not possible to recognize multiple target speakers, such as when it is necessary to recognize the voices of speaker A and speaker B. In addition, because the speech recognition model relies on a speaker embedding model, there was a problem that tuning the model to suit the environment was complicated, such as the need to retrain both models to respond to environmental changes and aging.

[0045] (First Modification of the Embodiment) Next, a first modification of the embodiment will be described. In the description of the first modification, the same description as in the embodiment will be omitted, and only the differences from the embodiment will be described.

[0046] Fig. 5 is a diagram illustrating an example of a functional configuration of the classification unit 3 in the first modified example of the embodiment. In the example of Fig. 5, the similarity calculation unit 35 further reads keyword information from the keyword information storage unit 105. The keyword information storage unit 105 stores combinations of registered speakers and keywords.

[0047] 6 is a diagram showing an example of keyword information according to the first modification of the embodiment. The keyword information according to the first modification represents a list of acceptable keywords for each of speakers A and B. For example, it indicates that keyword b is acceptable for speaker A but not acceptable for speaker B. The keyword information is used as a filter that determines acceptable keywords for each speaker.

[0048] The similarity calculation unit 35 may receive the speech recognition result of the keyword detection section from the extraction unit 32 via the embedding vector calculation unit 33, and calculate only the similarity with the speaker who can input the speech-recognized keyword. This can reduce the cost of similarity calculation.

[0049] That is, the similarity calculation unit 35 calculates the similarity between the first speaker embedding vector of a registered speaker whose speech recognition result includes an acceptable keyword defined in the keyword information of Fig. 6 and the second speaker embedding vector calculated by the embedding vector calculation unit 33. That is, the similarity calculation unit 35 does not calculate the similarity between the first speaker embedding vector of a registered speaker whose speech recognition result does not include an acceptable keyword defined in the keyword information of Fig. 6 and the second speaker embedding vector calculated by the embedding vector calculation unit 33.

[0050] The embedding vector calculation unit 33 may further receive the keyword recognition result from the speech recognition unit 2 via the extraction unit 32, and calculate the speaker embedding vector from the acoustic feature and the keyword recognition result. The keyword recognition result may be a keyword ID or a character string corresponding to the utterance, or an acoustic score at each time. The acoustic score indicates the probability that the voice at each time corresponds to each phoneme.

[0051] That is, the speech recognition result may include an acoustic score indicating the probability that the voice at each time corresponds to each phoneme, and the embedding vector calculation unit 33 may calculate a speaker embedding vector (second speaker embedding vector) from the acoustic score at each time and the feature at each time included in the feature sequence. This is expected to improve the identification performance when the speech content differs between the time of enrollment and the time of identification.

[0052] The determination unit 4 may further determine that the utterance is not that of any registered speaker based on a threshold value (first threshold value) of the similarity. In this case, the display control unit 5 displays on the display 6 information (such as a name) identifying the registered speaker determined from one or more similarities, and when all of the one or more similarities are equal to or lower than the first threshold value, displays on the display 6 information indicating that the reliability of the speaker identification accuracy is equal to or lower than the first threshold value.

[0053] This threshold value may be a fixed value, or the determination unit 4 may further receive a keyword detection result and use a different threshold value depending on the keyword detection result.

[0054] This makes it possible to reject utterances by speakers other than those registered. Also, when the detection unit 22 mistakenly reacts to non-voice signals such as environmental noise and outputs a detection result, it is possible to reject such a detection result.

[0055] In addition, if the similarity between the input utterance and the registered speaker embedding vector is below a threshold, it is possible to encourage the speaker to speak again or to reject errors in the speech recognition result caused by background noise.

[0056] (Modification 2 of the embodiment) Next, a description will be given of Modification 2 of the embodiment. In the description of Modification 2, the description of the same parts as in the embodiment will be omitted, and only the parts different from the embodiment will be described.

[0057] 7 is a diagram illustrating an example of a functional configuration of a voice recognition device 100-2 according to Modification 2 of the embodiment. The voice recognition unit 2 of Modification 2 includes a spontaneous speech recognition unit 23 and a language understanding unit 24 instead of the detection unit 22 of the embodiment.

[0058] The free speech recognition unit 23 recognizes free speech that does not depend on predetermined keywords, and converts the speech recognition result into a character string.

[0059] The language understanding unit 24 analyzes the character string obtained by the spontaneous speech recognition unit 23. For example, the language understanding unit 24 obtains a language understanding result that is understood from the character string based on a language understanding model.

[0060] Based on the language understanding result, the similarity calculation unit 35 selects one or more first speaker embedding vectors to be used for similarity calculation, and calculates the similarity between the selected one or more first speaker embedding vectors and the second speaker embedding vector calculated by the embedding vector calculation unit 33.

[0061] The configuration of Modification 2 shown in Fig. 7 can also be used for more advanced voice input interfaces that are not restricted by specific keywords. By applying speaker identification to large vocabulary speech recognition, it becomes possible to control the operation of a voice interface based on speaker information for a wider range of tasks.

[0062] For example, when the speech recognition device 100-2 of the second modification recognizes the speech of two people in the driver's seat and the passenger seat of a car, it can identify that the speech is related to driving assistance of the car from the language understanding result. In this case, the similarity calculation unit 35 can select, for example, the first speaker embedding vector of the registered speaker who is registered as the driver of the car as the target for calculating the similarity.

[0063] Also, instead of using a fixed number of utterances, when registering a speaker, registration may be terminated when the variance of the speaker embedding vector of each utterance becomes equal to or less than a threshold. That is, the registration unit 34 may prompt the same speaker to repeatedly speak, calculate the respective speaker embedding vectors (first speaker embedding vectors) for each utterance, and prompt the speaker to stop speaking when the variance of each speaker embedding vector becomes equal to or less than a second threshold.

[0064] This allows speaker registration with a minimum number of utterances, reducing the effort required for speaker registration and improving the user experience while maintaining speaker identification accuracy.

[0065] In addition, at the time of judgment, the value of the embedding vector storage unit may be updated sequentially using the utterance whose similarity is equal to or greater than the threshold. Since voice quality changes over time, if the initially registered speaker embedding vector continues to be used, the speaker identification accuracy may decrease, and re-registration may be required. Therefore, the registration unit 34 may update the embedding vector of the registered speaker whose similarity is equal to or greater than the threshold (third threshold) (the first speaker embedding vector registered in the embedding vector storage unit 104) using the speaker embedding vector whose similarity is equal to or greater than the threshold (third threshold) (the second speaker embedding vector calculated by the embedding vector calculation unit 33).

[0066] Sequential updating by the registration unit 34 eliminates the need to perform explicit re-registration work periodically (for example, re-registration work every predetermined number of years), and makes it possible to maintain speaker identification accuracy over time.

[0067] Finally, an example of the hardware configuration of the voice recognition device 100 according to the embodiment will be described.

[0068] [Example of hardware configuration] 7 is a diagram illustrating an example of a hardware configuration of the speech recognition device 100 according to the embodiment. The speech recognition device 100 according to the embodiment includes a processor 201, a main storage device 202, an auxiliary storage device 203, a display device 204, an input device 205, and a communication device 206. The processor 201, the main storage device 202, the auxiliary storage device 203, the display device 204, the input device 205, and the communication device 206 are connected via a bus 210.

[0069] It should be noted that some of the above configurations may not be included in the speech recognition device 100. For example, if the speech recognition device 100 can use the input function and display function of an external device, the speech recognition device 100 may not be provided with the display device 204 and the input device 205.

[0070] The processor 201 executes a program read from the auxiliary storage device 203 to the main storage device 202. The main storage device 202 is a memory such as a ROM and a RAM. The auxiliary storage device 203 is a hard disk drive (HDD), a memory card, or the like.

[0071] The display device 204 is, for example, a liquid crystal display (display 6 in the example of FIG. 1). The input device 205 is an interface for operating the voice recognition device 100. The display device 204 and the input device 205 may be realized by a touch panel or the like having a display function and an input function. The communication device 206 is an interface for communicating with other devices.

[0072] For example, the program executed by the speech recognition device 100 is provided as a computer program product in the form of a file in an installable or executable format, recorded on a computer-readable storage medium such as a memory card, a hard disk, a CD-RW, a CD-ROM, a CD-R, a DVD-RAM, or a DVD-R.

[0073] Also, for example, the program executed by the voice recognition device 100 may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network.

[0074] Also, for example, the program executed by the speech recognition device 100 may be provided via a network such as the Internet without being downloaded. Specifically, the speech recognition process may be performed by a so-called ASP (Application Service Provider) type service, which does not transfer the program from a server computer but realizes the processing function only by issuing an execution instruction and obtaining the result.

[0075] Also, for example, the program for the voice recognition device 100 may be provided by being pre-installed in a ROM or the like.

[0076] The program executed by the speech recognition device 100 has a modular configuration including functions that can be realized by the program among the above-mentioned functional configurations. As for each function, as actual hardware, the processor 201 reads the program from a storage medium and executes it, and the above-mentioned functional blocks are loaded onto the main storage device 202. In other words, the above-mentioned functional blocks are generated on the main storage device 202.

[0077] Note that some or all of the above-described functions may be realized by hardware such as an integrated circuit (IC) instead of by software.

[0078] Furthermore, each function may be realized using a plurality of processors 201. In this case, each processor 201 may realize one of the functions, or may realize two or more of the functions.

[0079] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0080] 1 microphone 2. Voice Recognition Unit 3. Identification section 4 Judgment section 5. Display control section 6 Display 7. Equipment Control Section 21 1st Analysis Department 22 Detection unit 23 Spontaneous speech recognition unit 24 Language Understanding Department 31 Second Analysis Department 32 Cut-out section 33 Embedding Vector Calculation Unit 34 Registration Department 35 Similarity calculation part 100 Voice recognition device 101 Speech recognition model storage unit 102 Embedded model storage unit 103 Personal setting memory 104 Embedding Vector Storage Unit 105 Keyword information storage unit 111 Air conditioner 112 Sound equipment 201 Processor 202 Main storage 203 Auxiliary storage device 204 Display device 205 Input Device 206 Communication Equipment 210 Bus

Claims

1. a memory unit for storing a first speaker embedding vector for one or more given enrollment speakers and personal settings for the enrollment speakers for use in controlling a device; a speech recognition unit that recognizes speech from an acoustic signal and obtains a speech recognition result; an analysis unit that analyzes the acoustic signal and extracts a feature quantity that represents a waveform feature of the acoustic signal; an extracting unit that extracts a sequence of the feature amounts included in a speech section from the speech recognition result; an embedding vector calculation unit that calculates a second speaker embedding vector using the sequence of features; a similarity calculation unit that calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors; a determination unit that determines which of the one or more registered speakers the utterance is from based on the one or more similarities; a device control unit that controls the device according to the personal settings read from the storage unit, based on a registered speaker determined from the one or more similarities and the speech recognition result; A voice recognition device comprising:

2. a display control unit that displays, on a display device, information identifying a registered speaker determined from the one or more similarities, and, when all of the one or more similarities are equal to or less than a first threshold, displays, on the display device, information indicating that the reliability of the speaker identification accuracy is equal to or less than the first threshold; The speech recognition device according to claim 1 , further comprising:

3. The storage unit further stores a combination of the registered speaker and a keyword, the similarity calculation unit calculates a similarity between the speech recognition result and a first speaker embedding vector of a registered speaker whose keyword is included in the speech recognition result, and does not calculate a similarity between the speech recognition result and a first speaker embedding vector of a registered speaker whose keyword is not included in the speech recognition result.

3. The speech recognition device according to claim 1 or 2.

4. the speech recognition unit converts the speech recognition result into a character string, and further obtains a language understanding result that is understood based on a language understanding model from the character string; the similarity calculation unit selects one or more first speaker embedding vectors to be subjected to similarity calculation based on the language understanding result, and calculates a similarity between the one or more selected first speaker embedding vectors.

3. The speech recognition device according to claim 1 or 2.

5. A registration unit that registers the one or more first speaker embedding vectors in the storage unit, The registration unit calculates a first speaker embedding vector for each of N utterances (N≧1) by the same speaker, and registers statistics of the first speaker embedding vectors in the storage unit as first speaker embedding vectors of the same speaker.

3. The speech recognition device according to claim 1 or 2.

6. The registration unit repeatedly prompts the same speaker to speak, calculates a first speaker embedding vector for each utterance, and prompts the speaker to stop speaking when a variance of each of the first speaker embedding vectors becomes equal to or less than a second threshold.

6. The speech recognition device according to claim 5.

7. The registration unit updates the first speaker embedding vector whose similarity is equal to or greater than a third threshold, using the second speaker embedding vector whose similarity is equal to or greater than a third threshold.

6. The speech recognition device according to claim 5.

8. the speech recognition result includes an acoustic score indicating a probability that the speech at each time corresponds to each phoneme; the embedding vector calculation unit calculates the second speaker embedding vector from the acoustic score at each time and the feature amount at each time included in the feature amount series; 3. The speech recognition device according to claim 1 or 2.

9. storing in a memory unit, a first speaker embedding vector for one or more given enrollment speakers and personal settings for said enrollment speakers that are used to control the device; a step of the speech recognition device recognizing speech from an acoustic signal and obtaining a speech recognition result; a step of the speech recognition device analyzing the acoustic signal and extracting a feature quantity representing a waveform feature of the acoustic signal; a step of the speech recognition device extracting a sequence of the feature amounts included in a speech section from the speech recognition result; The speech recognition device calculates a second speaker embedding vector using the sequence of features; the speech recognizer calculating one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors; a step of determining, by the speech recognition device, which of the one or more registered speakers the utterance is based on the one or more similarities; a step of controlling the device by the personal setting read from the storage unit based on the speech recognition result and the registered speaker determined from the one or more similarities; A speech recognition method comprising:

10. A computer including a memory unit for storing a first speaker embedding vector for one or more given enrollment speakers and personal settings for the enrollment speakers used to control a device, a speech recognition unit that recognizes speech from an acoustic signal and obtains a speech recognition result; an analysis unit that analyzes the acoustic signal and extracts a feature quantity that represents a waveform feature of the acoustic signal; an extracting unit that extracts a sequence of the feature amounts included in a speech section from the speech recognition result; a speaker embedding vector calculation unit that calculates a second speaker embedding vector using the sequence of the feature quantities; a similarity calculation unit that calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors; a determination unit that determines which of the one or more registered speakers the utterance is from based on the one or more similarities; a device control unit that controls the device according to the personal settings read from the storage unit, based on a registered speaker determined from the one or more similarities and the speech recognition result; A program that functions as a

Images