Speech recognition device, speech recognition method, and storage medium

Abstract

The speech of a specific plurality of speakers can be recognized and the device can be controlled according to the determined speaker and the recognized speech. A speech recognition apparatus according to an embodiment includes a storage unit, a speech recognition unit, an analysis unit, an interception unit, an embedded vector calculation unit, a similarity calculation unit, a determination unit, and a device control unit. The storage unit stores a first speaker embedding vector for a given one or more registered speakers and personal settings for the registered speakers. The analysis unit analyzes the acoustic signal and extracts a feature quantity. The interception unit intercepts a sequence of feature quantities included in the speech generation section from the speech recognition result. An embedding vector calculation unit calculates a second speaker embedding vector using the sequence of feature quantities. A similarity calculation unit calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors. The device control unit controls the device by the personal setting read from the storage unit on the basis of the registered speaker determined from the one or more similarities and the speech recognition result.

Description

Technical Field

[0001] The embodiments of the present invention relate to a speech recognition device, a speech recognition method and a storage medium. Background Art

[0002] Conventionally, speech recognition technology that recognizes only the speech of a specific speaker is known. For example, as a method of recognizing only the speech of a speaker identified by given speaker information, there is a technology that combines a speaker embedding vector with an input acoustic feature quantity to learn in a manner that recognizes only the speech of the speaker. Summary of the invention

[0003] However, in the related art, it is difficult to recognize the voices of specific multiple speakers and control a device according to the determined speakers and the recognized voices.

[0004] The speech recognition device of the embodiment includes a storage unit, a speech recognition unit, an analysis unit, a clipping unit, an embedding vector calculation unit, a similarity calculation unit, a determination unit, and a device control unit. The storage unit stores a first speaker embedding vector of one or more given registered speakers and personal settings of the registered speakers used for controlling the device. The speech recognition unit recognizes speech based on an acoustic signal and obtains a speech recognition result. The analysis unit analyzes the acoustic signal and extracts a feature quantity representing a feature of a waveform of the acoustic signal. The clipping unit clips a sequence (Japanese: series) of the feature quantities included in the utterance interval from the speech recognition result. The embedding vector calculation unit calculates a second speaker embedding vector using the sequence of feature quantities. The similarity calculation unit calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors. The determination unit determines which speaker of the one or more registered speakers is uttering based on the one or more similarities. The device control unit controls the device based on the registered speaker determined based on the one or more similarities and the speech recognition result by the personal settings read out from the storage unit.

[0005] According to the above-mentioned voice recognition device, it is possible to recognize the voices of a plurality of specific speakers and control the device based on the identified speakers and the recognized voices. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 It is a diagram showing an example of the functional configuration of a speech recognition device according to an embodiment.

[0007] Figure 2 This is a diagram showing an example of the functional configuration of a discriminating unit according to an embodiment.

[0008] Figure 3AThis is a flowchart showing an example of the overall flow of speaker registration according to the embodiment.

[0009] Figure 3B This is a flowchart showing an example of the detailed flow of step S2 in the embodiment.

[0010] Figure 4 This is a flowchart showing an example of a speech recognition process according to an embodiment.

[0011] Figure 5 This is a diagram showing an example of the functional configuration of a discrimination unit according to Modification 1 of the embodiment.

[0012] Figure 6 It is a diagram showing an example of keyword information according to Modification 1 of the embodiment.

[0013] Figure 7 This is a diagram showing an example of the functional configuration of a speech recognition device according to a second variation of the embodiment.

[0014] Figure 8 This is a diagram showing an example of the hardware configuration of the speech recognition device according to the embodiment.

[0015] Description of Reference Numerals

[0016] 1 Microphone

[0017] 2 Speech Recognition Unit

[0018] 3. Identification

[0019] 4. Judgment Department

[0020] 5 Display control unit

[0021] 6 Display

[0022] 7 Equipment Control Department

[0023] 21 First Analysis Department

[0024] 22. Inspection Department

[0025] 23 Free speech recognition unit

[0026] 24 Language Comprehension Department

[0027] 31 Second Analysis Department

[0028] 32 Interception

[0029] 33 Embedding vector calculation unit

[0030] 34 Registration Department

[0031] 35 Similarity calculation unit

[0032] 100 Voice recognition device

[0033] 101 Speech recognition model storage unit

[0034] 102 Embedded Model Storage Unit

[0035] 103 Personal setting storage unit

[0036] 104 Embedded vector storage unit

[0037] 105 Keyword information storage unit

[0038] 111 Air conditioning unit

[0039] 112 Sound installation

[0040] 201 Processor

[0041] 202 Main storage device

[0042] 203 Auxiliary storage device

[0043] 204 Display device

[0044] 205 Input Device

[0045] 206 Communication devices

[0046] 210 Bus DETAILED DESCRIPTION

[0047] Hereinafter, embodiments of a speech recognition device, a speech recognition method, and a program will be described in detail with reference to the accompanying drawings.

[0048] The speech recognition device 100 of the embodiment calculates the speaker embedding vector for the speech in the interval where the keyword is detected by keyword spotting, using the speaker embedding model learned independently from the speech recognition model. The speech recognition device 100 of the embodiment performs speaker identification based on the similarity with each speaker registered in advance. In addition, the speech recognition device 100 of the embodiment controls equipment such as an audio device and an air conditioning device based on the detected keywords and the results of speaker identification. As a result, speech recognition targeting multiple speakers can be performed, and inherent control corresponding to the speakers can be performed.

[0049] [Example of functional configuration]

[0050] Figure 11 is a diagram showing an example of the functional configuration of the speech recognition device 100 according to the embodiment. The speech recognition device 100 according to the embodiment includes a microphone 1, a speech recognition unit 2, a recognition unit 3, a determination unit 4, a display control unit 5, a display 6, and a device control unit 7. The speech recognition unit 2 includes a first analysis unit 21 and a detection unit 22. In addition, the speech recognition device 100 according to the embodiment includes a speech recognition model storage unit 101, an embedded model storage unit 102, and a personal setting storage unit 103.

[0051] The microphone 1 acquires the voices of one or more speakers, and inputs the acoustic signals acquired at each time into the first analysis unit 11 and the identification unit 3 .

[0052] The first analysis unit 21 extracts a feature quantity indicating a feature of a waveform of an acoustic signal from an acoustic signal input at each time from the microphone 1. The extracted feature quantity is, for example, MFCC (Mel-Frequency Cepstrum Coefficients) and a Mel filter bank feature quantity.

[0053] The detection unit 22 detects a keyword from the sequence of feature quantities, and inputs the keyword detection result, the start time of the keyword, and keyword detection information including the end time of the keyword to the identification unit 3 .

[0054] Figure 2 3 is a diagram showing an example of the functional configuration of the identification unit 3 according to the embodiment. The identification unit 3 according to the embodiment includes a second analysis unit 31 , a cutting unit 32 , an embedding vector calculation unit 33 , a registration unit 34 , a similarity calculation unit 35 , and an embedding vector storage unit 104 .

[0055] The second analysis unit 31 has the same feature extraction function as the first analysis unit 21. The feature extracted by the second analysis unit 31 may be the same as the feature extracted by the first analysis unit 21. In this case, the identification unit 3 may receive the feature from the first analysis unit 21.

[0056] The extraction unit 32 receives the feature amount from the second analysis unit 31 and, when receiving the keyword information from the detection unit 22 , extracts a sequence of feature amounts included between the start time of the keyword and the end time of the keyword (keyword detection interval).

[0057] The embedding vector calculation unit 33 reads the speaker embedding model from the embedding model storage unit 102, and calculates the speaker embedding vector based on the speaker embedding model. As a method for calculating the speaker embedding vector, for example, i-vector, d-vector (non-patent document 1), x-vector (non-patent document 2), and their derivative methods can be used.

[0058] The registration unit 34 averages a predetermined number of speaker embedding vectors calculated in advance for the speaker of the speech recognition target (registration target), and stores the averaged speaker embedding vector in the embedding vector storage unit 104 .

[0059] Similarity calculation unit 35 calculates the similarity between the speaker embedding vector calculated by embedding vector calculation unit 33 and the speaker embedding vector of the registered speaker stored in embedding vector storage unit 104. As a similarity calculation method, for example, cosine similarity and PLDA (Non-Patent Document 3) are used.

[0060] return Figure 1 The determination unit 4 determines the speaker of the utterance included in the keyword detection section based on the similarity calculated by the similarity calculation unit 35, and inputs the determination result (identification result) of the speaker to the display control unit 5 and the device control unit 7.

[0061] The display control unit 5 displays information identifying the speaker of the voice recognition on the display 6 based on the determination result input from the determination unit 4 .

[0062] The device control unit 7 reads the personal setting of the identified speaker from the personal setting storage unit 103 based on the determination result input from the determination unit 4 , and controls devices such as the air conditioner 111 and the audio device 112 based on the personal setting.

[0063] For example, the device control unit 7 implements device control based on a combination of the speaker identification result and the voice recognition result. Specifically, for example, when the audio device 112 is a car audio, the device control unit 7 plays the favorite songs corresponding to the speaker determined by the personal setting when the speaker utters "play favorite songs". In addition, for example, for the air conditioner 111, when the speaker utters "favorite settings", the device control unit 7 sets the favorite temperature / air volume corresponding to the speaker determined by the personal setting.

[0064] Figure 3A 1 is a flowchart showing an example of the overall flow of speaker registration according to the embodiment. First, the registration unit 34 initializes a variable k for identifying K speakers to be registered to 1 (step S1).

[0065] Next, the registration unit 34 performs speaker registration of speaker k (step S2). Figure 3B To be described later.

[0066] Next, the registration unit 34 determines whether k = K (step S3). If k = K (step S3, YES), the speaker registration process ends.

[0067] When k=K is not true (step S3 , No), the registration unit 34 increments the value of k (step S4 ), and the speaker registration process returns to step S2 .

[0068] Figure 3B : is a flowchart showing an example of the detailed flow of step S2 of the embodiment. First, the registration unit 34 initializes the vector V and the variable n used for calculating the average of the speaker embedding vectors to 0 (step S11).

[0069] Next, the speech recognition unit 2 recognizes the speech of the speaker k (step S12 ). Specifically, the first analysis unit 21 extracts feature quantities from the acoustic signal of the speaker k input from the microphone 1 at each time, and the detection unit 22 attempts to detect a keyword from the sequence of the feature quantities.

[0070] When no keyword is detected by the speech recognition in step S12 (step S13, No), the speaker registration process returns to step S12, and the speech recognition of speaker k is continued.

[0071] When a keyword is detected by the speech recognition in step S12 (step S13 , Yes), the cutting unit 32 cuts out the feature amount included in the keyword detection section (step S14 ).

[0072] Next, the embedding vector calculation unit 33 calculates the speaker embedding vector v based on the feature quantity extracted in step S14 (step S15).

[0073] Next, the registration unit 34 adds the speaker embedding vector v calculated in step S15 to the vector V, and increments the variable n (step S16). Next, the registration unit 34 determines whether n=N (step S17). If n=N is not (step S17, No), the speaker registration process returns to step S12, and the speech recognition of speaker k is continued.

[0074] When n=N (step S17, yes), the registration unit 34 calculates the average of the speaker embedding vectors v of speaker k calculated in step S15 by dividing the vector V by N, and registers the vector V / N as the speaker embedding vector of speaker k in the embedding vector storage unit 104 (step S18).

[0075] like Figure 3B As shown in FIG. 1 , the registration unit 34 calculates the speaker embedding vector (first speaker embedding vector) for each of the N utterances (N ≥ 1) of the same speaker. Furthermore, the registration unit 34 calculates the statistics of each speaker embedding vector (in Figure 3BIn the example of , the average) is registered as the speaker embedding vector of the same speaker in the embedding vector storage unit 104. By inputting a plurality of utterances when registering a speaker, it is possible to suppress the deviation of the speaker embedding vector of each utterance within the speaker.

[0076] Figure 4 1 is a flowchart showing an example of a speech recognition process of an embodiment. First, the speech recognition unit 2 recognizes the speech input from the microphone 1 (step S21). Specifically, the first analysis unit 21 extracts a feature value from the sound signal input from the microphone 1 at each time, and the detection unit 22 attempts to detect a keyword from the sequence of the feature values.

[0077] When no keyword is detected by the speech recognition in step S21 (step S22 , No), the speaker registration process returns to step S21 , and the speech recognition of speaker k is continued.

[0078] When a keyword is detected by the speech recognition in step S21 (step S22 , Yes), the cutting unit 32 cuts out the feature amount included in the keyword detection section (step S23 ).

[0079] Next, the embedding vector calculation unit 33 calculates the speaker embedding vector v based on the feature amount intercepted in step S23 (step S24). Next, the similarity calculation unit 35 calculates the similarity between the speaker embedding vector v calculated in step S24 and the speaker embedding vector of the registered speaker stored in advance in the embedding vector storage unit 104 by the registration unit 34 (step S25).

[0080] Next, the determination unit 4 determines the speaker of the utterance included in the keyword detection section based on the similarity calculated in step S25, and the device control unit 7 controls the device according to the personal setting of the identified speaker based on the speaker determination result (identification result) (step S26).

[0081] Next, the display control unit 5 displays information identifying the speaker identified in the voice recognition in step S21 on the display 6 (step S27 ).

[0082] As described above, in the speech recognition device 100 of the embodiment, the embedding model storage unit 102 and the personal setting storage unit 103 (an example of a storage unit) store the first speaker embedding vector of one or more given registered speakers and the personal settings of the registered speakers used for controlling the device. The speech recognition unit 2 recognizes speech from the sound signal and obtains the speech recognition result. The second analysis unit 31 analyzes the sound signal and extracts the feature quantity representing the feature of the waveform of the sound signal. The clipping unit 32 clips the sequence of feature quantities contained in the utterance interval from the speech recognition result. The embedding vector calculation unit 33 calculates the second speaker embedding vector using the sequence of feature quantities. The similarity calculation unit 35 calculates one or more similarities between the second speaker embedding vector and one or more first speaker embedding vectors. The determination unit 4 determines which speaker of the one or more registered speakers uttered the speech based on one or more similarities. Then, the device control unit 7 controls the device based on the registered speakers determined based on one or more similarities and the speech recognition result by the personal settings read from the personal setting storage unit 103.

[0083] According to the configuration of the speech recognition device 100 of the embodiment, speech recognition is performed for a specific plurality of speakers using a speaker embedding model (a first speaker embedding vector of one or more given registered speakers) that is independent of the speech recognition model used for speech recognition. As a result, the speech of the specific plurality of speakers can be recognized, and the device can be controlled based on the determined speakers and the recognized speech.

[0084] In addition, in the past, since only the voice of a given speaker is recognized, there is a problem that the target speaker cannot be set to multiple people when the voice of speaker A and the voice of speaker B are to be recognized. In addition, since the speech recognition model depends on the speaker embedding model, in order to cope with environmental changes and changes over time, the models of both sides need to be relearned, and there is a problem that the adjustment of the model corresponding to the environment becomes complicated.

[0085] (Variation 1 of the embodiment)

[0086] Next, Modification 1 of the embodiment will be described. In the description of Modification 1, the same description as that of the embodiment will be omitted, and only the differences from the embodiment will be described.

[0087] Figure 5 FIG. 2 is a diagram showing an example of the functional configuration of the identification unit 3 according to the first modification of the embodiment. Figure 5 In the example of , the similarity calculation unit 35 further reads keyword information from the keyword information storage unit 105. The keyword information storage unit 105 stores combinations of registered speakers and keywords.

[0088] Figure 6 This is a diagram showing an example of keyword information of Modification 1 of the embodiment. The keyword information of Modification 1 shows a list of keywords that can be accepted for each speaker A and B. For example, keyword b indicates that it can be accepted for speaker A but cannot be accepted for speaker B. The keyword information is used as a filter to determine the keywords that can be accepted for each speaker.

[0089] The similarity calculation unit 35 may receive the speech recognition result of the keyword detection section from the cutting unit 32 via the embedding vector calculation unit 33 and calculate the similarity only with the speaker who can input the keyword recognized by the speech. This can reduce the similarity calculation cost.

[0090] That is, the similarity calculation unit 35 calculates the similarity of the speech recognition result. Figure 6 The similarity between the first speaker embedding vector of the registered speaker of the acceptable keyword determined by the keyword information of the speaker and the second speaker embedding vector calculated by the embedding vector calculation unit 33. That is, the similarity calculation unit 35 does not calculate the similarity between the first speaker embedding vector of the registered speaker of the acceptable keyword determined by the keyword information of the speaker and the second speaker embedding vector calculated by the embedding vector calculation unit 33. Figure 6 The similarity between the first speaker embedding vector of the registered speaker of the acceptable keyword determined by the keyword information and the second speaker embedding vector calculated by the embedding vector calculation unit 33.

[0091] In addition, the embedding vector calculation unit 33 may further receive the keyword recognition result from the speech recognition unit 2 via the interception unit 32, and calculate the speaker embedding vector based on the acoustic feature and the keyword recognition result. The keyword recognition result may be a keyword ID or a string corresponding to the utterance, or an acoustic score at each moment, etc. The acoustic score indicates the probability that the speech at each moment corresponds to each phoneme.

[0092] That is, the speech recognition result may also include an acoustic score indicating the probability that the speech at each time point corresponds to each phoneme, and the embedding vector calculation unit 33 calculates the speaker embedding vector (second speaker embedding vector) based on the acoustic score at each time point and the feature quantity at each time point included in the sequence of feature quantities. Thus, when the utterance content is different during registration and during discrimination, it is expected that the discrimination performance will be improved.

[0093] In addition, the determination unit 4 may further determine which registered speaker the utterance is not based on the threshold value (first threshold value) of the similarity. In this case, the display control unit 5 displays information (such as name, etc.) for identifying the registered speaker determined based on one or more similarities on the display 6, and when one or more similarities are all below the first threshold value, the display 6 displays information indicating that the reliability of the speaker identification accuracy is below the first threshold value.

[0094] In addition, the threshold value may be a fixed value, or the determination unit 4 may further receive a keyword detection result and use a different threshold value according to the keyword detection result.

[0095] In this way, utterances other than those of the predetermined registered speaker can be discarded. In addition, when the detection unit 22 erroneously reacts to non-speech such as environmental noise and outputs a detection result, such a detection result can be discarded.

[0096] In addition, when the similarity between the input utterance and the registered speaker embedding vector is below a threshold, it is possible to prompt the user to utter the utterance again or to discard errors in the speech recognition result caused by background noise.

[0097] (Variation 2 of the embodiment)

[0098] Next, Modification 2 of the embodiment will be described. In the description of Modification 2, the same description as that of the embodiment will be omitted, and the differences from the embodiment will be described.

[0099] Figure 7 1 is a diagram showing an example of the functional configuration of a speech recognition device 100 - 2 according to Modification 2 of the embodiment. The speech recognition unit 2 according to Modification 2 includes a free utterance recognition unit 23 and a language understanding unit 24 instead of the detection unit 22 of the embodiment.

[0100] The free utterance recognition unit 23 recognizes the speech of free utterance that does not depend on a predetermined keyword, and converts the speech recognition result into a character string.

[0101] The language understanding unit 24 analyzes the character string obtained by the free utterance recognition unit 23. For example, the language understanding unit 24 acquires a language understanding result based on a language understanding model from the character string.

[0102] The similarity calculation unit 35 selects one or more first speaker embedding vectors as the objects of similarity calculation based on the language understanding results, and calculates the similarity between the one or more selected first speaker embedding vectors and the second speaker embedding vector calculated by the embedding vector calculation unit 33.

[0103] according to Figure 7 The configuration of the second variation shown can also cope with a more advanced voice input interface that is not limited to specific keywords. By applying speaker identification to large-vocabulary voice recognition, the operation of the voice interface based on speaker information can be controlled for a wider range of tasks.

[0104] For example, when recognizing the speech of two people in the driver's seat and the front passenger seat of a car, the speech recognition device 100-2 of Modified Example 2 can determine that the speech is related to driving assistance of the car based on the language understanding result. In this case, the similarity calculation unit 35 can select the first speaker embedding vector of the registered speaker registered as the driver of the car as the object of similarity calculation.

[0105] Furthermore, when registering a speaker, instead of using a fixed number of utterances, the registration may be terminated when the variance of the speaker embedding vector of each utterance becomes less than a threshold value. That is, the registering unit 34 may prompt the same speaker to utter repeatedly, calculate each speaker embedding vector (first speaker embedding vector) for each utterance, and prompt the speaker to stop uttering when the variance of each speaker embedding vector becomes less than a second threshold value.

[0106] This allows speaker registration to be performed with a minimum number of utterances, thereby reducing the effort required for speaker registration and improving user experience while maintaining speaker identification accuracy.

[0107] In addition, when making a judgment, the values ​​of the embedding vector storage unit may be updated sequentially using utterances with a similarity above a threshold. Since the sound quality changes over the years, if the speaker embedding vector initially registered is continuously used, the speaker identification accuracy decreases, and re-registration may be required. Therefore, the registration unit 34 may also use the speaker embedding vector (the second speaker embedding vector calculated by the embedding vector calculation unit 33) of a speaker with a similarity above a threshold (third threshold) to update the embedding vector (the first speaker embedding vector registered in the embedding vector storage unit 104) of a registered speaker with a similarity above a threshold (third threshold).

[0108] By sequentially updating the registration unit 34 , it is not necessary to perform explicit re-registration work at regular intervals (for example, re-registration work every predetermined number of years), and the speaker identification accuracy can be maintained even with the passage of time.

[0109] Finally, an example of the hardware configuration of the speech recognition device 100 according to the embodiment will be described.

[0110] [Example of hardware configuration]

[0111] Figure 7 2 is a diagram showing an example of the hardware configuration of the speech recognition device 100 according to the embodiment. The speech recognition device 100 according to the embodiment includes a processor 201, a main storage device 202, an auxiliary storage device 203, a display device 204, an input device 205, and a communication device 206. The processor 201, the main storage device 202, the auxiliary storage device 203, the display device 204, the input device 205, and the communication device 206 are connected via a bus 210.

[0112] Furthermore, the speech recognition device 100 may not include a part of the above-mentioned configurations. For example, when the speech recognition device 100 can use the input function and display function of an external device, the speech recognition device 100 may not include the display device 204 and the input device 205 .

[0113] The processor 201 executes the program read from the auxiliary storage device 203 to the main storage device 202. The main storage device 202 is a memory such as a ROM and a RAM. The auxiliary storage device 203 is a HDD (Hard Disk Drive) or a memory card.

[0114] The display device 204 is, for example, a liquid crystal display ( Figure 1 The input device 205 is an interface for operating the speech recognition device 100. In addition, the display device 204 and the input device 205 can also be implemented by a touch panel having a display function and an input function. The communication device 206 is an interface for communicating with other devices.

[0115] For example, the program executed by the speech recognition device 100 is recorded in a computer-readable storage medium such as a memory card, hard disk, CD-RW, CD-ROM, CD-R, DVD-RAM, and DVD-R based on a file in an installable form or an executable form, and is provided as a computer program product.

[0116] In addition, for example, a configuration may be adopted in which the program executed by the speech recognition device 100 is stored in a computer connected to a network such as the Internet and provided by downloading via the network.

[0117] In addition, for example, the program executed by the speech recognition device 100 may be provided via a network such as the Internet without downloading the program. Specifically, the speech recognition process may be performed by a so-called ASP (Application Service Provider) type service that implements the processing function only through its execution instructions and result acquisition, without transferring the program from the server computer.

[0118] In addition, for example, the program of the speech recognition device 100 may be provided by being pre-installed in a ROM or the like.

[0119] The program executed by the speech recognition device 100 is a module structure including the functions that can be realized by the program in the above-mentioned functional structure. As actual hardware, the functions are read out from the storage medium by the processor 201 and executed, and the above-mentioned functional blocks are loaded into the main storage device 202. That is, the above-mentioned functional blocks are generated on the main storage device 202.

[0120] Furthermore, part or all of the above-mentioned functions may be realized not by software but by hardware such as IC (Integrated Circuit).

[0121] Furthermore, each function may be implemented using a plurality of processors 201 . In this case, each processor 201 may implement one of the functions or two or more of the functions.

[0122] Several embodiments of the present invention have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the subject matter of the invention. These embodiments and their variations are included in the scope and subject matter of the invention, and are included in the invention described in the claims and the scope of their equivalents.

Claims

1. A speech recognition device, comprising: A storage unit storing first speaker embedding vectors of given one or more registered speakers and personal settings of the registered speakers used for control of the device; A speech recognition unit, which recognizes speech from an audio signal and obtains a speech recognition result; an analyzing unit that analyzes the acoustic signal and extracts a feature quantity representing a feature of a waveform of the acoustic signal; A cutting unit, cutting out the sequence of the feature quantities contained in the utterance interval from the speech recognition result; An embedding vector calculation unit, which uses the sequence of feature quantities to calculate a second speaker embedding vector; A similarity calculation unit, which calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors; a determination unit that determines which speaker of the one or more registered speakers uttered the utterance based on the one or more similarities; as well as The device control unit controls the device using the personal setting read from the storage unit based on the registered speakers determined based on the one or more similarities and the speech recognition result.

2. The speech recognition device according to claim 1, further comprising: The display control unit displays information identifying the registered speaker determined based on the one or more similarities on a display device, and when the one or more similarities are all below a first threshold, displays information indicating that the reliability of the speaker identification accuracy is below the first threshold on the display device.

3. The speech recognition device according to claim 1 or 2, The storage unit further stores a combination of the registered speaker and the keyword. The similarity calculation unit calculates the similarity with the first speaker embedding vector of the registered speaker including the keyword in the speech recognition result, and does not calculate the similarity with the first speaker embedding vector of the registered speaker not including the keyword in the speech recognition result.

4. The speech recognition device according to claim 1 or 2, The speech recognition unit converts the speech recognition result into a character string, and further obtains a language understanding result understood based on a language understanding model from the character string. The similarity calculation unit selects one or more first speaker embedding vectors as objects for similarity calculation based on the language understanding result, and calculates similarities with the one or more selected first speaker embedding vectors.

5. The speech recognition device according to claim 1 or 2, further comprising: a registering unit, registering the one or more first speaker embedding vectors in the storage unit, The registration unit calculates each first speaker embedding vector for N utterances (N≥1) of the same speaker, and registers the statistics of each first speaker embedding vector as the first speaker embedding vector of the same speaker in the storage unit.

6. The speech recognition device according to claim 5, The registration unit prompts the same speaker to utter repeatedly, calculates each first speaker embedding vector for each utterance, and prompts the speaker to stop uttering when a variance of each first speaker embedding vector is equal to or smaller than a second threshold.

7. The speech recognition device according to claim 5, The registration unit updates the first speaker embedding vector having a similarity greater than or equal to the third threshold value by using the second speaker embedding vector having a similarity greater than or equal to the third threshold value.

8. The speech recognition device according to claim 1 or 2, The speech recognition result includes an acoustic score indicating the probability that the speech at each moment corresponds to each phoneme. The embedding vector calculation unit calculates the second speaker embedding vector based on the acoustic score at each time point and the feature quantity at each time point included in the sequence of feature quantities.

9. A speech recognition method, comprising: The speech recognition device stores a first speaker embedding vector of one or more given registered speakers and the personal settings of the registered speakers used for controlling the device in a storage unit; The speech recognition device recognizes speech from the sound signal and obtains a speech recognition result; The speech recognition device analyzes the sound signal to extract a feature quantity representing a feature of a waveform of the sound signal; The speech recognition device extracts the sequence of the feature quantities contained in the utterance interval from the speech recognition result; The speech recognition device uses the sequence of feature quantities to calculate a second speaker embedding vector; The speech recognition device calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors; The speech recognition device determines which speaker of the one or more registered speakers uttered the speech based on the one or more similarities; as well as The speech recognition apparatus controls the device by the personal setting read out from the storage unit based on the registered speakers determined based on the one or more similarities and the speech recognition result.

10. A storage medium storing a program causing a computer to function as the following units, the computer having a storage unit storing first speaker embedding vectors of one or more given registered speakers and personal settings of the registered speakers used for controlling a device, the units comprising: A speech recognition unit, which recognizes speech from an audio signal and obtains a speech recognition result; an analyzing unit that analyzes the acoustic signal and extracts a feature quantity representing a feature of a waveform of the acoustic signal; A cutting unit, cutting out the sequence of the feature quantities contained in the utterance interval from the speech recognition result; A speaker embedding vector calculation unit, which uses the sequence of feature quantities to calculate a second speaker embedding vector; A similarity calculation unit, which calculates one or more similarities between the second speaker embedding vector and the one or more first speaker embedding vectors; a determination unit that determines which speaker of the one or more registered speakers uttered the utterance based on the one or more similarities; as well as The device control unit controls the device using the personal setting read from the storage unit based on the registered speakers determined based on the one or more similarities and the speech recognition result.

Images