Information processing method and information processing device

Abstract

To suppress a computational load for executing a speaker recognition process.SOLUTION: An information processing method acquires condition information and speaker features of speech utterances (S20, S21), acquires a plurality of registered speaker features based on the condition information (S221), and based on similarity between each of the acquired plurality of registered speaker features and the speaker feature, identifies which of registered speakers who respectively correspond to the plurality of registered speaker features the speaker of the utterance speech is (S231).SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to an information processing method and the like. [Background technology]

[0002] For example, there are teleconference systems such as telephone conference systems that allow two or more people to communicate using voice via telephone lines or a network.

[0003] In a remote conference system, when three or more people communicate using voice, it may be desirable to display who is speaking, that is, who is the speaker.

[0004] For example, Patent Document 1 proposes a speaker recognition device that uses speaker recognition processing technology to recognize speakers with high accuracy even in a noisy environment. Here, the speaker recognition technology is a technology that collects speech sounds of speakers to be registered in advance, and determines whether the speaker of the acquired speech sound is one of the registered speakers based on the similarity between a feature amount calculated from the collected speech sound and a newly acquired speech sound that is the target of speaker recognition. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-60757 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, when the speaker recognition process disclosed in the above-mentioned Patent Document 1 is applied to the remote conference system, it is applied to a receiving device that receives the speaker's speech, not to a transmitting device that transmits the speaker's speech. Therefore, the receiving device not only receives the speech, but also performs speaker recognition processing, which causes a problem that the calculation load of the receiving device is high.

[0007] The present disclosure has been made in consideration of the above-mentioned circumstances, and aims to provide an information processing method and the like that can reduce the computational load required to execute speaker recognition processing. [Means for solving the problem]

[0008] An information processing method according to one aspect of the present disclosure acquires condition information and speaker features of a speech voice, acquires a plurality of registered speaker features based on the condition information, and identifies which of a plurality of registered speakers corresponding to each of the plurality of registered speaker features the speaker of the speech voice is based on the similarity between each of the acquired plurality of registered speaker features and the speaker features.

[0009] Furthermore, these general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized by any combination of the system, the method, the integrated circuit, the computer program, and the recording medium. Effect of the Invention

[0010] According to the information processing method etc. of the present disclosure, it is possible to reduce the computational load for executing speaker recognition processing. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a communication system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of the sending communication terminal and the receiving communication terminal shown in FIG. [Diagram 3] FIG. 3 is a block diagram showing an example of a specific configuration of the information transmitting device and the information receiving device shown in FIG. [Figure 4] FIG. 4 is a diagram illustrating an example of a detailed configuration of an analysis unit according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of feature distribution when the genders of speakers of uttered voices are different. [Figure 6] FIG. 6 is a diagram illustrating an example of a configuration of a DNN according to an embodiment. [Figure 7] FIG. 7 is a flowchart showing the operation of the information transmitting device according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing details of the operation of step S11. [Figure 9] FIG. 9 is a flowchart showing the operation of the information receiving device according to the embodiment. [Figure 10] FIG. 10 is a flowchart showing details of the operation of the information receiving device according to the embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of a specific configuration of an information transmitting device and an information receiving device according to the first modification. [Figure 12] FIG. 12 is a diagram illustrating an example of a detailed configuration of an analysis unit according to the first modification. [Figure 13] FIG. 13 is a diagram illustrating an example of a detailed configuration of a speaker feature calculation unit according to the first modification. [Figure 14] FIG. 14 is a diagram illustrating an example of a detailed configuration of a speaker feature acquisition unit according to the first modification. [Figure 15] FIG. 15 is a diagram illustrating the configuration of a DNN according to the first modification example and the concept of components used in calculating the first speaker feature and the second speaker feature. [Figure 16A] FIG. 16A is a flowchart showing details of the operation of the information transmitting device according to the first modification. [Figure 16B] FIG. 16B is a flowchart showing details of the operation of the information transmitting device according to the first modification. [Figure 17] FIG. 17 is a flowchart showing details of the operation of the information receiving device according to the first modification. [Figure 18] FIG. 18 is a block diagram showing an example of a specific configuration of an information transmitting device and an information receiving device according to the second modification. [Figure 19]FIG. 19 is a diagram illustrating an example of a detailed configuration of an analysis unit according to the second modification. [Figure 20] FIG. 20 is a diagram illustrating an example of a detailed configuration of a speaker feature calculation unit according to the second modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (How one aspect of the present disclosure was achieved) In a remote conference system, when three or more people are communicating using only voice, it may be desirable to display who is speaking, that is, who is the speaker.

[0013] However, for example, when applying the speaker recognition process disclosed in Patent Document 1, the process is applied to a receiving device that receives the speech of the speaker to be recognized, not to a transmitting device that transmits the speech of the speaker to be recognized. Therefore, the receiving device not only receives the speech, but also performs the speaker recognition process.

[0014] Meanwhile, speaker recognition processing includes a processing for calculating speaker features from acoustic features obtained from the speech of a speaker to be recognized, and a processing for comparing with pre-stored registered speaker features to analyze a registered speaker that is closest to the calculated speaker features. The processing for calculating speaker features requires a huge amount of calculations, and therefore has a high calculation load. Furthermore, in the processing for analyzing a registered speaker that is closest to the calculated speaker features, an erroneous analysis will occur unless the registered speaker features that meet the conditions when the speech was recorded are compared with the calculated speaker features of the speech. Furthermore, comparing the calculated speaker features with all pre-stored registered speaker features results in a high calculation load.

[0015] Therefore, the receiving device has a problem that the calculation load is high. Therefore, the inventors came up with the idea of ​​distributing the calculation load of the speaker recognition process by having the transmitting device that transmits the speech voice take charge of part of the calculation of the speaker recognition process.

[0016] An information transmission device according to one embodiment of the present disclosure includes an acoustic feature calculation unit that calculates acoustic features for a speech voice, a speaker feature calculation unit that uses a DNN (Deep Neural Network) to calculate speaker features, which are features specific to a speaker of the speech voice, from the acoustic features, an analysis unit that analyzes condition information indicating conditions for calculating the speaker features based on the speech voice, and an information transmission unit that transmits the speaker features and the condition information to the information receiving device, which performs speaker recognition processing of the speech voice, as information used by the information receiving device to recognize the speaker of the speech voice.

[0017] According to this aspect, by transmitting information used to recognize the speaker of an utterance to an information receiving device that performs speaker recognition processing of an utterance, it is possible to cause the information receiving device to omit a part of the calculations required for performing the speaker recognition processing. Specifically, the information receiving device can not only omit the calculation of acoustic features for the speaker's utterance, but also omit at least a part of the processing for calculating speaker features from the acoustic features, thereby reducing the calculation load required for performing the speaker recognition processing. Thus, according to this aspect, it is possible to suppress the calculation load required for performing the speaker recognition processing.

[0018] Also, for example, the analysis unit may include a situation analysis unit that analyzes a recording situation of the uttered voice from the uttered voice, and a condition information setting unit that sets the recording situation as the condition information.

[0019] According to this aspect, by transmitting the recording status of the analyzed speech voice to the information receiving device as condition information, it is possible to cause the information receiving device to select registered speaker features according to the recording status and then execute the similarity calculation process. As a result, the information receiving device does not need to execute the similarity calculation process for registered speaker features under all conditions, and therefore the calculation load can be reduced. In addition, the information transmitting device transmits information used to recognize the speaker of the speech voice, including speaker features for the speech voice of the speaker to be recognized. As a result, the information receiving device can omit the calculation of speaker features for the speech voice of the speaker to be recognized. Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be further reduced.

[0020] Furthermore, for example, the recording status may indicate at least one of the noise level and the microphone used when recording the speech, and data attributes of the speech.

[0021] Also, for example, the situation analysis unit may further analyze a speech time of the speech voice, and the condition information setting unit may further set, as the condition information, a load control condition indicating that the DNN used by the speaker feature calculation unit is a first DNN that is a part of the DNN consisting of a first layer to an n-th layer (n is a positive integer) of the DNN based on the speech time, and the speaker feature calculation unit may calculate, as the speaker feature, a first speaker feature in the middle of calculating the speaker feature from the acoustic feature using the first DNN in accordance with the load control condition.

[0022] According to this aspect, information used to recognize the speaker of a speech voice is transmitted to an information receiving device that performs a speaker recognition process, including a first speaker feature that corresponds to a part of the speaker feature for the speech voice of the speaker to be recognized. This allows the information receiving device to omit a part of the calculation for executing the speaker recognition process. Specifically, by receiving the information, the information receiving device can not only omit the calculation of the acoustic feature for the speech voice of the speaker, but also omit a part of the calculation of the speaker feature using the first speaker feature. Furthermore, when the information receiving device receives the recording situation included in the information, it can select the registered speaker feature according to the recording situation and then execute the similarity calculation process, thereby reducing the calculation load.

[0023] Therefore, according to this aspect, it is possible to reduce the computational load required for executing the speaker recognition process.

[0024] Also, for example, the analysis unit may include a situation analysis unit that analyzes an utterance time of the speech voice, and a condition information setting unit that sets, as the condition information, a load control condition indicating that the DNN used by the speaker feature calculation unit is a first DNN that is a part of the DNN composed of a first layer to an nth layer (n is a positive integer) of the DNN based on the utterance time, and the speaker feature calculation unit may calculate, as the speaker feature, a first speaker feature in the middle of calculating the speaker feature from the acoustic feature using the first DNN in accordance with the load control condition.

[0025] According to this aspect, information used to recognize the speaker of a speech voice is transmitted to an information receiving device that performs a speaker recognition process, including a first speaker feature that corresponds to a part of the speaker feature for the speech voice of a speaker to be recognized. This allows the information receiving device to omit at least a part of the calculation for executing the speaker recognition process. Specifically, by receiving the information, the information receiving device can not only omit the calculation of the acoustic feature for the speech voice of the speaker, but also omit a part of the calculation of the speaker feature by using the first speaker feature.

[0026] Therefore, according to this aspect, it is possible to reduce the computational load required for executing the speaker recognition process.

[0027] Moreover, an information receiving device according to one embodiment of the present disclosure includes a memory unit in which registered speaker features, which are unique features for each of one or more registered speakers registered in advance, are stored for each condition when calculating the registered speaker features; a condition information acquisition unit that acquires the condition information included in information transmitted from an information transmitting device described in any of the above embodiments; a speaker feature acquisition unit that acquires the speaker features included in the information transmitted from the information transmitting device; a similarity calculation unit that calculates a similarity between each of the registered speaker features stored in the memory unit and the speaker features based on the acquired condition information and speaker features; and a speaker identification unit that identifies which of the one or more registered speakers stored in the memory unit the speaker of the spoken voice is, based on the similarity calculated by the similarity calculation unit, and outputs the identified speaker feature.

[0028] According to this aspect, the information receiving device that performs speaker recognition processing of an utterance voice can omit at least a part of the calculations for performing the speaker recognition processing by receiving information used for recognizing the speaker of the utterance voice. Specifically, the information receiving device can omit not only the calculation of acoustic features for the utterance voice of the speaker, but also at least a part of the processing for calculating speaker features from the acoustic features, thereby reducing the calculation load for performing the speaker recognition processing.

[0029] Therefore, according to this aspect, it is possible to reduce the computational load required for executing the speaker recognition process.

[0030] Also, for example, when the condition information acquired by the condition information acquisition unit includes a recording status of the speech voice, the similarity calculation unit may include a selection unit that selects registered speaker features for each of the one or more registered speakers that meet the recording status, and a similarity calculation processing unit that calculates a similarity between the registered speaker features for each of the one or more registered speakers selected by the selection unit and the speaker features acquired by the speaker feature acquisition unit, and the speaker identification unit may identify which of the one or more registered speakers selected by the selection unit the speaker of the speech voice is, based on the similarity calculated by the similarity calculation processing unit, and output the result.

[0031] According to this aspect, the information receiving device can acquire the recording status of the speech voice included in the received condition information, select the registered speaker feature according to the recording status, and then execute the similarity calculation process. This allows the information receiving device to reduce the calculation load because it does not need to execute the similarity calculation process for the registered speaker feature under all conditions. Furthermore, by receiving the speaker feature for the speech voice of the speaker to be recognized, the information receiving device can omit the calculation of the speaker feature for the speech voice of the speaker to be recognized. Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be further reduced.

[0032] Also, for example, when the acquired condition information further includes a load control condition indicating that a first DNN, which is a part of the DNN consisting of the first layer to the nth layer (n is a positive integer), has been used instead of the DNN, the condition information acquisition unit may transmit the load control condition to the speaker feature acquisition unit, and the speaker feature acquisition unit may acquire, as the speaker feature included in the information, a first speaker feature that is in the middle of calculating the speaker feature and is calculated using the first DNN, and when the load control condition is transmitted, may calculate a second speaker feature, which is the speaker feature, from the first speaker feature based on the load control condition using a second DNN that is a part of the DNN consisting of the n+1th layer to a final layer.

[0033] According to this aspect, the information receiving device can omit a part of the calculation for executing the speaker recognition process by acquiring the first speaker feature corresponding to a part of the speaker feature for the speech of the speaker to be recognized. Specifically, by receiving the first speaker feature, the information receiving device can not only omit the calculation of the acoustic feature for the speech of the speaker, but also omit a part of the calculation of the speaker feature by using the first speaker feature. Furthermore, when the information receiving device receives the recording situation included in the information, it can select the registered speaker feature according to the recording situation and then execute the similarity calculation process, thereby reducing the calculation load.

[0034] Therefore, according to this aspect, it is possible to reduce the computational load required for executing the speaker recognition process.

[0035] Furthermore, for example, when the acquired condition information indicates a load control condition indicating that a first DNN, which is a part of the DNN consisting of the first layer to the nth layer (n is a positive integer), has been used instead of the DNN, the condition information acquisition unit may transmit the load control condition to the speaker feature acquisition unit, and the speaker feature acquisition unit may acquire, as the speaker feature included in the information, a first speaker feature that is in the middle of calculating the speaker feature and that has been calculated using the first DNN, and when the load control condition is transmitted, may calculate a second speaker feature, which is the speaker feature, from the first speaker feature based on the load control condition using a second DNN that is a part of the DNN consisting of the n+1th layer to a final layer of the DNN.

[0036] According to this aspect, the information receiving device can omit a part of the calculations for executing the speaker recognition process by acquiring the first speaker feature corresponding to a part of the speaker feature for the speech of the speaker to be recognized. Specifically, by receiving the first speaker feature, the information receiving device can not only omit the calculation of the acoustic feature for the speech of the speaker, but also omit a part of the calculation of the speaker feature by using the first speaker feature. Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be suppressed.

[0037] An information transmission method according to one embodiment of the present disclosure is an information transmission method performed by a computer, which calculates acoustic features for a speech voice, calculates speaker features that are features specific to the speaker of the speech voice from the acoustic features using a DNN (Deep Neural Network), analyzes condition information indicating conditions for calculating the speaker features based on the speech voice, and transmits the speaker features and the condition information to an information receiving device that performs speaker recognition processing of the speech voice as information used by the information receiving device to recognize the speaker of the speech voice.

[0038] Moreover, a system according to one aspect of the present disclosure is a system including an information transmitting device and an information receiving device that performs speaker recognition processing, the information transmitting device having an acoustic feature calculation unit that calculates acoustic features for an uttered voice, a speaker feature calculation unit that calculates speaker features capable of identifying a speaker of the uttered voice from the acoustic features using a DNN (Deep Neural Network), an analysis unit that analyzes condition information indicating conditions for calculating the speaker features based on the uttered voice, and an information transmitting unit that transmits the speaker features and the condition information to the information receiving device as information used by the information receiving device to recognize the speaker of the uttered voice, and the information receiving device transmits registered speaker features, which are features unique to each of one or more registered speakers that have been pre-registered, to the information receiving device for each condition for calculating the registered speaker features. the information transmission device includes a memory unit in which information is stored, a condition information acquisition unit for acquiring the condition information included in the information transmitted from the information transmission device, a speaker feature acquisition unit for acquiring the speaker features included in the information transmitted from the information transmission device, a similarity calculation unit for calculating a similarity between each of the registered speaker features stored in the memory unit and the speaker features based on the acquired condition information and the speaker features, and a speaker identification unit for identifying which of the one or more registered speakers stored in the memory unit the speaker of the spoken voice is, based on the similarity calculated by the similarity calculation unit, and outputting the identified speaker feature.

[0039] These comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized by any combination of the system, the method, the integrated circuit, the computer program, and the recording medium.

[0040] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing a top concept are described as optional components. Furthermore, in all of the embodiments, the contents of each of the embodiments can be combined.

[0041] (Embodiment) Hereinafter, an information transmitting device and the like in the embodiments will be described with reference to the drawings.

[0042] [1.1 Communication System 100] FIG. 1 is a block diagram showing an example of a configuration of a communication system 100 according to the present embodiment.

[0043] A communication system 100 according to this embodiment is a remote conference system that allows two or more people to communicate using voice, and is used, for example, as a telephone conference system.

[0044] 1, the communication system 100 includes a sending communication terminal 1 and a receiving communication terminal 2. These are connected via a network 3 such as a telephone line, a LAN (Local Area Network) or the Internet.

[0045] [1.2 Configuration of the sending communication terminal 1 and the receiving communication terminal 2] FIG. 2 is a block diagram showing an example of the configuration of the sending communication terminal 1 and the receiving communication terminal 2 shown in FIG.

[0046] The calling communication terminal 1 is a terminal used by the calling party, i.e. the speaker, and when the speaker speaks, it transmits the voice signal of the speech and information obtained from the spoken voice to the receiving communication terminal 2 via the network 3.

[0047] 2, the sending communication terminal 1 includes an information transmitting device 10 for transmitting the information, and a voice transmitting device 11 for transmitting a voice signal of an utterance. Here, the voice transmitting device 11 includes a microphone 111 for receiving an utterance of a speaker and outputting a voice signal of the uttered voice, and a voice signal transmitting unit 112 for transmitting the voice signal received from the microphone 111 to the receiving communication terminal 2 via the network 3. The information transmitting device 10 will be described later.

[0048] The receiving communication terminal 2 is a terminal used by a person on the receiving side, i.e., the transmitting side, who listens to the speech of a speaker, and receives the audio signal of the speaker's speech transmitted by the transmitting communication terminal 1 via the network 3 and information obtained from the speech voice.

[0049] 2, the receiving communication terminal 2 includes an information receiving device 20 that receives the information, and a voice receiving device 21 that receives a voice signal of an utterance by a speaker on the transmitting side. Here, the voice receiving device 21 includes a voice signal receiving unit 211 that receives a voice signal of an utterance transmitted by the transmitting communication terminal 1 via the network 3, and a speaker 212 that outputs an utterance voice from the voice signal of the utterance voice received from the voice signal receiving unit 211. The information receiving device 20 will be described later.

[0050] [1.3 Specific configurations of the information transmitting device 10 and the information receiving device 20] FIG. 3 is a block diagram showing an example of a specific configuration of the information transmitting device 10 and the information receiving device 20 shown in FIG.

[0051] [1.3.1 Information transmitting device 10] The information transmitting device 10 is realized by, for example, a computer including a processor (microprocessor), a memory, a communication interface, etc., and undertakes part of the calculations required for the information receiving device 20 to execute the speaker recognition process. As shown in Fig. 3, the information transmitting device 10 includes a speech voice acquiring unit 111a, an analyzing unit 12, an acoustic feature calculating unit 13, a speaker feature calculating unit 14, and an information transmitting unit 15. Each component will be described below.

[0052] [1.3.1.1 Speech acquisition unit 111a] The speech voice acquiring unit 111a acquires the speech of the speaker. In this embodiment, the speech voice acquiring unit 111a acquires a voice signal of the speech voice from the microphone 111, and outputs the voice signal to the analysis unit 12 and the acoustic feature calculation unit 13. Note that the speech voice acquiring unit 111a and the microphone 111 may be the same or different.

[0053] [1.3.1.2 Analysis section 12] FIG. 4 is a diagram showing an example of a detailed configuration of the analysis unit 12 according to the present embodiment.

[0054] The analysis unit 12 analyzes condition information indicating conditions for calculating speaker features based on the speech voice obtained from the speech voice acquisition unit 111a. In this embodiment, the analysis unit 12 includes a situation analysis unit 121 and a condition information setting unit 122, as shown in FIG.

[0055] The situation analyzing unit 121 analyzes the recording situation of the speech voice from the speech voice obtained from the speech voice acquiring unit 111a. The situation analyzing unit 121 analyzes at least one of the noise level and the microphone used when recording the speech voice, and the data attributes of the speech voice as the recording situation of the speech voice. This is because the state of the speaker feature, which is a feature unique to the speaker of the speech voice calculated by the speaker feature calculating unit 14 described later, differs depending on the recording situation of the speech voice, such as the noise level, the microphone used, or the data attributes of the speech voice, i.e., the situation when the speech voice was recorded. Note that the data attributes are, for example, the gender of the speaker of the speech voice, the language spoken by the speaker of the speech voice, etc.

[0056] FIG. 5 is a diagram showing an example of feature distribution when the gender of the speaker of the speech voice is different. The feature distribution shown in FIG. 5 is a diagram showing feature values ​​extracted from each of 33,000 pieces of speech data, compressed into two dimensions. Here, the x-Vector method is a method for calculating speaker features, which are speaker-specific features called x-Vector. As shown in FIG. 5, it can be seen that the feature distribution for male voices is different from the feature distribution for female voices.

[0057] The condition information setting unit 122 sets the recording situation analyzed by the situation analyzing unit 121 as condition information. In this embodiment, the condition information setting unit 122 transmits the recording situation set as the condition information to the speaker feature calculation unit 14 and the information transmitting unit 15.

[0058] [1.3.1.3 Acoustic feature calculation unit 13] The acoustic feature calculation unit 13 calculates acoustic features for the speech voice. In this embodiment, the acoustic feature calculation unit 13 calculates acoustic features for the speech voice from the speech signal of the speech voice acquired from the speech voice acquisition unit 111a. In this embodiment, the acoustic feature calculation unit 13 calculates MFCCs (Mel-Frequency Cepstrum Coefficients), which are speech physical quantities, from the speech signal of the speech voice. MFCCs are features that represent vocal tract characteristics of a speaker. Note that the acoustic feature calculation unit 13 is not limited to calculating MFCCs as the acoustic feature for the speech voice, and may calculate a value obtained by applying a Mel filter bank to the speech signal of the speech voice, or may calculate a spectrogram of the speech signal of the speech voice. In addition, the acoustic feature calculation unit 13 may calculate acoustic features for the speech voice from the speech signal of the speech voice using a DNN (Deep Neural Network).

[0059] [1.3.1.4 Speaker feature calculation unit 14] The speaker feature calculation unit 14 uses the DNN to calculate speaker features, which are features specific to the speaker of the speech voice, from the acoustic features. In this embodiment, the speaker feature calculation unit 14 causes the DNN to calculate the speaker features from the acoustic features calculated by the acoustic feature calculation unit 13. Note that, for example, a trained DNN is used, but the present invention is not limited to this and may be trained in the background and updated periodically.

[0060] FIG. 6 is a diagram illustrating an example of a configuration of a DNN according to the present embodiment.

[0061] The DNN used by the speaker feature calculation unit 14 is composed of a neural network including an input layer, a plurality of intermediate layers, and an output layer. The DNN is composed of n+1 (n is a positive integer) or more layers, and the input layer may be called the first layer and the output layer may be called the final layer, for example, as shown in FIG. 6. The DNN is also composed of, for example, 10 or more layers, and each layer is composed of 200 to 500 nodes, as shown in FIG. 6. The DNN may be a CNN (Convolutional Neural Network) or may be other than a CNN, for example, a neural network using FFT (Fast Fourier Transform).

[0062] [1.3.1.5 Information transmission unit 15] The information transmitting unit 15 transmits the speaker features and the condition information to the information receiving device 20 as information used by the information receiving device 20, which performs speaker recognition processing of the speech voice, to recognize the speaker of the speech voice. In this embodiment, the information transmitting unit 15 transmits to the information receiving device 20 the condition information including the recording status of the speech voice obtained from the analysis unit 12, and the speaker features calculated by the speaker feature calculation unit 14. Note that the condition information and the speaker features are transmitted while being included in the information used by the information receiving device 20 to recognize the speaker of the speech voice, but are not limited to this.

[0063] [1.3.2 Information receiving device 20] The information receiving device 20 is realized by, for example, a computer equipped with a processor (microprocessor), a memory, a communication interface, etc., and performs speaker recognition processing using information received from the information transmitting device 10. As shown in Fig. 3, the information receiving device 20 includes an information receiving unit 201, a speaker recognition device 22, and a speaker display unit 28.

[0064] The information receiving unit 201 receives information transmitted from the information transmitting device 10 via the network 3. In this embodiment, the information receiving unit 201 receives from the information transmitting device 10 information used to recognize a speaker of an uttered voice, the information including condition information and speaker features.

[0065] The speaker recognition device 22 performs speaker recognition processing and transmits the recognized speaker to the speaker display unit 28. Details will be described later.

[0066] The speaker display unit 28 displays the speaker of the uttered voice recognized by the speaker recognition device 22. The speaker display unit 28 is configured with, for example, a display.

[0067] Note that the information receiving device 20 does not need to include the information receiving unit 201 and the speaker display unit 28, as long as the speaker recognition device 22 can acquire the condition information and speaker features included in the information transmitted via the network 3. In other words, in the information receiving device 20, as long as the speaker recognition device 22 can acquire the condition information and speaker features included in the information transmitted via the network 3, the information receiving unit 201 and the speaker display unit 28 are not essential components.

[0068] [1.3.2.1 Speaker Recognition Device 22] The speaker recognition device 22 is realized by, for example, a computer including a processor (microprocessor), a memory, a communication interface, etc. As shown in Fig. 3, the speaker recognition device 22 includes a condition information acquisition unit 23, a storage unit 24, a similarity calculation unit 25, a speaker feature acquisition unit 26, and a speaker identification unit 27. Note that the speaker recognition device 22 does not need to include the condition information acquisition unit 23 and the speaker feature acquisition unit 26, as long as the similarity calculation unit 25 described later can acquire the condition information and the speaker feature from the information receiving unit 201. Each component will be described below.

[0069] [1.3.2.1.1 Condition information acquisition unit 23] The condition information acquisition unit 23 acquires the condition information included in the information transmitted from the information transmission device 10. In the present embodiment, the condition information acquired by the condition information acquisition unit 23 includes the recording status of the speech voice. As described above, the recording status of the speech voice means at least one of the noise level and the microphone used when recording the speech voice, and the data attributes of the speech voice.

[0070] The condition information acquisition unit 23 transmits the recording status included in the acquired condition information to the similarity calculation unit 25 .

[0071] [1.3.2.1.2 Storage section 24] The storage unit 24 is configured with a rewritable non-volatile memory such as a hard disk (drive) or a solid state drive. The storage unit 24 stores registered speaker features, which are unique features for each of one or more registered speakers registered in advance, for each condition for calculating the registered speaker features.

[0072] 3, the storage unit 24 stores registered speaker features for each condition, such as condition A, condition B, and condition C. Condition A, condition B, and condition C are conditions that indicate the noise level or the microphone used when recording the speech voice, or at least one of the recording conditions of the speech voice as data attributes of the speech voice.

[0073] The registered speaker features for each condition are speaker features calculated based on speech sounds of registered speakers that are collected in advance for each condition and are to be recognized by the speaker recognition device 22. The registered speaker features for each condition include speaker features of one or more registered speakers.

[0074] [1.3.2.1.3 Speaker feature acquisition unit 26] The speaker feature acquisition unit 26 acquires speaker features included in the information transmitted from the information transmission device 10. Here, the speaker features acquired by the speaker feature acquisition unit 26 are features specific to the speaker of the uttered voice, calculated from the acoustic features of the uttered voice of the speaker using a DNN, as described above. As described above, in this embodiment, the speaker feature acquisition unit 26 acquires the calculated speaker features, so that the speaker recognition device 22 can not only omit the acoustic features of the uttered voice of the speaker, but also omit the calculation of the speaker features from the acoustic features.

[0075] [1.3.2.1.4 Similarity calculation unit 25] The similarity calculation unit 25 calculates the similarity between each of the registered speaker features stored in the storage unit 24 and the speaker features based on the acquired condition information and speaker features.

[0076] In this embodiment, the similarity calculation section 25 includes a selection section 251 and a similarity calculation processing section 252, as shown in FIG.

[0077] The selection unit 251 selects, from among the registered speaker features stored in the storage unit 24, registered speaker features for each of one or more registered speakers that satisfy a condition matching the recording situation acquired by the condition information acquisition unit 23. In the example shown in Fig. 3, the selection unit 251 selects, from among the registered speaker features for conditions A, B, C, etc. stored in the storage unit 24, a registered speaker feature for a condition matching the recording situation, for example, condition A.

[0078] The similarity calculation processing unit 252 calculates the similarity between the registered speaker feature for each of the one or more registered speakers selected by the selection unit 251 and the speaker feature acquired by the speaker feature acquisition unit 26. In the example shown in Fig. 3, the similarity calculation processing unit 252 calculates the similarity between each of the registered speaker features of the condition A selected by the selection unit 251 and the speaker feature acquired by the speaker feature acquisition unit 26.

[0079] The similarity calculation processing unit 252 may calculate the similarity between each of the registered speaker features of the conditions selected by the selection unit 251 and the speaker features acquired by the speaker feature acquisition unit 26 by calculating a cosine using an inner product in a vector space model. In this case, the larger the numerical value of the angle between the vectors, the lower the similarity. The similarity calculation processing unit 252 may also calculate a cosine distance, which takes a value from -1 to 1, using the inner product of a vector indicating each of the registered speaker features of the selected conditions and a vector indicating the speaker features acquired by the speaker feature acquisition unit 26. In this case, the larger the numerical value indicating the cosine distance, the higher the similarity. Of course, the method of calculating the similarity performed by the similarity calculation processing unit 252 is not limited to these.

[0080] [1.3.2.1.5 Speaker Identification Unit 27] The speaker identifying unit 27 identifies which of one or more registered speakers stored in the storage unit 24 the speaker of the uttered voice is, based on the similarity calculated by the similarity calculation unit 25, and outputs the identified speaker.

[0081] In this embodiment, the speaker identification unit 27 identifies which of the one or more registered speakers selected by the selection unit 251 the speaker of the uttered voice is, based on the similarity calculated by the similarity calculation processing unit 252, and outputs the identified registered speaker feature value to the speaker display unit 28. Here, the speaker identification unit 27 identifies the registered speaker feature value having the largest similarity calculated by the similarity calculation processing unit 252, and thereby identifies the registered speaker of the identified registered speaker feature value as the speaker of the uttered voice.

[0082] In this manner, the speaker identifying unit 27 identifies the registered speaker feature that is closest to the speaker feature acquired by the speaker feature acquiring unit 26, thereby identifying the speaker of the speech.

[0083] [1.4 Operation of the information transmitting device 10 and the information receiving device 20] Next, the operation of the information transmitting device 10 and the information receiving device 20 configured as above will be described.

[0084] Fig. 7 is a flowchart showing the operation of the information transmitting device 10 according to the present embodiment. Fig. 8 is a flowchart showing the details of the operation of step S11.

[0085] First, the information transmitting device 10 calculates an acoustic feature amount for the speech voice acquired by the speech voice acquiring unit 111a (S10).

[0086] Next, the information transmitting device 10 analyzes condition information indicating conditions for calculating speaker features based on the acquired speech voice (S11). More specifically, first, the information transmitting device 10 analyzes the recording status of the speech voice from the speech voice acquired by the speech voice acquisition unit 111a as shown in Fig. 8 (S111). Next, the information transmitting device 10 sets the recording status analyzed in step S111 as condition information (S112).

[0087] Next, the information transmission device 10 uses the DNN to calculate speaker features, which are features unique to the speaker of the speech voice, from the acoustic features calculated in step S10 (S12). Note that the processing of steps S11 and S12 is not limited to being processed in this order, and may be performed in reverse order or in parallel.

[0088] Next, the information transmitting device 10 transmits the speaker features calculated in step S12 and the condition information analyzed in step S11 to the information receiving device 20, which performs speaker recognition processing of the speech voice, as information used by the information receiving device 20 to recognize the speaker of the speech voice (S13).

[0089] FIG. 9 is a flowchart showing the operation of the information receiving device 20 according to the present embodiment.

[0090] First, the information receiving device 20 acquires the condition information included in the information transmitted from the information transmitting device 10 (S20).

[0091] Next, the information receiving device 20 acquires speaker features included in the information transmitted from the information transmitting device 10 (S21). Note that the processing of steps S20 and S21 is not limited to being performed in this order, and may be performed in the reverse order or in parallel.

[0092] Next, the information receiving device 20 calculates the similarity between each of the registered speaker features stored in the storage unit 24 and the speaker feature acquired in step S21 based on the condition information acquired in step S20 and the speaker feature acquired in step S21 (S22). As described above, the storage unit 24 stores registered speaker features, which are unique features for each of one or more registered speakers registered in advance, for each condition when the registered speaker features were calculated.

[0093] Next, the information receiving device 20 identifies which of one or more registered speakers stored in the storage unit 24 the speaker of the uttered voice is, based on the similarity calculated in step S22, and outputs the identified speaker (S23).

[0094] Fig. 10 is a flowchart showing the details of the operation of information receiving device 20 according to this embodiment. The same processes as those in Fig. 9 are denoted by the same reference numerals, and detailed explanations will be omitted.

[0095] In step S221 following step S21, if the condition information acquired in step S20 includes the recording status of the speech voice, the information receiving device 20 selects registered speaker features for each of one or more registered speakers that have conditions that match the recording status.

[0096] Next, the information reception device 20 calculates the similarity between the registered speaker feature amount for each of the one or more registered speakers selected in step S221 and the speaker feature amount acquired in step S21 (S222).

[0097] Next, the information reception device 20 identifies which of the one or more registered speakers selected in step S221 the speaker of the uttered voice is based on the similarity calculated in step S222, and outputs the identified speaker (S231).

[0098] [1.5 Effects, etc.] As described above, according to the present embodiment, the speaker recognition process performed by the information receiving device 20 can be distributed and part of the process can be assigned to the information transmitting device 10. That is, the information transmitting device 10 according to the present embodiment transmits information used by the information receiving device 20 to recognize the speaker of an uttered voice, thereby allowing the information receiving device 20 to omit part of the calculations for executing the speaker recognition process.

[0099] Specifically, the information transmitting device 10 calculates acoustic features for the speech voice, performs a process of calculating speaker features from the calculated acoustic features, and transmits the results of the process to the information receiving device 20 together with the information. This allows the information receiving device 20 to omit not only the calculation of acoustic features for the speech voice of the speaker, but also the calculation of speaker features. In other words, the information receiving device 20 can omit a part of the calculations for executing the speaker recognition process, thereby reducing the calculation load for executing the speaker recognition process.

[0100] Therefore, according to the present embodiment, the computational load of the speaker recognition process can be distributed, and therefore the computational load imposed on the information reception device 20 when the information reception device 20 executes the speaker recognition process can be reduced.

[0101] Furthermore, according to the information transmitting device 10 of this embodiment, the recording status of the analyzed speech voice is included in the condition information and transmitted to the information receiving device 20. This allows the information receiving device 20 to select registered speaker features according to the recording status and then calculate the similarity between the speaker features and the registered speaker features. In other words, the information receiving device 20 does not need to calculate the similarity between the registered speaker features and the registered speaker features for all registered speaker features under all conditions, and therefore the calculation load can be reduced. Therefore, according to this embodiment, the calculation load of the speaker recognition process can be distributed, and the calculation load required for the information receiving device 20 to execute the speaker recognition process can be reduced.

[0102] (Variation 1) In the above embodiment, the case where the information transmitting device uses the DNN to calculate the speaker feature of the speech voice and includes it in the information to be transmitted has been described, but the present invention is not limited to this. The information transmitting device may use a part of the DNN to calculate a first speaker feature corresponding to a part of the speaker feature of the speech voice and include it in the information to be transmitted. In this case, the information receiving device may use another part of the DNN to calculate the speaker feature of the speech voice from the first speaker feature. Hereinafter, this case will be described as Modification 1. Note that the following description will focus on the points that are different from the above embodiment.

[0103] [2.1 Specific configurations of the information transmitting device 10A and the information receiving device 20A] Fig. 11 is a block diagram showing an example of a specific configuration of an information transmitting device 10A and an information receiving device 20A according to Modification 1. Elements similar to those in Fig. 3 are given the same reference numerals, and detailed description thereof will be omitted.

[0104] [2.1.1 Information transmitting device 10A] An information transmitting device 10A shown in Fig. 11 differs from the information transmitting device 10 shown in Fig. 3 in the configurations of an analysis unit 12A and a speaker feature calculation unit 14A. The following mainly describes the points that differ from the above embodiment.

[0105] [2.1.1.1 Analysis section 12A] Fig. 12 is a diagram showing an example of a detailed configuration of an analysis unit 12A according to Modification 1. Elements similar to those in Fig. 4 are given the same reference numerals, and detailed description thereof will be omitted.

[0106] The analysis unit 12A analyzes condition information indicating conditions for calculating speaker features based on the speech voice obtained from the speech voice acquisition unit 111a. In this modification, the analysis unit 12A includes a situation analysis unit 121A and a condition information setting unit 122A, as shown in FIG.

[0107] The situation analyzing unit 121A analyzes the recording situation of the speech voice from the speech voice obtained from the speech voice acquiring unit 111a. The situation analyzing unit 121A further analyzes the speaking time of the speech voice obtained from the speech voice acquiring unit 111a.

[0108] The condition information setting unit 122A sets the recording situation analyzed by the situation analysis unit 121A as the condition information. The condition information setting unit 122A further sets, as the condition information, a load control condition indicating that the DNN used by the speaker feature calculation unit 14A is a first DNN, which is a part of the DNN, based on the speech time analyzed by the situation analysis unit 121A. Here, if the DNN is configured from the first layer to the final layer as shown in FIG. 6, the first DNN is a part of the DNN configured from the first layer to the nth layer (n is a positive integer) of the DNN.

[0109] In this way, the condition information setting unit 122A sets the recording status of the speech voice and the load control conditions as the condition information. Note that the condition information setting unit 122A sets the recording status of the speech voice and the load control conditions as the condition information by including the recording status of the speech voice and the load control conditions in the condition information.

[0110] [2.1.1.2 Speaker feature calculation unit 14A] The speaker feature calculation unit 14A is a functional unit for calculating speaker features from acoustic features by using a DNN.

[0111] FIG. 13 is a diagram showing an example of a detailed configuration of the speaker feature calculation unit 14A according to the first modification.

[0112] The speaker feature calculation unit 14A includes a calculation condition control unit 141 and a first speaker feature calculation unit 142, as shown in FIG.

[0113] The calculation condition control unit 141 acquires condition information set by the condition information setting unit 122A. Here, the condition information includes the recording status of the speech voice and the load control condition. The calculation condition control unit 141 causes the first speaker feature calculation unit 142 to perform feature calculation processing using the first DNN according to the load control condition included in the acquired condition information.

[0114] The first speaker feature calculation unit 142 uses the first DNN to calculate first speaker features that are in the middle of calculating speaker features that are features specific to the speaker of the speech voice, from the acoustic features acquired from the acoustic feature calculation unit 13. Note that the first speaker feature calculation unit 142 may calculate the first speaker features that are in the middle of calculating the speaker features by making the DNN calculate the speaker features from the acquired acoustic features and outputting the output from the final layer of the first DNN as the first speaker features.

[0115] In this way, the speaker feature calculation unit 14A is in the process of calculating speaker features from acquired acoustic features using the first DNN, which is a part of the DNN, in accordance with the load control conditions, and can calculate a first speaker feature corresponding to a part of the speaker features. Then, the speaker feature calculation unit 14A transmits the calculated first speaker feature to the information transmission unit 15 as the speaker feature.

[0116] As a result, the information transmitting device 10A causes the information transmitting unit 15 to transmit to the information receiving device 20A condition information including the recording status and load control conditions of the speech voice obtained from the analysis unit 12A, and the first speaker feature as a speaker feature calculated by the speaker feature calculation unit 14A.

[0117] [2.1.2 Information receiving device 20A] The information receiving device 20A receives information from the information transmitting device 10A, and performs speaker recognition processing using the received information.

[0118] In this modification, as shown in Fig. 11, information receiving device 20A includes information receiving unit 201, speaker recognition device 22A, and speaker display unit 28. That is, information receiving device 20A shown in Fig. 11 differs from information receiving device 20 shown in Fig. 3 in the configuration of speaker recognition device 22A. The following mainly describes the points that differ from the above embodiment.

[0119] [2.1.2.1 Speaker Recognition Device 22A] The speaker recognition device 22A performs speaker recognition processing using the information acquired from the information receiving unit 201, and transmits the recognized speaker to the speaker display unit .

[0120] A speaker recognition device 22A shown in FIG. 11 differs from the speaker recognition device 22 shown in FIG. 3 in the configurations of a condition information acquisition unit 23A and a speaker feature amount acquisition unit 26A.

[0121] [2.1.2.1.1 Condition information acquisition unit 23A] The condition information acquisition unit 23A acquires the condition information included in the information transmitted from the information transmission device 10A. In this modification, the condition information acquisition unit 23A acquires the condition information included in the information transmitted from the information transmission device 10A from the information receiving unit 201. The condition information acquired by the condition information acquisition unit 23A includes, in addition to the recording status of the speech voice, a load control condition indicating that a first DNN, which is a part of the DNN configured from the first layer to the nth layer (n is a positive integer) of the DNN, has been used instead of the DNN.

[0122] The condition information acquiring section 23A transmits the load control conditions, out of the recording conditions and the load control conditions included in the acquired condition information, to the speaker feature amount acquiring section 26A, and transmits the recording conditions to the similarity calculating section 25.

[0123] [2.1.2.1.2 Speaker feature acquisition unit 26A] The speaker feature acquisition unit 26A acquires a first speaker feature in the middle of calculating the speaker feature as a speaker feature included in the information transmitted from the information transmission device 10A, and calculates a second speaker feature, which is the speaker feature, from the acquired first speaker feature.

[0124] Fig. 14 is a diagram showing an example of a detailed configuration of the speaker feature acquisition unit 26A according to Modification 1. Fig. 15 is a diagram showing the configuration of a DNN according to Modification 1 and the concept of components used in calculating the first speaker feature and the second speaker feature.

[0125] The speaker feature acquisition unit 26A includes a first speaker feature acquisition unit 261, a calculation condition control unit 262, and a second speaker feature calculation unit 263, as shown in FIG.

[0126] The first speaker feature acquisition unit 261 acquires a first speaker feature in the middle of calculating the speaker feature as a speaker feature included in the information transmitted from the information transmission device 10A. Here, the first speaker feature is a first speaker feature in the middle of calculating the speaker feature, as described above, and is a first speaker feature calculated using the first DNN. Here, if the DNN has a configuration as shown in FIG. 6, the first DNN corresponds to a part of the DNN shown in FIG. 15 and is composed of the first layer to the n-th layer (n is a positive integer) of the DNN.

[0127] The calculation condition control unit 262 acquires the load control conditions transmitted by the condition information acquisition unit 23A. The calculation condition control unit 262 causes the second speaker feature calculation unit 263 to perform feature calculation processing using the second DNN according to the acquired load control conditions. Here, the second DNN corresponds to another part of the DNN shown in FIG. 15 and is a part of the DNN configured from the n+1th layer to the final layer of the DNN.

[0128] The second speaker feature calculation unit 263 uses the second DNN to calculate the second speaker feature, which is the speaker feature, from the acquired first speaker feature. The second speaker feature calculation unit 263 inputs the first speaker feature to a layer next to the final layer of the first DNN in the same DNN as the DNN used in the speaker feature calculation unit 14A of the information transmission device 10A. This allows the second speaker feature calculation unit 263 to cause the second DNN to calculate the second speaker feature.

[0129] The second speaker feature calculation unit 263 outputs the calculated second speaker feature to the similarity calculation unit 25 (more specifically, the similarity calculation processing unit 252).

[0130] In this way, the speaker feature acquisition unit 26A can calculate the second speaker feature, which is the speaker feature for the speech voice, from the acquired first speaker feature based on the transmitted load control condition using the second DNN, which is the other part of the DNN.

[0131] [2.2 Operation of the information transmitting device 10A and the information receiving device 20A] Next, the operations of the information transmitting device 10A and the information receiving device 20A configured as above will be described.

[0132] Fig. 16A and Fig. 16B are flowcharts showing details of the operation of the information transmitting device 10A according to Modification 1. Fig. 16A shows details of the operation according to Modification 1 in step S11 shown in Fig. 7. Fig. 16B shows details of the operation according to Modification 1 in step S12 shown in Fig. 7.

[0133] In step S11 shown in FIG. 7, the information transmitting device 10A first analyzes the recording status of the speech voice from the acquired speech voice and analyzes the speaking time of the speech voice (S111A) as shown in FIG. 16A. Next, the information transmitting device 10A sets the recording status analyzed in step S111A as condition information (S112A). In addition, the information transmitting device 10A sets a load control condition indicating that the DNN used in the speaker feature calculation process is the first DNN, which is a part of the DNN composed of the first layer to the nth layer (n is a positive integer) of the DNN, as condition information based on the analyzed speaking time (S113A). Note that the processing of steps S112A and S113A is not limited to being processed in this order, but may be performed in reverse order or in parallel.

[0134] In step S12 shown in Fig. 7, the information transmitting device 10A calculates a first speaker feature in the middle of calculating the speaker feature from the acoustic feature calculated in step S10 using the first DNN according to the load control condition as shown in Fig. 16B (S121A). Note that the load control condition set in step S113A shown in Fig. 16A is transmitted to the information transmitting device 10A, and the information transmitting device 10A calculates the first speaker feature according to the transmitted load control condition.

[0135] Fig. 17 is a flowchart showing details of the operation of the information receiving device 20A according to Modification 1. The same processes as those in Figs. 9 and 10 are denoted by the same reference numerals.

[0136] First, the information receiving device 20A acquires the condition information included in the information transmitted from the information transmitting device 10A (S20).

[0137] Next, the information receiving device 20A acquires a first speaker feature as a speaker feature included in the information transmitted from the information transmitting device 10A (S211A). As described above, the first speaker feature is an intermediate feature in calculating a speaker feature that is a feature unique to a speaker of an uttered voice, and is calculated from the acquired acoustic feature by using a first DNN that is a part of a DNN. The first DNN is, for example, a part of a DNN that is composed of the first layer to the n-th layer (n is a positive integer) of the DNN.

[0138] Next, the information receiving device 20A calculates a second speaker feature, which is a feature specific to the speaker of the speech voice, from the acquired first speaker feature using a second DNN, which is another part of the DNN, based on the load control condition included in the condition information acquired in step S20 (S212A). Here, the second DNN corresponds to the other part of the DNN, and is, for example, a part of the DNN that is composed of the n+1th layer to the final layer of the DNN.

[0139] Next, the information receiving device 20A selects registered speaker features for each of one or more registered speakers that satisfy the condition of the recording state of the speech voice included in the condition information acquired in step S20 (S221A). Note that the processing of steps S211A and S211B and step S221A is not limited to being processed in this order, and may be performed in reverse order or in parallel.

[0140] Next, the information reception device 20A calculates the similarity between the enrollment speaker feature for each of the one or more enrollment speakers selected in step S221A and the second speaker feature calculated in step S212A (S222A).

[0141] Next, the information reception device 20A identifies which of the one or more registered speakers selected in step S221A the speaker of the uttered voice is based on the similarity calculated in step S222A, and outputs the identified speaker (S223A).

[0142] [2.3 Effects, etc.] As described above, according to the first modification, the speaker recognition process performed by the information receiving device 20A can be distributed and part of the process can be assigned to the information transmitting device 10A. That is, the information transmitting device 10A according to the first modification transmits information used by the information receiving device 20A to recognize the speaker of an uttered voice, thereby allowing the information receiving device 20A to omit part of the calculations for executing the speaker recognition process.

[0143] Specifically, the information transmitting device 10A calculates an acoustic feature for the speech voice, and performs a process of calculating a first speaker feature corresponding to a part of the speaker feature from the calculated acoustic feature, which is a feature in the middle of calculating the speaker feature. Then, the information transmitting device 10A transmits the calculated first speaker feature and an additional control condition indicating the first DNN used to calculate the first speaker feature to the information receiving device 20A, including the above information. This allows the information receiving device 20A to not only omit the calculation of the acoustic feature for the speech voice of the speaker, but also omit part of the calculation of the speaker feature using the first speaker feature. In other words, the information receiving device 20A can omit part of the calculation for executing the speaker recognition process, and therefore the calculation load for executing the speaker recognition process can be reduced.

[0144] Therefore, according to the first modification, the computational load of the speaker recognition process can be distributed, and therefore the computational load imposed on the information reception device 20A when the information reception device 20A executes the speaker recognition process can be reduced.

[0145] Moreover, according to the information transmitting device 10A of the first modification, the recording status of the analyzed speech voice is included in the condition information and transmitted to the information receiving device 20A. This allows the information receiving device 20A to select registered speaker features according to the recording status and then calculate the similarity between the speaker features and the registered speaker features. In other words, the information receiving device 20A does not need to calculate the similarity between the registered speaker features and the registered speaker features for all registered speaker features under all conditions, and therefore the calculation load can be reduced. Therefore, according to the first modification, the calculation load of the speaker recognition process can be distributed, and the calculation load required for the information receiving device 20A to execute the speaker recognition process can be reduced.

[0146] (Variation 2) In the above modification 1, the similarity calculation unit selects registered speaker features for a condition matching the recording situation included in the condition information transmitted from the information transmitting device side from among the registered speaker features for a plurality of conditions stored in the storage unit, and then performs the similarity calculation process, but this is not limited to the above. The similarity calculation unit may perform the similarity calculation process for all registered speaker features for a plurality of conditions stored in the storage unit. This case will be described as modification 2. Note that the following will mainly describe the points that are different from the above embodiment and modification 1.

[0147] [3.1 Specific configurations of the information transmitting device 10B and the information receiving device 20B] Fig. 18 is a block diagram showing an example of a specific configuration of an information transmitting device 10B and an information receiving device 20B according to Modification 2. The same elements as those in Fig. 3 and Fig. 11 are given the same reference numerals, and detailed description thereof will be omitted.

[0148] [3.1.1 Information transmitting device 10B] The information transmitting device 10B shown in Fig. 18 differs from the information transmitting device 10A shown in Fig. 11 in the configurations of an analysis unit 12B and a speaker feature calculation unit 14B. The following mainly describes the differences from the first modification.

[0149] [3.1.1.1 Analysis section 12B] Fig. 19 is a diagram showing an example of a detailed configuration of an analysis unit 12B according to Modification 2. Elements similar to those in Figs. 4 and 12 are given the same reference numerals, and detailed description thereof will be omitted.

[0150] The analysis unit 12B analyzes condition information indicating conditions for calculating speaker features based on the speech voice obtained from the speech voice acquisition unit 111a. In this modification, the analysis unit 12B includes a situation analysis unit 121B and a condition information setting unit 122B, as shown in FIG.

[0151] The situation analyzing unit 121B analyzes the speaking time of the uttered voice acquired from the uttered voice acquiring unit 111a. Note that, in this modification, unlike the above-mentioned embodiment and modification 1, the situation analyzing unit 121B does not need to analyze the recording situation of the uttered voice.

[0152] The condition information setting unit 122B sets, as condition information, a load control condition indicating that the DNN used by the speaker feature calculation unit 14B is a first DNN, which is a part of the DNN, based on the speech time analyzed by the situation analysis unit 121B. As described in the first modification example, the first DNN is a part of the DNN that is composed of the first layer to the n-th layer (n is a positive integer) of the DNN.

[0153] In this manner, the condition information setting unit 122B sets the load control condition as the condition information. Note that the condition information setting unit 122B sets the load control condition as the condition information by including the load control condition in the condition information.

[0154] [3.1.1.2 Speaker feature calculation unit 14B] The speaker feature calculation unit 14B is a functional unit for calculating speaker features from acoustic features by using a DNN.

[0155] FIG. 20 is a diagram showing an example of a detailed configuration of the speaker feature calculation unit 14B according to the second modification.

[0156] 20, the speaker feature calculation unit 14B includes a calculation condition control unit 141B and a first speaker feature calculation unit 142. The first speaker feature calculation unit 142 is the same as described in the first modification, and therefore will not be described again.

[0157] The calculation condition control unit 141B acquires condition information set by the condition information setting unit 122B. Here, the condition information includes a load control condition. The calculation condition control unit 141B causes the first speaker feature calculation unit 142 to perform feature calculation processing using the first DNN according to the load control condition included in the acquired condition information.

[0158] In this way, the speaker feature calculation unit 14B can calculate the first speaker feature, which is in the middle of calculating the speaker feature, from the acquired acoustic feature using the first DNN, which is a part of the DNN, according to the load control conditions. Then, the speaker feature calculation unit 14B transmits the calculated first speaker feature to the information transmission unit 15 as the speaker feature.

[0159] As a result, the information transmitting device 10B causes the information transmitting unit 15 to transmit, to the information receiving device 20B, condition information including the load control condition obtained from the analysis unit 12B and the first speaker feature as the speaker feature calculated by the speaker feature calculation unit 14B.

[0160] [3.1.2 Information receiving device 20B] The information receiving device 20B receives information from the information transmitting device 10B, and performs speaker recognition processing using the received information.

[0161] In this modification, as shown in Fig. 18, information receiving device 20B includes information receiving unit 201, speaker recognition device 22B, and speaker display unit 28. That is, information receiving device 20B shown in Fig. 18 differs from information receiving device 20A shown in Fig. 11 in the configuration of speaker recognition device 22B. The following mainly describes the differences from modification 1.

[0162] [3.1.2.1 Speaker Recognition Device 22B] The speaker recognition device 22B performs speaker recognition processing using the information acquired from the information receiving unit 201, and transmits the recognized speaker to the speaker display unit .

[0163] Speaker recognition device 22B shown in FIG. 18 differs from speaker recognition device 22A shown in FIG. 11 in the configurations of condition information acquisition section 23B and similarity calculation section 25B.

[0164] [3.1.2.1.1 Condition information acquisition unit 23B] The condition information acquisition unit 23B acquires the condition information included in the information transmitted from the information transmission device 10B. In this modification, the condition information acquisition unit 23B acquires the condition information included in the information transmitted from the information transmission device 10B from the information receiving unit 201. The condition information acquired by the condition information acquisition unit 23B includes a load control condition indicating that a first DNN, which is a part of the DNN configured from the first layer to the nth layer (n is a positive integer) of the DNN, has been used instead of the DNN. Note that in this modification, unlike modification 1, the condition information acquired by the condition information acquisition unit 23B does not include the recording status of the speech voice.

[0165] The condition information acquiring section 23B transmits the load control condition included in the acquired condition information to the speaker feature amount acquiring section 26A.

[0166] [3.1.2.1.2 Similarity calculation unit 25B] The similarity calculation unit 25B calculates the similarity between each of the registered speaker features stored in the memory unit 24 and the second speaker features calculated by the speaker feature acquisition unit 26A using the first speaker features.

[0167] In this modified example, the similarity calculation unit 25B calculates the similarity between each of all registered speaker features under condition A, condition B, condition C, etc. stored in the memory unit 24 and the second speaker feature obtained from the speaker feature acquisition unit 26A.

[0168] [3.2 Effects, etc.] As described above, according to Modification 2, the speaker recognition process performed by the information receiving device 20B can be distributed and part of the process can be assigned to the information transmitting device 10B. That is, the information transmitting device 10B according to Modification 2 transmits information used by the information receiving device 20B to recognize the speaker of an uttered voice, thereby allowing the information receiving device 20B to omit part of the calculations for executing the speaker recognition process.

[0169] Specifically, the information transmitting device 10B calculates an acoustic feature for the speech voice, and performs a process of calculating a first speaker feature corresponding to a part of the speaker feature from the calculated acoustic feature, which is a feature in the middle of calculating the speaker feature. Then, the information transmitting device 10A transmits the calculated first speaker feature and an additional control condition indicating the first DNN used to calculate the first speaker feature to the information receiving device 20B as the above information. This allows the information receiving device 20B to not only omit the calculation of the acoustic feature for the speech voice of the speaker, but also to omit part of the calculation of the speaker feature using the first speaker feature. In other words, the information receiving device 20B can omit part of the calculation for executing the speaker recognition process, thereby reducing the calculation load for executing the speaker recognition process.

[0170] Therefore, according to the second modification, the computational load of the speaker recognition process can be distributed, and therefore the computational load imposed on the information reception device 20B when the information reception device 20B executes the speaker recognition process can be reduced.

[0171] While the information transmitting device and the information receiving device according to the embodiment and the modified examples have been described above, the present disclosure is not limited to the embodiment.

[0172] For example, in the above embodiment and modified examples, the case where the speaker to be recognized is a human has been described, but this is not limiting. The same can be said even if the speech of the speaker to be recognized is the voice of an animal other than a human. The same can be said even if the speech of the speaker to be recognized is further expanded to the operating sound of mechanical equipment, or the normal or abnormal sound of a machine, etc. In this case, instead of calculating MFCC, etc. as the acoustic feature, it is sufficient to calculate FFT (Fast Fourier Transform) for the sound corresponding to the speech sound.

[0173] In the above modification, the configuration of the first DNN and the second DNN used when calculating the first speaker feature and the second speaker feature is fixed, but the present invention is not limited to this. The number of layers constituting the first DNN and the second DNN used when calculating the first speaker feature and the second speaker feature may be changed depending on the transmission load of the network 3.

[0174] For example, each processing unit included in the information transmitting device and the information receiving device according to the above-mentioned embodiment and modification is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or may be integrated in part or all of them as a single chip.

[0175] The integrated circuit is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. A field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may also be used.

[0176] Furthermore, the present disclosure may be realized as an information transmitting method and an information receiving method executed by an information transmitting device and an information receiving device, etc.

[0177] In each of the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0178] In addition, the division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple blocks, some functions may be transferred to other functional blocks, and the functions of multiple functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

[0179] In addition, the order in which each step is performed in the flowchart is merely an example for specifically explaining the present disclosure, and the steps may be performed in an order other than the above. In addition, some of the steps may be performed simultaneously (in parallel) with other steps.

[0180] The information transmitting device and the information receiving device according to one or more aspects have been described above based on the embodiment and the modified examples, but the present disclosure is not limited to the embodiment and the modified examples. As long as it does not deviate from the spirit of the present disclosure, various modifications conceived by a person skilled in the art to the embodiment and the modified examples, and forms constructed by combining components in different embodiments and modified examples may also be included within the scope of one or more aspects. [Industrial Applicability]

[0181] The present disclosure can be used in an information transmitting device, an information receiving device, an information transmitting method, a program, and a system, and can be used in, for example, a remote conference system such as a telephone conference system. [Explanation of symbols]

[0182] 1. Calling communication terminal 2 Receiving communication terminal 3. Network 10, 10A, 10B Information transmitting device 11 Audio transmitting device 12, 12A, 12B Analysis Department 13 Acoustic feature calculation unit 14, 14A, 14B Speaker feature calculation unit 15 Information transmission section 20, 20A, 20B Information receiving device 21 Audio receiving device 22, 22A, 22B Speaker recognition device 23, 23A, 23B Condition information acquisition section 24 Memory section 25, 25B Similarity calculation part 26, 26A Speaker feature acquisition unit 27 Speaker Identification Unit 28 Speaker display unit 100 Communication Systems 111 Mike 111a Speech voice acquisition unit 112 Audio signal transmitter 121, 121A, 121B Situation Analysis Department 122, 122A, 122B Condition information setting section 141, 141B, 262 Calculation condition control section 142 1st speaker feature calculation unit 201 Information Receiving Department 211 Audio signal receiving unit 212 Speaker 251 Selection Section 252 Similarity calculation processing unit 261 1st Speaker Feature Acquisition Unit 263 Second speaker feature calculation unit

Claims

1. Acquire condition information and speaker features of the speech voice; Acquiring a plurality of registered speaker features based on the condition information; determining whether the speaker of the speech voice is one of a plurality of registered speakers corresponding to each of the plurality of registered speaker features based on a similarity between each of the acquired plurality of registered speaker features and the speaker feature; Information processing methods.

2. The condition information is information indicating a recording status of the speech voice or a data attribute of the speech voice. The information processing method according to claim 1 .

3. The recording status indicates at least one of a noise level and a microphone used when recording the speech sound, The data attribute of the speech indicates the gender of a speaker of the speech or the language of the speech. The information processing method according to claim 2 .

4. each of the plurality of enrollment speaker features is a feature stored in a storage device in association with any one of the plurality of enrollment speakers and any one of a plurality of conditions; the plurality of enrollment speaker features that match the condition information are selected from the plurality of enrollment speaker features stored in the storage device. The information processing method according to any one of claims 1 to 3.

5. calculating, from the speech voice, a speaker feature that is a feature specific to a speaker of the speech voice, thereby acquiring the speaker feature; acquiring the condition information by analyzing the speech sound; The information processing method according to any one of claims 1 to 4.

6. Furthermore, an acoustic feature of the speech is calculated, calculating the speaker features from the acoustic features of the speech voice using a deep neural network (DNN), thereby acquiring the speaker features; The information processing method according to any one of claims 1 to 5.

7. A processor; A memory. The processor uses the memory to: Acquire condition information and speaker features of the speech voice; Acquiring a plurality of registered speaker features based on the condition information; determining whether the speaker of the speech voice is one of a plurality of registered speakers corresponding to each of the plurality of registered speaker features based on a similarity between each of the acquired plurality of registered speaker features and the speaker feature; Information processing device.

8. A program for causing a computer to execute the information processing method according to any one of claims 1 to 6.

Images