Information processing method, information processing apparatus, and computer program product

Abstract

An information processing method, an information processing apparatus, and a computer program product are provided. In the information processing method, condition information and a speaker feature amount of a speech sound are obtained, a plurality of registered speaker feature amounts are obtained based on the condition information, and a speaker feature amount of a speech sound is obtained based on a degree of similarity between each of the obtained plurality of registered speaker feature amounts and the speaker feature amount. It is determined which of a plurality of registered speakers corresponding to each of the plurality of registered speaker feature amounts the speaker of the speech sound is.

Description

[0001] This application was submitted on March 4, 2021. The Chinese patent application number is 202180021012.0 (international application number is PCT / JP2021 / 008536). The invention name is a divisional application of a patent application entitled "Information sending device, information receiving device, information sending method, program and system". Technical Field

[0002] The present disclosure relates to an information sending device, an information receiving device, an information sending method, a recording medium, and a system. Background Art

[0003] There is a remote conference system such as a telephone conference system, which enables two or more persons to communicate by voice via a telephone network or the Internet.

[0004] In a teleconference system, when three or more people are communicating by voice, there may be a case where it is desired to display who is speaking, that is, the speaker.

[0005] For example, Patent Document 1 proposes a speaker recognition device that can recognize a speaker with high accuracy even in a noisy environment by using speaker recognition processing technology. Here, speaker recognition technology is a technology that collects speech sounds of speakers to be registered in advance, and determines which speaker of the acquired speech sound is among the registered speakers based on the similarity between a feature amount calculated from the collected speech sound and a newly acquired speech sound to be the speaker recognition target.

[0006] (Prior art literature)

[0007] (Patent Document)

[0008] Patent Document 1: Japanese Patent Application Publication No. 2020-60757

[0009] However, in the case where the speaker recognition process disclosed in the above-mentioned Patent Document 1 is applied to the teleconference system as described above, the speaker recognition process is applied to the receiving side device that receives the speech sound, and is not applied to the sending side device that sends the speech sound of the speaker to be recognized. Therefore, the receiving side device performs speaker recognition process in addition to receiving the speech sound, and thus there is a problem of increased computational load on the receiving side device. Summary of the invention

[0010] In view of the above circumstances, an object of the present disclosure is to provide an information transmitting device, an information receiving device, an information transmitting method, a recording medium, and a system capable of suppressing a calculation load for executing a speaker recognition process.

[0011] An information processing method involved in one form of the present disclosure comprises obtaining condition information and speaker characteristics of a speech sound, obtaining a plurality of registered speaker characteristics based on the condition information, and determining which of a plurality of registered speakers corresponding to the plurality of registered speaker characteristics is the speaker of the speech sound based on the similarity between each of the plurality of registered speaker characteristics obtained and the speaker characteristics.

[0012] An information processing device involved in one form of the present disclosure comprises: a processor; and a memory, wherein the processor uses the memory to execute: obtaining condition information and speaker feature quantities of a speech sound, obtaining multiple registered speaker feature quantities based on the condition information, and determining which of multiple registered speakers corresponding to the multiple registered speaker feature quantities is the speaker of the speech sound based on similarity between each of the multiple registered speaker feature quantities obtained and the speaker feature quantity.

[0013] A computer program product according to one aspect of the present disclosure includes a program for causing a computer to execute the above-mentioned information processing method.

[0014] An information sending device involved in one form of the present disclosure comprises: an acoustic feature quantity calculation unit, which calculates acoustic feature quantities for speech sounds; a speaker feature quantity calculation unit, which uses a deep neural network to calculate speaker feature quantities based on the acoustic feature quantities, wherein the speaker feature quantities are feature quantities inherent to the speaker of the speech sounds; an analysis unit, which analyzes condition information based on the speech sounds, wherein the condition information is information indicating the conditions for calculating the speaker feature quantities based on the speech sounds; and an information sending unit, which sends the speaker feature quantities and the condition information to an information receiving device, wherein the speaker feature quantities and the condition information are information used by the information receiving device, which performs speaker recognition processing on the speech sounds, to identify the speaker of the speech sounds.

[0015] In addition, these general or specific forms may also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0016] According to the information transmitting device and the like of the present disclosure, it is possible to suppress the calculation load for executing the speaker recognition process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram showing an example of the configuration of a communication system according to the embodiment.

[0018] Figure 2 It is shown Figure 1 A block diagram showing an example of the configuration of a transmission-side communication terminal and a reception-side communication terminal shown.

[0019] Figure 3 It is shown Figure 2 A block diagram showing an example of the specific configuration of the information transmitting device and the information receiving device shown.

[0020] Figure 4 This is a diagram showing an example of a detailed configuration of an analysis unit according to an embodiment.

[0021] Figure 5 This is a diagram showing an example of feature quantity distribution when the gender of the speaker of the speech sound is different.

[0022] Figure 6 This is a diagram showing an example of the configuration of a DNN according to an embodiment.

[0023] Figure 7 This is a flowchart showing the operation of the information transmission device involved in the embodiment.

[0024] Figure 8 It is a flowchart showing the specific operation of step S11.

[0025] Fig. 9 This is a flowchart showing the operation of the information receiving device involved in the embodiment.

[0026] Fig.10 This is a flowchart showing the specific operation of the information receiving device involved in the embodiment.

[0027] Fig.11 This is a block diagram showing an example of the specific configuration of an information transmitting device and an information receiving device according to Modification Example 1.

[0028] Fig.12 This is a diagram showing an example of a detailed configuration of an analysis unit according to Modification 1.

[0029] Fig.13 This is a diagram showing an example of a detailed configuration of a speaker feature value calculation unit according to Modification 1.

[0030] Fig.14 This is a diagram showing an example of a detailed configuration of a speaker feature quantity obtaining unit according to Modification 1.

[0031] Fig.15 This is a diagram showing the configuration of a DNN according to Modification 1 and a component used in calculation of a first speaker feature value and a second speaker feature value.

[0032] Fig.16AThis is a flowchart showing the specific operation of the information transmission device involved in Modification Example 1.

[0033] Fig. 16B This is a flowchart showing the specific operation of the information transmission device involved in Modification Example 1.

[0034] Fig.17 This is a flowchart showing the specific operation of the information receiving device involved in Modification Example 1.

[0035] Fig.18 This is a block diagram showing an example of the specific configuration of an information transmitting device and an information receiving device according to Modification Example 2.

[0036] Fig.19 This is a diagram showing an example of a detailed configuration of an analysis unit according to Modification 2.

[0037] Fig. 20 This is a diagram showing an example of a detailed configuration of a speaker feature value calculation unit according to Modification 2.

[0038] Explanation of symbols

[0039] 1. Sending side communication terminal

[0040] 2 Receiving side communication terminal

[0041] 3 Network

[0042] 10, 10A, 10B information transmission device

[0043] 11 Sound transmitting device

[0044] 12, 12A, 12B Analysis Department

[0045] 13Acoustic feature quantity calculation unit

[0046] 14, 14A, 14B Speaker characteristic quantity calculation unit

[0047] 15 Information sending department

[0048] 20, 20A, 20B information receiving device

[0049] 21 Sound receiving device

[0050] 22, 22A, 22B Speaker identification device

[0051] 23, 23A, 23B Condition Information Acquisition Unit

[0052] 24 Storage

[0053] 25, 25B Similarity calculation unit

[0054] 26, 26A Speaker characteristic quantity acquisition unit

[0055] 27 Speaker Identification

[0056] 28 Speaker display unit

[0057] 100 Communication Systems

[0058] 111 Microphone

[0059] 111a Speech sound acquisition unit

[0060] 112 Sound signal sending unit

[0061] 121, 121A, 121B Situation Analysis Department

[0062] 122, 122A, 122B condition information setting unit

[0063] 141, 141B, 262 Calculation Condition Control Unit

[0064] 142 1st speaker feature quantity calculation unit

[0065] 201 Information Receiving Department

[0066] 211 Sound signal receiving unit

[0067] 212 Speakers

[0068] 251 Selection Department

[0069] 252 Similarity calculation processing unit

[0070] 261 First speaker characteristic quantity obtaining unit

[0071] 263 Second speaker feature calculation unit DETAILED DESCRIPTION

[0072] (The process of obtaining one form of the present disclosure)

[0073] In a teleconference system, when three or more people communicate only by voice, there may be a case where it is desired to display who is talking, that is, the speaker.

[0074] However, when the speaker recognition process disclosed in Patent Document 1 is applied, the speaker recognition process is applied to the receiving side device that receives the speech sound, and is not applied to the emitting side device that emits the speech sound of the speaker to be recognized. Therefore, the receiving side device performs speaker recognition process in addition to receiving the speech sound.

[0075] However, the speaker recognition process includes: a process of calculating speaker characteristics based on acoustic characteristics obtained based on the speech sound of the speaker to be recognized; and a process of analyzing the registered speaker closest to the calculated speaker characteristics by comparing with the pre-stored registered speaker characteristics. The process of calculating the speaker characteristics requires a huge amount of calculation, so the calculation load is large. Moreover, in the process of analyzing the registered speaker closest to the calculated speaker characteristics, if the registered speaker characteristics that meet the conditions when the speech sound is recorded are not compared with the calculated speaker characteristics of the speech sound, analysis errors will occur. Furthermore, if all the pre-stored registered speaker characteristics are compared with the calculated speaker characteristics, it will become a calculation load.

[0076] Therefore, there is a problem that the computation load of the receiving device increases. Then, the inventors thought of a method of distributing the computation load of the speaker identification process by making the transmitting device that transmits the speech sound bear part of the computation of the speaker identification process.

[0077] An information sending device involved in one form of the present disclosure comprises: an acoustic feature quantity calculation unit, which calculates acoustic feature quantities for speech sounds; a speaker feature quantity calculation unit, which uses a deep neural network to calculate speaker feature quantities based on the acoustic feature quantities, wherein the speaker feature quantities are feature quantities inherent to the speaker of the speech sounds; an analysis unit, which analyzes condition information based on the speech sounds, wherein the condition information is information indicating the conditions for calculating the speaker feature quantities based on the speech sounds; and an information sending unit, which sends the speaker feature quantities and the condition information to an information receiving device, wherein the speaker feature quantities and the condition information are information used by the information receiving device, which performs speaker recognition processing on the speech sounds, to identify the speaker of the speech sounds.

[0078] According to this aspect, by sending information used to identify the speaker of a speech sound to an information receiving device that performs speaker recognition processing of a speech sound, a part of the calculation performed by the information receiving device to perform speaker recognition processing can be omitted. Specifically, the information receiving device can not only omit the calculation of the acoustic feature quantity for the speaker's speech sound, but also at least omit a part of the processing of calculating the speaker feature quantity based on the acoustic feature quantity, thereby reducing the calculation load for performing speaker recognition processing. Therefore, according to this aspect, the calculation load for performing speaker recognition processing can be suppressed.

[0079] Furthermore, for example, the analyzing unit may include a situation analyzing unit that analyzes a recording situation of the speech sound based on the speech sound and a condition information setting unit that sets the recording situation as the condition information.

[0080] According to this aspect, by sending the collection status of the analyzed speech sound as condition information to the information receiving device, the information receiving device can perform similarity calculation processing based on selecting the registered speaker feature corresponding to the collection status. Accordingly, the information receiving device does not need to perform similarity calculation processing on the registered speaker feature under all conditions, so the calculation load can be reduced. In addition, the information sending device includes the speaker feature of the speech sound of the speaker to be identified in the information used to identify the speaker of the speech sound and sends it. Accordingly, the information receiving device can omit the calculation of the speaker feature of the speech sound of the speaker to be identified. Therefore, according to this aspect, the calculation load for performing speaker recognition processing can be further suppressed.

[0081] Furthermore, for example, the recording status may indicate at least one of a noise level and a microphone used when recording the speech sound, and a data attribute of the speech sound.

[0082] Furthermore, for example, the situation analysis unit may further analyze the speech time of the speech sound, and the condition information setting unit may further set a load control condition as the condition information according to the speech time, wherein the load control condition refers to using the deep neural network used by the speaker feature calculation unit as a first deep neural network, and the first deep neural network is a part of the deep neural network composed of the first layer to the nth layer in the deep neural network, where n is a positive integer, and the speaker feature calculation unit uses the first deep neural network according to the load control condition to calculate a first speaker feature according to the acoustic feature as the speaker feature, and the first speaker feature is a feature obtained in the process of calculating the speaker feature.

[0083] According to the present embodiment, the first speaker feature is included in the information used to identify the speaker of the speech sound, and is sent to the information receiving device that performs speaker recognition processing. The first speaker feature is equivalent to a part of the speaker feature of the speech sound of the speaker to be recognized. Accordingly, a part of the calculation performed by the information receiving device to perform the speaker recognition processing can be omitted. Specifically, by receiving the information, the information receiving device can not only omit the calculation of the acoustic feature of the speaker's speech sound, but also omit a part of the calculation of the speaker feature by using the first speaker feature. Furthermore, when the information receiving device receives the collection status included in the information, it can perform the similarity calculation processing based on the selection of the registered speaker feature corresponding to the collection status, thereby reducing the calculation load.

[0084] Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be suppressed.

[0085] Furthermore, for example, the analysis unit may include: a situation analysis unit, which analyzes the speech time of the speech sound; and a condition information setting unit, which sets a load control condition as the condition information according to the speech time, wherein the load control condition refers to using the deep neural network used by the speaker feature calculation unit as a first deep neural network, which is a part of the deep neural network composed of the first layer to the nth layer in the deep neural network, where n is a positive integer, and the speaker feature calculation unit uses the first deep neural network according to the load control condition to calculate a first speaker feature according to the acoustic feature as the speaker feature, and the first speaker feature is a feature obtained in the process of calculating the speaker feature.

[0086] According to this aspect, the first speaker feature is included in the information used to identify the speaker of the speech sound, and is sent to the information receiving device that performs speaker recognition processing. The first speaker feature is equivalent to a part of the speaker feature of the speech sound of the speaker to be recognized. According to this, at least a part of the calculation performed by the information receiving device to perform the speaker recognition processing can be omitted. Specifically, by receiving this information, the information receiving device can not only omit the calculation of the acoustic feature of the speaker's speech sound, but also omit a part of the calculation of the speaker feature by using the first speaker feature.

[0087] Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be suppressed.

[0088] Furthermore, an information receiving device involved in one form of the present disclosure comprises: a storage unit, in which the registered speaker feature quantity is stored according to each condition when calculating the registered speaker feature quantity, and the registered speaker feature quantity is an inherent feature quantity for each of one or more registered speakers registered in advance; a condition information acquisition unit, which obtains the condition information included in the information sent by the information sending device of one of the above forms; a speaker feature quantity acquisition unit, which obtains the speaker feature quantity included in the information sent by the information sending device; a similarity calculation unit, which calculates the similarity between each of the registered speaker feature quantities stored in the storage unit and the speaker feature quantity based on the obtained condition information and the speaker feature quantity; and a speaker determination unit, which determines and outputs which registered speaker of the speech sound is one of the one or more registered speakers stored in the storage unit based on the similarity calculated by the similarity calculation unit.

[0089] According to this aspect, an information receiving device that performs speaker recognition processing of speech sound can omit at least a part of the calculation performed for executing speaker recognition processing by receiving information used to identify the speaker of the speech sound. Specifically, the information receiving device can not only omit the calculation of the acoustic feature quantity for the speaker's speech sound, but also can omit at least a part of the processing for calculating the speaker feature quantity based on the acoustic feature quantity, thereby reducing the calculation load for executing speaker recognition processing.

[0090] Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be suppressed.

[0091] Furthermore, for example, the similarity calculation unit may include: a selection unit, which, when the condition information obtained by the condition information acquisition unit includes the recording status of the speech sound, selects a registered speaker feature value for each of the one or more registered speakers, the registered speaker feature value being also a registered speaker feature value under the condition that satisfies the recording status; and a similarity calculation processing unit, which calculates the similarity between the registered speaker feature value selected by the selection unit and the speaker feature value obtained by the speaker feature value acquisition unit, the registered speaker feature value being a feature value for each of the one or more registered speakers, and the speaker determination unit, which determines and outputs, based on the similarity calculated by the similarity calculation processing unit, which registered speaker of the speech sound is one of the one or more registered speakers selected by the selection unit.

[0092] According to this aspect, the information receiving device can perform similarity calculation processing based on the selection of the registered speaker feature quantity corresponding to the recording status by obtaining the recording status of the speech sound included in the received condition information. Accordingly, the information receiving device does not need to perform similarity calculation processing on the registered speaker feature quantities under all conditions, thereby reducing the calculation load. In addition, the information receiving device can omit the calculation of the speaker feature quantity for the speech sound of the speaker to be recognized by receiving the speaker feature quantity of the speech sound of the speaker to be recognized. Therefore, according to this aspect, the calculation load for performing speaker recognition processing can be further suppressed.

[0093] Furthermore, for example, the condition information obtaining unit, when the condition information obtained also includes a load control condition, passes the load control condition to the speaker feature quantity obtaining unit, the load control condition means that a first deep neural network in the deep neural network is used instead of the deep neural network, the first deep neural network is a part of the deep neural network composed of the first layer to the nth layer in the deep neural network, and n is a positive integer, and the speaker feature quantity obtaining unit, when the first speaker feature quantity is obtained as the speaker feature quantity included in the information and the load control condition is passed on, calculates a second speaker feature quantity as the speaker feature quantity based on the first speaker feature quantity according to the load control condition using the second deep neural network in the deep neural network, the first speaker feature quantity being a feature quantity obtained in the process of calculating the speaker feature quantity and being a feature quantity calculated using the first deep neural network, and the second deep neural network being a part of the deep neural network composed of the n+1th layer to the last layer in the deep neural network.

[0094] According to this aspect, the information receiving device can omit a part of the calculation performed for executing the speaker recognition processing by obtaining the first speaker feature value corresponding to a part of the speaker feature value of the speech sound of the speaker to be recognized. Specifically, by receiving the first speaker feature value, the information receiving device can not only omit the calculation of the acoustic feature value for the speech sound of the speaker, but also omit a part of the calculation of the speaker feature value by using the first speaker feature value. Furthermore, when the information receiving device receives the collection status included in the information, it can perform the similarity calculation processing based on selecting the registered speaker feature value corresponding to the collection status, thereby reducing the calculation load.

[0095] Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be suppressed.

[0096] Furthermore, for example, the condition information obtaining unit, when the load control condition shown in the obtained condition information indicates that the first deep neural network in the deep neural network is used instead of the deep neural network, passes the load control condition to the speaker feature quantity obtaining unit, the first deep neural network being a part of the deep neural network composed of the first layer to the nth layer in the deep neural network, n being a positive integer, and the speaker feature quantity obtaining unit, when the first speaker feature quantity is obtained as the speaker feature quantity included in the information and the load control condition is passed on, calculates the second speaker feature quantity as the speaker feature quantity based on the first speaker feature quantity according to the load control condition using the second deep neural network in the deep neural network, the first speaker feature quantity being a feature quantity obtained in the process of calculating the speaker feature quantity and being a feature quantity calculated using the first deep neural network, and the second deep neural network being a part of the deep neural network composed of the n+1th layer to the last layer in the deep neural network.

[0097] According to this aspect, the information receiving device can omit a part of the calculation performed for executing the speaker recognition process by obtaining the first speaker feature value corresponding to a part of the speaker feature value of the speech sound of the speaker to be recognized. Specifically, by receiving the first speaker feature value, the information receiving device can not only omit the calculation of the acoustic feature value for the speech sound of the speaker, but also omit a part of the calculation of the speaker feature value by using the first speaker feature value. Therefore, according to this aspect, the calculation load for executing the speaker recognition process can be suppressed.

[0098] One form of the present disclosure involves an information sending method, which is an information sending method performed by a computer, calculating an acoustic feature quantity for a speech sound, using a deep neural network to calculate a speaker feature quantity based on the acoustic feature quantity, the speaker feature quantity being a feature quantity inherent to the speaker of the speech sound, analyzing condition information showing the conditions when calculating the speaker feature quantity based on the speech sound, and sending the speaker feature quantity and the condition information to an information receiving device, the speaker feature quantity and the condition information are information used by the information receiving device that performs speaker recognition processing on the speech sound in order to identify the speaker of the speech sound.

[0099] Furthermore, a system involved in one form of the present disclosure comprises an information sending device and an information receiving device that performs speaker recognition processing, the information sending device comprising: an acoustic feature quantity calculation unit that calculates acoustic feature quantities for a speech sound; a speaker feature quantity calculation unit that uses a deep neural network to calculate a speaker feature quantity that can identify the speaker of the speech sound based on the acoustic feature quantities; an analysis unit that analyzes condition information showing conditions when calculating the speaker feature quantity based on the speech sound; and an information sending unit that sends the speaker feature quantity and the condition information to an information receiving device, the speaker feature quantity and the condition information being information used by the information receiving device that identifies the speaker of the speech sound in order to identify the speaker of the speech sound, the information receiving device comprising: a storage unit, in which According to each condition when calculating the registered speaker feature quantity, the registered speaker feature quantity is stored, and the registered speaker feature quantity is an inherent feature quantity for each of one or more registered speakers registered in advance; a condition information acquisition unit obtains the condition information included in the information sent by the information sending device; a speaker feature quantity acquisition unit obtains the speaker feature quantity included in the information sent by the information sending device; a similarity calculation unit calculates the similarity between each of the registered speaker feature quantities stored in the storage unit and the speaker feature quantity based on the obtained condition information and the speaker feature quantity; and a speaker determination unit determines and outputs which registered speaker of the speech sound is among the one or more registered speakers stored in the storage unit based on the similarity calculated by the similarity calculation unit.

[0100] In addition, these general or specific forms may also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0101] The following describes the embodiments of the present disclosure with reference to the accompanying drawings. The embodiments to be described below are all specific examples of the present disclosure. The numerical values, shapes, constituent elements, steps, the order of steps, etc. shown in the following embodiments are all examples, and their main purpose is not to limit the present disclosure. In addition, the constituent elements of the following embodiments that are not recorded in the independent technical solutions showing the highest-level concepts will be described as arbitrary constituent elements. In addition, in all embodiments, the contents of each embodiment can be combined.

[0102] (Implementation Method)

[0103] Hereinafter, an information transmission device and the like in the embodiments and the like will be described with reference to the drawings.

[0104] [1.1 Communication system 100]

[0105] Figure 1 This is a block diagram showing an example of the configuration of the communication system 100 according to the present embodiment.

[0106] The communication system 100 according to the present embodiment is a remote conference system in which two or more persons can communicate using voice, and is used in, for example, a telephone conference system.

[0107] like Figure 1 As shown, the communication system 100 includes a transmission-side communication terminal 1 and a reception-side communication terminal 2. These terminals are connected via a network 3 such as a telephone network, a LAN (Local Area Network) or the Internet.

[0108] [1.2 Configuration of the sending-side communication terminal 1 and the receiving-side communication terminal 2]

[0109] Figure 2 It is shown Figure 1 A block diagram showing an example of the configuration of the transmission-side communication terminal 1 and the reception-side communication terminal 2 shown.

[0110] The transmission-side communication terminal 1 is a terminal used by the transmission side, that is, the speaker, and when the speaker speaks, the transmission-side communication terminal 1 transmits a speech sound signal and information obtained based on the speech sound to the reception-side communication terminal 2 via the network 3 .

[0111] like Figure 2 As shown, the transmitting communication terminal 1 includes an information transmitting device 10 for transmitting the information, and a voice transmitting device 11 for transmitting a voice signal of a speech. Here, the voice transmitting device 11 includes a microphone 111 and a voice signal transmitting unit 112. The microphone 111 outputs a voice signal of a speech sound after a speech of a speaker is input, and the voice signal transmitting unit 112 transmits the voice signal received from the microphone 111 to the receiving communication terminal 2 via the network 3. The information transmitting device 10 will be described later.

[0112] The receiving side communication terminal 2 is a terminal on the receiving side, and is used by people who listen to the speech of the speaker on the sending side. It receives the sound signal of the speaker sent by the sending side communication terminal 1 via the network 3 and the information obtained based on the speech sound.

[0113] like Figure 2As shown, the receiving side communication terminal 2 includes an information receiving device 20 for receiving the information, and a voice receiving device 21 for receiving a voice signal of a speaker on the transmitting side. Here, the voice receiving device 21 includes a voice signal receiving unit 211 and a speaker 212. The voice signal receiving unit 211 receives a voice signal of a speech voice transmitted from the transmitting side communication terminal 1 via the network 3, and the speaker 212 outputs the speech voice based on the voice signal of the speech voice received from the voice signal receiving unit 211. The information receiving device 20 will be described later.

[0114] [1.3 Specific Configuration of Information Transmitting Device 10 and Information Receiving Device 20]

[0115] Figure 3 It is shown Figure 2 The block diagram is a block diagram of an example of a specific configuration of the information transmitting device 10 and the information receiving device 20 shown.

[0116] [1.3.1 Information transmitting device 10]

[0117] For example, the information transmitting device 10 is implemented by a computer having a processor (microprocessor), a memory, a communication interface, etc., and is responsible for a part of the calculation required by the information receiving device 20 to perform the speaker recognition process. Figure 3 As shown, the information transmitting device 10 includes a speech sound obtaining unit 111a, an analyzing unit 12, an acoustic feature quantity calculating unit 13, a speaker feature quantity calculating unit 14, and an information transmitting unit 15. Each configuration will be described below.

[0118] [1.3.1.1 Speech sound acquisition unit 111a]

[0119] The speech sound acquisition unit 111a acquires the speech sound of the speaker. In this embodiment, the speech sound acquisition unit 111a acquires the speech sound signal from the microphone 111 and outputs it to the analysis unit 12 and the acoustic feature quantity calculation unit 13. The speech sound acquisition unit 111a and the microphone 111 may be the same or different.

[0120] [1.3.1.2 Analysis Department 12]

[0121] Figure 4 It is a diagram showing an example of a detailed configuration of the analysis unit 12 according to the present embodiment.

[0122] The analyzing unit 12 analyzes the condition information indicating the conditions for calculating the speaker feature quantity based on the speech sound obtained from the speech sound obtaining unit 111a. Figure 4 As shown, the analyzing unit 12 includes a situation analyzing unit 121 and a condition information setting unit 122 .

[0123] The situation analysis unit 121 analyzes the recording status of the speech sound based on the speech sound obtained from the speech sound acquisition unit 111a. The situation analysis unit 121 analyzes at least one of the noise level when recording the speech sound, the microphone used, and the data attribute of the speech sound as the recording status of the speech sound. This is because the recording status of the speech sound, such as the noise level, the microphone used, or the data attribute of the speech sound, that is, the status when the speech sound is recorded, will cause the speaker characteristic quantity of the speech sound calculated by the speaker characteristic quantity calculation unit 14 described later, that is, the speaker characteristic quantity, to be different in form. In addition, the data attribute is, for example, the gender of the speaker of the speech sound, the language spoken by the speaker of the speech sound, etc.

[0124] Figure 5 FIG. 1 is a diagram showing an example of feature quantity distribution when the gender of the speaker of the speech sound is different. Figure 5 The feature quantity distribution shown is obtained by compressing the feature quantity extracted by the x-vector method into two dimensions for each of the 33,000 voice data. Here, the x-vector method is a method of calculating the speaker characteristic quantity, which is a speaker-specific feature called an x-vector. Figure 5 It can be seen that the distribution of feature quantities for male voices is different from that for female voices.

[0125] The condition information setting unit 122 sets the recording status analyzed by the status analysis unit 121 as condition information. In the present embodiment, the condition information setting unit 122 transmits the recording status set as condition information to the speaker feature calculation unit 14 and the information transmission unit 15.

[0126] [1.3.1.3 Acoustic feature quantity calculation unit 13]

[0127] The acoustic feature quantity calculation unit 13 calculates the acoustic feature quantity for the speech sound. In the present embodiment, the acoustic feature quantity calculation unit 13 calculates the acoustic feature quantity for the speech sound based on the sound signal of the speech sound obtained from the speech sound acquisition unit 111a. In the present embodiment, the acoustic feature quantity calculation unit 13 calculates MFCC (Mel-Frequency Cepstrum Coefficients) as a sound physical quantity based on the sound signal of the speech sound. MFCC is a feature quantity that represents the vocal tract characteristics of the speaker. In addition, as the acoustic feature quantity for the speech sound, it is not limited to the case where the acoustic feature quantity calculation unit 13 calculates MFCC, and it can also be calculated by filtering the sound signal of the speech sound through a Mel filter group, and the frequency spectrum of the sound signal of the speech sound can be calculated. In addition, the acoustic feature quantity calculation unit 13 can also use DNN (Deep Neural Network) to calculate the acoustic feature quantity for the speech sound based on the sound signal of the speech sound.

[0128] [1.3.1.4 Speaker feature calculation unit 14]

[0129] The speaker feature calculation unit 14 uses the DNN to calculate the speaker feature, which is a feature inherent to the speaker of the speech sound, from the acoustic feature. In the present embodiment, the speaker feature calculation unit 14 causes the DNN to calculate the speaker feature based on the acoustic feature calculated by the acoustic feature calculation unit 13. In addition, although the DNN uses, for example, a learned DNN, the present invention is not limited thereto, and learning may be performed in the background and periodically updated.

[0130] Figure 6 This is a diagram showing an example of the configuration of the DNN according to this embodiment.

[0131] The DNN used by the speaker feature calculation unit 14 is composed of a neural network having an input layer, a plurality of intermediate layers, and an output layer. The DNN is composed of n+1 (n is a positive integer) or more layers, such as Figure 6 As shown in , for example, there is a case where the input layer is called the first layer and the output layer is called the last layer. Figure 6 As shown, the DNN is composed of, for example, more than 10 layers, and each layer is composed of 200 to 500 nodes. In addition, the DNN can be a CNN (Convolutional Neural Network), a neural network other than a CNN, or a neural network that uses, for example, FFT (Fast Fourier Transform).

[0132] [1.3.1.5 Information sending unit 15]

[0133] The information transmitting unit 15 transmits the speaker characteristic quantity and condition information, which are information used by the information receiving device 20 that performs speaker recognition processing of the speech sound to recognize the speaker of the speech sound, to the information receiving device 20. In the present embodiment, the information transmitting unit 15 transmits the condition information including the recording status of the speech sound obtained from the analyzing unit 12 and the speaker characteristic quantity calculated by the speaker characteristic quantity calculating unit 14 to the information receiving device 20. In addition, although the condition information and the speaker characteristic quantity are included in the information used by the information receiving device 20 to recognize the speaker of the speech sound and transmitted, the present invention is not limited to this.

[0134] [1.3.2 Information receiving device 20]

[0135] For example, the information receiving device 20 is implemented by a computer having a processor (microprocessor), a memory, a communication interface, etc., and performs speaker recognition processing using the information received from the information transmitting device 10. Figure 3 As shown, the information receiving device 20 includes an information receiving unit 201 , a speaker identification device 22 , and a speaker display unit 28 .

[0136] The information receiving unit 201 receives information transmitted from the information transmitting device 10 via the network 3. In the present embodiment, the information receiving unit 201 receives information used to identify a speaker of a speech sound from the information transmitting device 10, and the information also includes condition information and speaker feature quantities.

[0137] The speaker identification device 22 performs speaker identification processing and sends the identified speaker to the speaker display unit 28. This will be described in detail later.

[0138] The speaker display unit 28 displays the speaker of the speech sound recognized by the speaker recognition device 22. The speaker display unit 28 is composed of, for example, a display or the like.

[0139] In addition, if the speaker recognition device 22 can obtain the condition information and speaker characteristics included in the information sent via the network 3, the information receiving device 20 may not include the information receiving unit 201 and the speaker display unit 28. In other words, in the information receiving device 20, when the speaker recognition device 22 can obtain the condition information and speaker characteristics included in the information sent via the network 3, the information receiving unit 201 and the speaker display unit 28 are not essential components.

[0140] [1.3.2.1 Speaker identification device 22]

[0141] For example, the speaker recognition device 22 is implemented by a computer having a processor (microprocessor), a memory, a communication interface, etc. Figure 3 As shown, the speaker recognition device 22 includes a condition information acquisition unit 23, a storage unit 24, a similarity calculation unit 25, a speaker characteristic quantity acquisition unit 26, and a speaker identification unit 27. In addition, if the similarity calculation unit 25 described later can obtain the condition information and the speaker characteristic quantity from the information receiving unit 201, the speaker recognition device 22 may not include the condition information acquisition unit 23 and the speaker characteristic quantity acquisition unit 26. The following describes each configuration.

[0142] [1.3.2.1.1 Condition information acquisition unit 23]

[0143] The condition information acquisition unit 23 acquires the condition information included in the information transmitted by the information transmission device 10. In the present embodiment, the condition information acquired by the condition information acquisition unit 23 includes the recording condition of the speech sound. As described above, the recording condition of the speech sound means at least one of the noise level and the microphone used when recording the speech sound, and the data attribute of the speech sound.

[0144] The condition information acquisition unit 23 passes the recording status included in the acquired condition information to the similarity calculation unit 25 .

[0145] [1.3.2.1.2 Storage unit 24]

[0146] The storage unit 24 is composed of a rewritable nonvolatile memory such as a hard disk drive or a solid state drive. The storage unit 24 stores the unique feature quantity of each of one or more registered speakers registered in advance, that is, the registered speaker feature quantity, according to each condition when calculating the registered speaker feature quantity.

[0147] exist Figure 3 In the example shown, the storage unit 24 stores the registered speaker feature values ​​for each condition, such as condition A, condition B, and condition C. Condition A, condition B, and condition C are conditions of speech sound recording conditions that indicate at least one of the noise level when recording the speech sound, the microphone used, or the data attribute of the speech sound.

[0148] The registered speaker feature of each condition is a speaker feature calculated based on speech sounds collected in advance for each condition, which are also speech sounds of registered speakers that are to be recognized by the speaker recognition device 22. The registered speaker feature of each condition includes speaker feature values ​​of one or more registered speakers.

[0149] [1.3.2.1.3 Speaker characteristic quantity acquisition unit 26]

[0150] The speaker feature quantity acquisition unit 26 acquires the speaker feature quantity included in the information transmitted by the information transmission device 10. Here, the speaker feature quantity acquired by the speaker feature quantity acquisition unit 26 is a speaker-specific feature quantity of the speech sound calculated based on the acoustic feature quantity of the speaker's speech sound using the DNN as described above. Thus, in this embodiment, since the speaker feature quantity acquisition unit 26 acquires the calculated speaker feature quantity, the speaker recognition device 22 can not only omit the calculation of the acoustic feature quantity for the speaker's speech sound, but also omit the calculation of the speaker feature quantity based on the acoustic feature quantity.

[0151] [1.3.2.1.4 Similarity calculation unit 25]

[0152] The similarity calculation unit 25 calculates the similarity between each of the registered speaker feature quantities stored in the storage unit 24 and the speaker feature quantity based on the acquired condition information and the speaker feature quantity.

[0153] In this embodiment, if Figure 3 As shown, the similarity calculation unit 25 includes a selection unit 251 and a similarity calculation processing unit 252 .

[0154] The selection unit 251 selects a registered speaker feature value that meets the condition of the recording condition obtained by the condition information obtaining unit 23 from the registered speaker feature values ​​stored in the storage unit 24, and the registered speaker feature value is also a registered speaker feature value for each of more than one registered speaker. Figure 3 In the example, the selection unit 251 selects a registered speaker feature quantity that meets the condition of the recording condition, such as the registered speaker feature quantity of condition A, from the registered speaker feature quantities of condition A, condition B, condition C, etc. stored in the storage unit 24.

[0155] The similarity calculation processing unit 252 calculates the similarity between the registered speaker feature value for each of the one or more registered speakers selected by the selection unit 251 and the speaker feature value obtained by the speaker feature value obtaining unit 26. Figure 3 In the example of , the similarity calculation processing unit 252 calculates the similarity between each of the registered speaker feature quantities of the condition A selected by the selection unit 251 and the speaker feature quantity obtained by the speaker feature quantity obtaining unit 26 .

[0156] In addition, the similarity calculation processing unit 252 may also calculate the cosine by using the inner product in the vector space model to calculate the similarity between each of the registered speaker feature quantities under the conditions selected by the selection unit 251 and the speaker feature quantity obtained by the speaker feature quantity obtaining unit 26. In this case, the larger the value of the angle between the vectors, the lower the similarity. Furthermore, the similarity calculation processing unit 252 may also calculate the cosine distance in the value range of negative 1 to 1 by using the inner product of the vector showing each of the registered speaker feature quantities under the selected conditions and the vector showing the speaker feature quantity obtained by the speaker feature quantity obtaining unit 26. In this case, the larger the value of the cosine distance, the higher the similarity. Of course, the method by which the similarity calculation processing unit 252 calculates the similarity is not limited thereto.

[0157] [1.3.2.1.5 Speaker identification unit 27]

[0158] The speaker identification unit 27 identifies which registered speaker the speaker of the speech sound is among the one or more registered speakers stored in the storage unit 24 based on the similarity calculated by the similarity calculation unit 25 , and outputs the identification.

[0159] In the present embodiment, the speaker identification unit 27 identifies which of the one or more registered speakers selected by the selection unit 251 the speaker of the speech sound is based on the similarity calculated by the similarity calculation processing unit 252, and outputs the identification to the speaker display unit 28. Here, the speaker identification unit 27 identifies the registered speaker feature value of the largest similarity calculated by the similarity calculation processing unit 252, and identifies the registered speaker of the identified registered speaker feature value as the speaker of the speech sound.

[0160] In this way, the speaker identifying unit 27 identifies the registered speaker feature value that is closest to the speaker feature value obtained by the speaker feature value obtaining unit 26 , thereby identifying the speaker of the speech sound.

[0161] [1.4 Operation of the information transmitting device 10 and the information receiving device 20]

[0162] Next, the operations of the information transmitting device 10 and the information receiving device 20 configured as above will be described.

[0163] Figure 7 This is a flowchart showing the operation of the information transmitting device 10 according to the present embodiment. Figure 8 It is a flowchart showing the specific operation of step S11.

[0164] First, the information transmitting device 10 calculates the acoustic feature quantity for the speech sound obtained by the speech sound obtaining unit 111 a ( S10 ).

[0165] Next, the information transmitting device 10 analyzes the condition information indicating the conditions for calculating the speaker characteristic quantity based on the acquired speech sound (S11). Figure 8 As shown, the information transmitting device 10 analyzes the recording status of the speech sound based on the speech sound obtained by the speech sound obtaining unit 111a (S111). Next, the information transmitting device 10 sets the recording status analyzed in step S111 as condition information (S112).

[0166] Next, the information transmitting device 10 uses DNN to calculate the speaker characteristic quantity (S12) of the speaker of the speech sound based on the acoustic characteristic quantity calculated in step S10. In addition, the processing of step S11 and step S12 is not limited to being processed in this order, and can be processed in the reverse order or in parallel.

[0167] Next, the information sending device 10 sends the speaker characteristic value calculated in step S12 and the condition information analyzed in step S11 to the information receiving device 20. The speaker characteristic value and condition information are information used by the information receiving device 20 that performs speaker recognition processing on the speech sound in order to identify the speaker of the speech sound (S13).

[0168] Fig. 9 This is a flowchart showing the operation of the information receiving device 20 according to the present embodiment.

[0169] First, the information receiving device 20 obtains condition information included in the information transmitted by the information transmitting device 10 (S20).

[0170] Next, the information receiving device 20 obtains the speaker characteristic value included in the information transmitted by the information transmitting device 10 (S21). The processing of step S20 and step S21 is not limited to be performed in this order, and can be performed in reverse order or in parallel.

[0171] Next, the information receiving device 20 calculates the similarity between each of the registered speaker feature values ​​stored in the storage unit 24 and the speaker feature value based on the condition information obtained in step S20 and the speaker feature value obtained in step S21 (S22). As described above, the storage unit 24 stores the unique feature value for each of one or more registered speakers registered in advance, that is, the registered speaker feature value, according to each condition when calculating the registered speaker feature value.

[0172] Next, the information receiving device 20 specifies which of the one or more registered speakers stored in the storage unit 24 the speaker of the speech audio is, based on the similarity calculated in step S22, and outputs the identification (S23).

[0173] Fig.10 is a flowchart showing the specific operation of the information receiving device 20 involved in this embodiment. Fig. 9 The same processes are given the same symbols and their detailed description is omitted here.

[0174] In step S221 after step S21, when the condition information obtained in step S20 includes the recording condition of the speech sound, the information receiving device 20 selects the registered speaker feature value under the condition that meets the recording condition, and the registered speaker feature value is also the registered speaker feature value for each of more than one registered speakers.

[0175] Next, the information receiving device 20 calculates the similarity between the registered speaker feature value for each of the one or more registered speakers selected in step S221 and the speaker feature value obtained in step S21 ( S222 ).

[0176] Next, the information receiving device 20 specifies which of the one or more registered speakers selected in step S221 the speaker of the speech sound is, based on the similarity calculated in step S222, and outputs the identification (S231).

[0177] [1.5 Effects, etc.]

[0178] As described above, according to the present embodiment, the speaker recognition processing performed by the information receiving device 20 can be dispersed and part of it can be performed by the information transmitting device 10. That is, the information transmitting device 10 according to the present embodiment can omit part of the calculation performed by the information receiving device 20 to perform the speaker recognition processing by transmitting the information used by the information receiving device 20 to recognize the speaker of the speech sound.

[0179] Specifically, the information sending device 10 calculates the acoustic feature quantity for the speech sound, and performs processing to calculate the speaker feature quantity based on the calculated acoustic feature quantity, and includes the result obtained by the processing in the above-mentioned information, and sends it to the information receiving device 20. Accordingly, the information receiving device 20 can not only omit the calculation of the acoustic feature quantity for the speech sound of the speaker, but also omit the calculation of the speaker feature quantity. In other words, the information receiving device 20 can omit a part of the calculation performed for performing the speaker recognition processing, so that the calculation load for performing the speaker recognition processing can be reduced.

[0180] Therefore, according to the present embodiment, since the calculation load of the speaker identification process can be distributed, the calculation load for the information receiving device 20 to execute the speaker identification process can be suppressed.

[0181] Furthermore, the information sending device 10 involved in this embodiment includes the recording status of the analyzed speech sound as condition information in the above-mentioned information and sends it to the information receiving device 20. Accordingly, the information receiving device 20 can calculate the similarity between the speaker feature and the registered speaker feature based on the selected registered speaker feature corresponding to the recording status. In other words, the information receiving device 20 does not need to calculate the similarity between the registered speaker feature under all conditions and the registered speaker feature, so the calculation load can be reduced. Therefore, through this embodiment, the calculation load of the speaker recognition processing can be dispersed to suppress the calculation load used by the information receiving device 20 to perform the speaker recognition processing.

[0182] (Variant 1)

[0183] In the above-mentioned embodiment, the case where the information sending device uses DNN to calculate the speaker characteristic of the speech sound and includes it in the information to be sent is described, but it is not limited to this. The information sending device may also use a part of the DNN to calculate the first speaker characteristic equivalent to a part of the speaker characteristic of the speech sound, and include it in the information to be sent. In this case, the information receiving device may also use other parts of the DNN to calculate the speaker characteristic of the speech sound based on the first speaker characteristic. This case is described below as Modification Example 1. In addition, the following description focuses on the differences from the above-mentioned embodiment.

[0184] [2.1 Specific Configuration of Information Transmitting Device 10A and Information Receiving Device 20A]

[0185] Fig.11 1 is a block diagram showing an example of a specific configuration of an information transmitting device 10A and an information receiving device 20A according to Modification 1. Figure 3 The same elements are given the same reference numerals, and detailed description thereof is omitted here.

[0186] [2.1.1 Information Transmitting Device 10A]

[0187] Fig.11 The information transmitting device 10A shown is relative to Figure 3 The information transmitting device 10 shown in the figure has different configurations of the analyzing unit 12A and the speaker characteristic amount calculating unit 14A. The following description will focus on the differences from the above-mentioned embodiment.

[0188] [2.1.1.1 Analysis Unit 12A]

[0189] Fig.12 1 is a diagram showing an example of a detailed configuration of the analysis unit 12A according to the first modification. Figure 4 The same elements are given the same reference numerals, and detailed description thereof is omitted here.

[0190] The analyzing unit 12A analyzes the condition information indicating the conditions for calculating the speaker feature quantity based on the speech sound obtained from the speech sound obtaining unit 111a. Fig.12 As shown, the analyzing unit 12A includes a situation analyzing unit 121A and a condition information setting unit 122A.

[0191] The situation analysis unit 121A analyzes the recorded situation of the speech sound based on the speech sound obtained from the speech sound acquisition unit 111a. The situation analysis unit 121A also analyzes the speech time of the speech sound obtained from the speech sound acquisition unit 111a.

[0192] The condition information setting unit 122A sets the recording status analyzed by the status analysis unit 121A as the condition information. The condition information setting unit 122A further sets the load control condition as the condition information based on the speech time analyzed by the status analysis unit 121A. The load control condition is to use the DNN used by the speaker feature calculation unit 14A, that is, a part of the DNN, as the first DNN. Here, if the DNN is Figure 6 As shown, it is composed of the 1st layer to the last layer, then the 1st DNN is a part of the DNN composed of the 1st layer to the nth layer (n is a positive integer) in the DNN.

[0193] Thus, the condition information setting unit 122A sets the speech sound recording status and the load control condition as the condition information. In addition, the condition information setting unit 122A sets the speech sound recording status and the load control condition as the condition information by including the speech sound recording status and the load control condition in the condition information.

[0194] [2.1.1.2 Speaker Feature Calculation Unit 14A]

[0195] The speaker feature calculation unit 14A is a functional unit for calculating a speaker feature from an acoustic feature using a DNN.

[0196] Fig.13 This is a diagram showing an example of a detailed configuration of the speaker feature value calculation unit 14A according to the first modification.

[0197] like Fig.13As shown, the speaker feature quantity calculation unit 14A includes a calculation condition control unit 141 and a first speaker feature quantity calculation unit 142 .

[0198] The calculation condition control unit 141 obtains the condition information set by the condition information setting unit 122A. Here, the condition information includes the recording status of the speech sound and the load control condition. The calculation condition control unit 141 causes the first speaker feature quantity calculation unit 142 to perform feature quantity calculation processing using the first DNN according to the load control condition included in the obtained condition information.

[0199] The first speaker feature quantity calculation unit 142 uses the first DNN to calculate the first speaker feature quantity obtained in the process of calculating the speaker feature quantity, which is a feature quantity inherent to the speaker of the speech sound, based on the acoustic feature quantity obtained from the acoustic feature quantity calculation unit 13. In addition, the first speaker feature quantity calculation unit 142 can also calculate the first speaker feature quantity obtained in the process of calculating the speaker feature quantity by causing the DNN to calculate the speaker feature quantity based on the obtained acoustic feature quantity and outputting the output from the last layer of the first DNN as the first speaker feature quantity.

[0200] In this way, the speaker feature calculation unit 14A can calculate the first speaker feature corresponding to a part of the speaker feature obtained in the process of calculating the speaker feature based on the obtained acoustic feature using the first DNN as a part of the DNN according to the load control condition. Then, the speaker feature calculation unit 14A sends the calculated first speaker feature as the speaker feature to the information transmission unit 15.

[0201] Accordingly, the information sending device 10A causes the information sending unit 15 to send the condition information including the recording status of the speech sound and the load control condition obtained from the analyzing unit 12A, and the first speaker feature value which is the speaker feature value calculated by the speaker feature value calculating unit 14A to the information receiving device 20A.

[0202] [2.1.2 Information receiving device 20A]

[0203] The information receiving device 20A receives information from the information transmitting device 10A, and performs speaker recognition processing using the received information.

[0204] In this variation, if Fig.11 As shown, the information receiving device 20A includes an information receiving unit 201, a speaker identification device 22A, and a speaker display unit 28. That is, Fig.11 The information receiving device 20A shown is relative to Figure 3The configuration of the speaker identification device 22A is different from that of the information receiving device 20 shown in the figure. The following description will focus on the differences from the above-mentioned embodiment.

[0205] [2.1.2.1 Speaker Identification Device 22A]

[0206] The speaker identification device 22A performs speaker identification processing using the information obtained from the information receiving unit 201 , and transmits the identified speaker to the speaker display unit 28 .

[0207] Fig.11 The speaker identification device 22A shown is relative to Figure 3 The speaker identification device 22 shown in the figure is different in the configuration of the condition information acquisition unit 23A and the speaker characteristic quantity acquisition unit 26A.

[0208] [2.1.2.1.1 Condition Information Acquisition Unit 23A]

[0209] The condition information acquisition unit 23A acquires the condition information included in the information sent by the information sending device 10A. In this modification, the condition information acquisition unit 23A acquires the condition information included in the information sent by the information sending device 10A from the information receiving unit 201. The condition information acquired by the condition information acquisition unit 23A includes not only the recording status of the speech sound but also the load control condition, which indicates that the first DNN in the DNN is used instead of the DNN, and the first DNN is a part of the DNN composed of the first layer to the nth layer (n is a positive integer) in the DNN.

[0210] The condition information acquisition unit 23A transmits the recording status included in the acquired condition information and the load control condition among the load control conditions to the speaker feature quantity acquisition unit 26A, and transmits the recording status to the similarity calculation unit 25 .

[0211] [2.1.2.1.2 Speaker Feature Quantity Acquisition Unit 26A]

[0212] As the speaker feature included in the information sent by the information sending device 10A, the speaker feature obtaining unit 26A obtains the first speaker feature obtained in the process of calculating the speaker feature, and calculates the second speaker feature as the speaker feature based on the obtained first speaker feature.

[0213] Fig.14 This is a diagram showing an example of a detailed configuration of the speaker characteristic amount obtaining unit 26A according to the first modification. Fig.15 This is a diagram showing the configuration of a DNN according to Modification 1 and a component used in calculation of a first speaker feature value and a second speaker feature value.

[0214] like Fig.14 As shown, the speaker feature value obtaining unit 26A includes a first speaker feature value obtaining unit 261 , a calculation condition control unit 262 , and a second speaker feature value calculating unit 263 .

[0215] As the speaker feature included in the information transmitted by the information transmitting device 10A, the first speaker feature obtaining unit 261 obtains the first speaker feature obtained in the process of calculating the speaker feature. Here, as described above, the first speaker feature is the first speaker feature obtained in the process of calculating the speaker feature, and is also the first speaker feature calculated using the first DNN. Here, if the DNN is Figure 6 The structure shown in the figure, then the first DNN is equivalent to Fig.15 The portion of the DNN shown is composed of the 1st to nth layers (n is a positive integer) in the DNN.

[0216] The calculation condition control unit 262 obtains the load control condition transmitted by the condition information obtaining unit 23A. The calculation condition control unit 262 causes the second speaker feature quantity calculation unit 263 to perform feature quantity calculation processing using the second DNN according to the obtained load control condition. Here, the second DNN corresponds to Fig.15 The other parts of the DNN shown, the 2nd DNN is a part of the DNN consisting of the n+1th layer to the last layer in the DNN.

[0217] The second speaker feature calculation unit 263 uses the second DNN to calculate the second speaker feature as the speaker feature based on the obtained first speaker feature. In addition, the second speaker feature calculation unit 263 inputs the first speaker feature to the next layer of the last layer of the first DNN in the following DNN, which is the same DNN as the DNN used by the speaker feature calculation unit 14A of the information transmission device 10A. According to this, the second speaker feature calculation unit 263 can make the second DNN calculate the second speaker feature.

[0218] The second speaker feature value calculation unit 263 outputs the calculated second speaker feature value to the similarity calculation unit 25 (more specifically, the similarity calculation processing unit 252 ).

[0219] In this way, the speaker feature acquisition unit 26A can calculate the second speaker feature as the speaker feature for the speech sound based on the obtained first speaker feature using the second DNN which is another part of the DNN according to the transmitted load control condition.

[0220] [2.2 Operation of the Information Transmitting Device 10A and the Information Receiving Device 20A]

[0221] Next, the operations of the information transmitting device 10A and the information receiving device 20A configured as above will be described.

[0222] Fig.16A as well as Fig. 16B This is a flowchart showing the specific operation of the information transmitting device 10A according to the first modification. Fig.16A Shows Figure 7 The specific operation related to the modification example 1 in step S11 is shown. Fig. 16B Shows Figure 7 The specific operation related to the modification example 1 in step S12 is shown.

[0223] exist Figure 7 In step S11 shown in FIG. Fig.16A As shown, the information sending device 10A first analyzes the recording status of the speech sound based on the acquired speech sound, and analyzes the speech time of the speech sound (S111A). Next, the information sending device 10A sets the recording status analyzed in step S111A as condition information (S112A). Furthermore, the information sending device 10A sets the load control condition as condition information based on the analyzed speech time. The load control condition refers to using the DNN used for speaker feature quantity calculation processing as the first DNN, which is a part of the DNN composed of the first layer to the nth layer (n is a positive integer) in the DNN (S113A). In addition, the processing of step S112A and step S113A is not limited to being processed in this order, and can be processed in the reverse order or in parallel.

[0224] exist Figure 7 In step S12 shown in FIG. Fig. 16B As shown, the information transmitting device 10A uses the first DNN according to the load control condition to calculate the first speaker feature obtained in the process of calculating the speaker feature based on the acoustic feature calculated in step S10 as the speaker feature (S121A). Fig.16A The load control condition set in step S113A shown is transmitted to the information transmitting device 10A, and the information transmitting device 10A calculates the first speaker feature value according to the transmitted load control condition.

[0225] Fig.17 is a flowchart showing the specific operation of the information receiving device 20A according to the modification example 1. Fig. 9 as well as Fig.10 The same treatments are given the same symbols.

[0226] First, the information receiving device 20A obtains condition information included in the information transmitted by the information transmitting device 10A (S20).

[0227] Next, the information receiving device 20A obtains the first speaker feature as the speaker feature included in the information transmitted by the information transmitting device 10A (S211A). As described above, the first speaker feature is a feature obtained in the process of calculating the speaker feature, which is a feature inherent to the speaker of the speech sound, and is also a feature calculated based on the obtained acoustic feature using the first DNN as a part of the DNN. The first DNN is, for example, a part of the DNN consisting of the first layer to the nth layer (n is a positive integer) in the DNN.

[0228] Next, the information receiving device 20A calculates the second speaker feature quantity, which is a feature quantity inherent to the speaker of the speech sound, based on the load control condition included in the condition information obtained in step S20, using the second DNN, which is another part of the DNN, based on the obtained first speaker feature quantity (S212A). Here, the second DNN corresponds to another part of the DNN, and the second DNN is, for example, a part of the DNN consisting of the n+1th layer to the last layer in the DNN.

[0229] Next, the information receiving device 20A selects a registered speaker feature value that satisfies the recording condition of the speech sound included in the condition information obtained in step S20, and the registered speaker feature value is also a registered speaker feature value for each of more than one registered speaker (S221A). In addition, the processing of step S211A, step S211B, and step S221A is not limited to being processed in this order, and can be processed in the reverse order or in parallel.

[0230] Next, the information receiving device 20A calculates the similarity between the registered speaker feature value for each of the one or more registered speakers selected in step S221A and the second speaker feature value calculated in step S212A ( S222A).

[0231] Next, the information receiving device 20A specifies which of the one or more registered speakers selected in step S221A the speaker of the speech audio is, based on the similarity calculated in step S222A, and outputs the identification ( S223A).

[0232] [2.3 Effects, etc.]

[0233] As described above, through Modification 1, the speaker recognition processing performed by the information receiving device 20A can be dispersed and part of it can be performed by the information transmitting device 10A. That is, the information transmitting device 10A involved in Modification 1 can omit part of the calculation performed by the information receiving device 20A to perform the speaker recognition processing by transmitting the information used by the information receiving device 20A to recognize the speaker of the speech sound.

[0234] Specifically, the information sending device 10A performs the following processing, namely, calculating the acoustic feature quantity for the speech sound, and calculating the first speaker feature quantity corresponding to a part of the speaker feature quantity based on the calculated acoustic feature quantity, and the first speaker feature quantity is also a feature quantity obtained in the process of calculating the speaker feature quantity. Then, the information sending device 10A includes the calculated first speaker feature quantity and the load control condition of the first DNN used to calculate the first speaker feature quantity in the above-mentioned information, and sends it to the receiving device 20A. Accordingly, the information receiving device 20A can not only omit the calculation of the acoustic feature quantity for the speech sound of the speaker, but also omit a part of the calculation of the speaker feature quantity by using the first speaker feature quantity. That is, the information receiving device 20A can omit a part of the calculation performed to perform the speaker recognition processing, and thus can reduce the calculation load for performing the speaker recognition processing.

[0235] Therefore, according to Modification 1, since the calculation load of the speaker identification process can be distributed, the calculation load for the information receiving device 20A to execute the speaker identification process can be suppressed.

[0236] Furthermore, the information sending device 10A involved in the modification example 1 includes the collection status of the analyzed speech sound as condition information in the above-mentioned information and sends it to the information receiving device 20A. Accordingly, the information receiving device 20A can calculate the similarity between the speaker feature quantity and the registered speaker feature quantity based on the selected registered speaker feature quantity corresponding to the collection status. In other words, the information receiving device 20A does not need to calculate the similarity between the registered speaker feature quantity and the registered speaker feature quantity under all conditions, so the calculation load can be reduced. Therefore, through the modification example 1, the calculation load of the speaker recognition processing can be dispersed to suppress the calculation load for the information receiving device 20A to perform the speaker recognition processing.

[0237] (Variant 2)

[0238] In the above-mentioned modification example 1, the following case is described, that is, the similarity calculation unit performs similarity calculation processing based on the registered speaker feature quantity under the condition that meets the collection status included in the condition information sent from the information sending device side among the registered speaker feature quantities of multiple conditions stored in the storage unit, but the present invention is not limited to this. The similarity calculation unit may also perform similarity calculation processing on all the registered speaker feature quantities of multiple conditions stored in the storage unit. This case is described as modification example 2. In addition, the following description focuses on the differences from the above-mentioned embodiment and modification example 1.

[0239] [3.1 Specific Configuration of Information Transmitting Device 10B and Information Receiving Device 20B]

[0240] Fig.18 1 is a block diagram showing an example of a specific configuration of an information transmitting device 10B and an information receiving device 20B according to Modification 2. Figure 3 as well as Fig.11 The same elements are given the same reference numerals, and detailed description thereof is omitted here.

[0241] [3.1.1 Information Transmitting Device 10B]

[0242] Fig.18 The information transmitting device 10B shown is relative to Fig.11 The information transmitting device 10A shown in the figure has different configurations of the analyzing unit 12B and the speaker characteristic amount calculating unit 14B. The following description will focus on the differences from the modification example 1.

[0243] [3.1.1.1 Analysis unit 12B]

[0244] Fig.19 1 is a diagram showing an example of a detailed configuration of the analysis unit 12B according to the second modification. Figure 4 as well as Fig.12 The same elements are given the same reference numerals, and detailed description thereof is omitted here.

[0245] The analyzing unit 12B analyzes the condition information indicating the conditions for calculating the speaker feature quantity based on the speech sound obtained from the speech sound obtaining unit 111a. Fig.19 As shown, the analyzing unit 12B includes a situation analyzing unit 121B and a condition information setting unit 122B.

[0246] The situation analysis unit 121B analyzes the speech time of the speech sound obtained from the speech sound acquisition unit 111a. In addition, in this modification, unlike the above-mentioned embodiment and modification 1, the situation analysis unit 121B may not analyze the recording situation of the speech sound.

[0247] The condition information setting unit 122B sets the load control condition as the condition information based on the speech time analyzed by the situation analysis unit 121B, and the load control condition is to use the DNN used by the speaker feature calculation unit 14B as the first DNN, and the first DNN is a part of the DNN. As described in the description of the modification example 1, the first DNN is a part of the DNN composed of the first layer to the nth layer (n is a positive integer) in the DNN.

[0248] In this way, the condition information setting unit 122B sets the load control condition as the condition information. In addition, the condition information setting unit 122B sets the load control condition as the condition information by including the load control condition in the condition information.

[0249] [3.1.1.2 Speaker Feature Calculation Unit 14B]

[0250] The speaker feature calculation unit 14B is a functional unit for calculating the speaker feature from the acoustic feature using DNN.

[0251] Fig. 20 This is a diagram showing an example of a detailed configuration of the speaker feature value calculation unit 14B according to the second modification.

[0252] like Fig. 20 As shown, the speaker feature value calculation unit 14B includes a calculation condition control unit 141B and a first speaker feature value calculation unit 142. Since the first speaker feature value calculation unit 142 has been described in Modification 1, its description is omitted.

[0253] The calculation condition control unit 141B obtains the condition information set by the condition information setting unit 122B. Here, the condition information includes a load control condition. The calculation condition control unit 141B causes the first speaker feature quantity calculation unit 142 to perform feature quantity calculation processing using the first DNN according to the load control condition included in the obtained condition information.

[0254] In this way, the speaker feature calculation unit 14B can calculate the first speaker feature obtained in the process of calculating the speaker feature based on the obtained acoustic feature using the first DNN as a part of the DNN according to the load control condition. Then, the speaker feature calculation unit 14B sends the calculated first speaker feature as the speaker feature to the information sending unit 15.

[0255] In response to this, the information transmitting device 10B causes the information transmitting unit 15 to transmit the condition information including the load control condition obtained from the analyzing unit 12B and the first speaker feature value which is the speaker feature value calculated by the speaker feature value calculating unit 14B to the information receiving device 20B.

[0256] [3.1.2 Information receiving device 20B]

[0257] The information receiving device 20B receives information from the information transmitting device 10B, and performs speaker recognition processing using the received information.

[0258] In this variation, if Fig.18 As shown, the information receiving device 20B includes an information receiving unit 201, a speaker identification device 22B, and a speaker display unit 28. That is, Fig.18 The information receiving device 20B shown is relative to Fig.11 The configuration of the speaker identification device 22B is different from that of the information receiving device 20A shown in the figure. The following description will focus on the differences from the first modification.

[0259] [3.1.2.1 Speaker Identification Device 22B]

[0260] The speaker identification device 22B performs speaker identification processing using the information obtained from the information receiving unit 201 , and transmits the identified speaker to the speaker display unit 28 .

[0261] Fig.18 The speaker identification device 22B shown is relative to Fig.11 The speaker identification device 22A shown in FIG. 1 is different in the configurations of the condition information acquisition unit 23B and the similarity calculation unit 25B.

[0262] [3.1.2.1.1 Condition Information Acquisition Unit 23B]

[0263] The condition information obtaining unit 23B obtains the condition information included in the information sent by the information sending device 10B. In this variant, the condition information obtaining unit 23B obtains the condition information included in the information sent by the information sending device 10B from the information receiving unit 201. The condition information obtained by the condition information obtaining unit 23B includes a load control condition, which means that the first DNN is used instead of the DNN, and the first DNN is a part of the DNN composed of the first layer to the nth layer (n is a positive integer) in the DNN. In addition, in this variant, unlike variant 1, the condition information obtained by the condition information obtaining unit 23B does not include the recording status of the speech sound.

[0264] The condition information obtaining section 23B passes the load control condition included in the obtained condition information to the speaker characteristic amount obtaining section 26A.

[0265] [3.1.2.1.2 Similarity calculation unit 25B]

[0266] The similarity calculation unit 25B calculates the similarity between each of the registration speaker feature values ​​stored in the storage unit 24 and the second speaker feature value based on the second speaker feature value calculated by the speaker feature value acquisition unit 26A using the first speaker feature value.

[0267] In this variation, the similarity calculation unit 25B calculates the similarity between each of all the registered speaker feature values ​​of conditions A, B, C, etc. stored in the storage unit 24 and the second speaker feature value obtained from the speaker feature value acquisition unit 26A.

[0268] [3.2 Effects, etc.]

[0269] As described above, through Modification 2, the speaker recognition processing performed by the information receiving device 20B can be dispersed and part of it can be performed by the information transmitting device 10B. That is, the information transmitting device 10B involved in Modification 2 can omit part of the calculation performed by the information receiving device 20B to perform the speaker recognition processing by transmitting the information used by the information receiving device 20B to recognize the speaker of the speech sound.

[0270] Specifically, the information sending device 10B performs the following processing, namely, calculating the acoustic feature quantity for the speech sound, and calculating the first speaker feature quantity corresponding to a part of the speaker feature quantity based on the calculated acoustic feature quantity, and the first speaker feature quantity is also a feature quantity obtained in the process of calculating the speaker feature quantity. Then, the information sending device 10A sends the calculated first speaker feature quantity and the load control condition of the first DNN used to calculate the first speaker feature quantity as the above-mentioned information to the information receiving device 20B. Accordingly, the information receiving device 20B can not only omit the calculation of the acoustic feature quantity for the speech sound of the speaker, but also omit a part of the calculation of the speaker feature quantity by using the first speaker feature quantity. That is, the information receiving device 20B can omit a part of the calculation performed to perform the speaker recognition processing, and thus can reduce the calculation load for performing the speaker recognition processing.

[0271] Therefore, according to Modification 2, since the calculation load of the speaker identification process can be distributed, the calculation load for the information receiving device 20B to execute the speaker identification process can be suppressed.

[0272] Although the information transmitting device and the information receiving device according to the embodiments and the modified examples have been described above, the present disclosure is not limited to the embodiments.

[0273] For example, in the above-mentioned embodiments and variations, the case where the speaker of the recognition object is a human is described, but the invention is not limited thereto. Even if the speech sound of the speaker of the recognition object is the sound of an animal other than a human, the recognition can be performed in the same manner. Furthermore, even if the speech sound of the speaker of the recognition object is further expanded to the working sound of machinery and equipment, or the normal sound or abnormal sound of machinery, the same description can be made. In addition, in this case, as the acoustic feature quantity, MFCC, etc. is not calculated, but FFT (Fast Fourier Transform) for the sound equivalent to the speech sound can be calculated.

[0274] Furthermore, in the above-mentioned modification, the case where the configuration of the first DNN and the second DNN used when calculating the first speaker feature and the second speaker feature is a fixed configuration is used as an example for explanation, but the present invention is not limited thereto. The number of layers of the first DNN and the second DNN used when calculating the first speaker feature and the second speaker feature may also be changed according to the transmission load of the network 3.

[0275] For example, each processing unit included in the information transmitting device and the information receiving device according to the above-mentioned embodiments and modifications may be implemented as a typical integrated circuit, namely, LSI. These may be individually made into one chip, or part or all of them may be made into one chip.

[0276] In addition, the method of integrated circuitization is not limited to LSI, and can also be realized by a dedicated circuit or a general-purpose processor. It is also possible to use a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connection or setting of the circuit unit inside the LSI after LSI manufacturing.

[0277] In addition, the present disclosure may also be implemented as an information transmitting method and an information receiving method executed by an information transmitting device and an information receiving device, and the like.

[0278] In addition, each component in the above-mentioned embodiments may also be constituted by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a storage medium such as hardware or a semiconductor memory.

[0279] In addition, the division of the functional blocks in the block diagram is an example, and multiple functional blocks can be implemented as one functional block, and one functional block can also be divided into multiple blocks, and a part of the functions can also be transferred to other functional blocks. In addition, the functions of multiple functional blocks with similar functions can also be processed in parallel or in time division by a single hardware or software.

[0280] In addition, the order in which the steps in the flowchart are executed is to specifically illustrate the examples of the present disclosure, and may be an order other than the above. Furthermore, part of the above steps may be executed simultaneously (in parallel) with other steps.

[0281] Although the information sending device and the information receiving device involved in one or more forms are described above based on the embodiments and variations, the present disclosure is not limited to the embodiments and variations. Without departing from the scope of the present disclosure, forms obtained by performing various modifications that can be thought of by those skilled in the art on the present embodiments and variations, and forms formed by combining constituent elements in different embodiments and variations, are all included in the scope of one or more forms.

[0282] The present disclosure can be used for an information transmitting device, an information receiving device, an information transmitting method, a recording medium, and a system, and can be used for a remote conference system such as a telephone conference system, for example.

Claims

1. An information processing method, wherein: Acquire condition information and speaker characteristics of speech sound, Based on the condition information, a plurality of registered speaker feature quantities are obtained, Based on the obtained similarity between each of the plurality of registered speaker feature quantities and the speaker feature quantity, it is determined which of the plurality of registered speakers respectively corresponding to the plurality of registered speaker feature quantities the speaker of the speech sound is.

2. The information processing method according to claim 1, wherein: The condition information is information indicating a recording status of the speech sound or a data attribute of the speech sound.

3. The information processing method according to claim 2, wherein: The recording condition indicates a noise level when recording the speech sound and at least one of the microphones used, The data attribute of the speech sound indicates the gender of the speaker of the speech sound or the language of the speech sound.

4. The information processing method according to claim 1, wherein: Each of the plurality of registered speaker feature quantities is a feature quantity stored in the storage device in correspondence with a certain registered speaker among the plurality of registered speakers and a certain condition among a plurality of conditions, A plurality of registered speaker feature quantities that match the condition information are selected from the plurality of registered speaker feature quantities stored in the storage device.

5. The information processing method according to any one of claims 1 to 3, wherein: The speaker characteristic amount is calculated based on the speech sound, which is a characteristic amount inherent to the speaker of the speech sound, thereby obtaining the speaker characteristic amount, The condition information is obtained by analyzing the speech sound.

6. The information processing method according to any one of claims 1 to 3, wherein: Then, the acoustic feature quantity for the speech sound is calculated. Based on the acoustic feature quantity for the speech sound, the speaker feature quantity is calculated using a deep neural network (DNN), thereby obtaining the speaker feature quantity.

7. An information processing device, wherein: have: processor; as well as Memory, The processor uses the memory to perform: Acquire condition information and speaker characteristics of speech sound, Based on the condition information, a plurality of registered speaker feature quantities are obtained, Based on the obtained similarity between each of the plurality of registered speaker feature quantities and the speaker feature quantity, it is determined which of the plurality of registered speakers respectively corresponding to the plurality of registered speaker feature quantities the speaker of the speech sound is.

8. A computer program product, wherein: The present invention includes a program for causing a computer to execute the information processing method according to any one of claims 1 to 6.

Images