Speaker age estimation device, speaker age estimation method, and program

Abstract

This speaker age estimation device improves accuracy of speaker age estimation based on voices of an utterance by including: a determination unit configured to determine each weight of a first age, which is estimated by a person for a speaker on the basis of voices of an utterance of the speaker, and a second age, which is estimated by a machine for the speaker using a machine learning model, on the basis of one or more of information related to the person, information related to the machine, and information related to the voices; and an output unit configured to output an estimated value of the age of the speaker on the basis of the first age, the second age, and the weights.

Description

Speaker age estimation device, speaker age estimation method and program

[0001] The present invention relates to a speaker age estimation device, a speaker age estimation method, and a program.

[0002] In the past, call centers estimated the age of the speaker (customer) by subjective evaluation of the speaker's voice. In call centers for insurance and other services, the estimated speaker's age is compared with registered customer information and used as auxiliary material for identity verification. The estimated speaker's age is also used as a material for customer analysis in the repair reception department where visitors call.

[0003] However, there is variation in subjective evaluation ability among operators, and different operators may make different age estimates for the same voice (Non-Patent Document 1).

[0004] In addition, bias due to the gender and age of the operator has been pointed out. For example, it has been reported that it is difficult for older operators to accurately estimate the age of young people, especially for voices from younger people, for reasons such as their hearing becoming more difficult to distinguish high-pitched sounds as they age and their opportunities to interact with younger people decreasing (Non-Patent Document 2).

[0005] It has also been reported that the ages of speakers who appear to be young are estimated to be older than they actually are, and that the ages of speakers who appear to be elderly are estimated to be younger than they actually are, and in particular, that the voices of middle-aged female speakers tend to be estimated to be younger than they actually are (Non-Patent Document 2).

[0006] "What does speech communicate? - The speech science of emotion, paralinguistic information, and individuality," Acoustical Society of Japan (ed.), 2014. "Perception, Analysis, and Synthesis of Speaker Age," Susanne Schoetz, Linguistics and Phonetics, 2006.

[0007] In recent years, with the advancement of machine learning technology, the accuracy of automatic age estimation technology from voice using machine learning has improved, and practical application is expected. While previous research has clarified the accuracy and trends of age estimation by humans, the trends of age estimation using machine learning have not been analyzed, and it is not clear what differences there are between the results of estimation by humans and those by machines.

[0008] The present invention has been made in view of the above points, and has as its object to improve the accuracy of estimating a speaker's age based on the voice of an utterance.

[0009] In order to solve the above problem, the speaker age estimation device has a determination unit configured to determine weights for a first age estimated by a human based on the speaker's speech and a second age estimated by a machine using a machine learning model, based on one or more of information related to the person, information related to the machine, and information related to the speech, and an output unit configured to output an estimate of the speaker's age based on the first age, the second age, and the weights.

[0010] The accuracy of estimating the speaker's age based on the voice of the utterance can be improved.

[0011] 1 is a diagram illustrating an example of a hardware configuration of a speaker age estimation device 10 according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a functional configuration of the speaker age estimation device 10 according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a configuration of an age determination unit 13. FIG. 4 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 in a first pattern. FIG. 5 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 in a second pattern. FIG. 6 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 in a third pattern. FIG. 7 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 in a fourth pattern. FIG. 8 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 in a fifth pattern. FIG. 9 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 in a sixth pattern. FIG. 10 is a flowchart illustrating an example of a processing procedure executed by the determination unit 131 when combining the first to sixth patterns. FIG. 11 is a diagram illustrating experimental results (MAE / ρ) for age estimation using machine learning. FIG. 12 is a diagram illustrating the effect of test segment length on MAE for a call center dataset. FIG. 13 is a diagram illustrating experimental results of PA / EA for a call center dataset. 10A and 10B show C-MAE for female and male speakers by CA class, respectively;

[0012] In this embodiment, a technology is disclosed that focuses on the differences in behavior (tendencies) between humans and machine learning models when estimating a speaker's age from voice, and automatically determines which estimated age, human or machine, should ultimately be adopted, or which estimated age should be given a relatively greater weight.

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] 1 is a diagram showing an example of the hardware configuration of a speaker age estimation device 10 according to an embodiment of the present invention. The speaker age estimation device 10 in FIG. 1 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, and an interface device 105, which are all interconnected by a bus B.

[0015] A program that realizes the processing in the speaker age estimation device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0016] When an instruction to start the program is received, the memory device 103 reads and stores the program from the auxiliary storage device 102. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the speaker age estimation device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0017] 2 is a diagram showing an example of the functional configuration of a speaker age estimation device 10 according to an embodiment of the present invention. In FIG. 2, the speaker age estimation device 10 includes an automatic estimation unit 11, a manual estimation unit 12, an age determination unit 13, and an analysis unit 14. Each of these units is realized by a process executed by a processor 104 of one or more programs installed in the speaker age estimation device 10.

[0018] The automatic estimation unit 11 receives input of speech (speech), extracts acoustic features from the speech, and inputs the acoustic features into a trained machine learning model to automatically estimate an age. The age estimation performed by the automatic estimation unit 11 is hereinafter referred to as "automatic age estimation."

[0019] The manual estimation unit 12 outputs an age estimate made by a person (for example, a call center operator) for the speaker of the same voice as the voice input by the automatic estimation unit 11 .

[0020] The age determination unit 13 inputs the age (estimated value) output from the automatic estimation unit 11 and the age (estimated value) output from the manual estimation unit 12, determines the age to be finally adopted (hereinafter referred to as the "final estimated age"), and outputs the final estimated age.

[0021] The analysis unit 14 performs an analysis process using the final estimated age. For example, when the speaker age estimation device 10 is used in a call center for insurance business or the like, the analysis unit 14 may compare the final estimated age with an age registered in advance for the speaker whose age is to be estimated and output the comparison result, or may output the final estimated age so that it can be visually confirmed by an operator, or may register the final estimated age in a predetermined database as part of customer information. Alternatively, when the speaker age estimation device 10 is used in a repair reception department or the like, the analysis unit 14 may use a set of final estimated ages as one material for customer analysis. Alternatively, the analysis unit 14 may store the final estimated age in association with the original voice.

[0022] Although the present embodiment is described as being limited to age estimation from voice, any other data such as a facial image, an MRI image, etc. may be used for age estimation. Furthermore, although the present embodiment shows an example in which the person who estimates the age is an operator, the person who estimates the age may be changed as appropriate to suit the usage scenario of the present embodiment.

[0023] The automatic estimation unit 11, the manual estimation unit 12, and the age determination unit 13 will be described in more detail below.

[0024] [Automatic Estimation Unit 11] The automatic estimation unit 11 estimates the speaker age from speech using a feature extractor that extracts acoustic features to be used for automatic estimation of the speaker age using machine learning, and a machine learning model that has learned the correspondence between the acoustic features and age (hereinafter referred to as the "age estimation model"). The model includes a model structure and learned parameters.

[0025] The feature extractor and the age estimation model may be realized using any known technology. For example, the feature extractor may extract time-series acoustic features such as mel-frequency cepstrum coefficients (MFCC) or a log mel filterbank. Alternatively, a trained speaker expression vector extractor may be used as the feature extractor. In this case, the feature extractor extracts and outputs a speaker expression vector. The trained speaker expression vector extractor refers to a model that has trained the correspondence between speech and speakers. The speaker expression vector refers to a numerical vector that identifies a speaker.

[0026] The age estimation model may have a model structure according to the input data. For example, if the age estimation model receives time-series acoustic features as input, a model structure capable of handling time-series acoustic features, such as a recurrent neural network (RNN), a time delay neural network (TDNN), or a transformer, may be used. If the age estimation model receives speaker vectors as input, a model structure such as a fully connected layer, a support vector machine (SVM), or a support vector regression (SVR) may be used.

[0027] The age output (estimation result) of the automatic estimation unit 11 may be an age class discretized at any granularity (age range) such as "child," "young person," "adult," or "elderly person," or may be an age indicated by a numerical value in one-year increments. In this embodiment, as an example, the age estimation model automatically estimates the speaker's age in one-year increments, similar to Published Papers 1 to 3 by the inventor of the present application (however, Published Papers 2 and 3 are scheduled to be published after the filing of this application). Note that the literature information of the published papers is listed at the end.

[0028] The age estimation model may be multi-task trained to recognize and output factors other than speaker age (e.g., gender, speaker vector, etc.) The age estimation model may be trained using any known technology, publicly available data, independently collected data, etc.

[0029] Hereinafter, a speech S of a certain utterance by a speaker i is ｉ The acoustic feature x ｉ The speaker age automatically estimated by the machine learning model (age estimation model) is expressed as the automatically estimated age y^ ｉ ｍ Here, y^ corresponds to the symbol ^ placed above y in the drawings.

[0030] [Manual Estimation Unit 12] The manual estimation unit 12 receives input from an operator or the like about the speaker's age estimated from the speech, and outputs the age. The operator may simultaneously estimate not only the speaker's age but also the gender, personality, and the like from the speech. In this embodiment, the operator simultaneously estimates the speaker's gender (male / female), as in Published Papers 1 to 3. Therefore, the manual estimation unit 12 also receives input of the speaker's gender from the operator and outputs the gender.

[0031] Below, the acoustic feature x ｉ The speaker age estimated by the operator o_j is calculated as the manually estimated age y^ ｉ ｏ＿ｊ and the estimated speaker's gender is denoted as g^ ｉ ｏ＿ｊ ∈{Male, Female}, where o_j corresponds to the symbol in the drawings where j is a subscript of o, and g^ corresponds to the symbol in the drawings where g has ^ above it.

[0032] Since the accuracy of gender estimation is very high both by humans and by machines, g^ ｉ ｏ＿ｊ may be replaced by an automatic machine estimation result.

[0033] [Age Determination Unit 13] The age determination unit 13 determines the automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Based on the above, the final estimate of the speaker's age (hereinafter referred to as the "final estimated age y^") is ｉ ") is determined and output.

[0034] The age determination unit 13 determines the final estimated age as y^ based on only one of the age estimation results by humans and the machine learning based on the tendency of the speaker age estimation by humans and the tendency of the automatic speaker age estimation by machine learning. ｉ Or, the weighted sum using arbitrary weights is used to estimate the final age as y^ ｉ These trends may be those that have been made clear through past research, such as Non-Patent Document 2, or may be those based on published papers 1 to 3.

[0035] 3 is a diagram showing an example of the configuration of the age determining unit 13. In FIG. 3, the age determining unit 13 includes a determining unit 131 and an output unit 132.

[0036] The determination unit 131 determines the manually estimated age y^ ｉ ｏ＿ｊ (automatic age estimation y^) ｉ ｍ and automatically estimating the age y^ based on at least one of machine-related information (which estimates the age y^) and speech-related information (which is used for age estimation). ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Regarding the final estimated age y^ ｉ The range of each weight is 0 to 1, and the sum of the two weights is 1.0. If one of the weights is 1 (in this case, the other weight is 0), the determining unit 131 essentially sets the weight of that weight as the final estimated age y^ ｉ In the following, the automatically estimated age y^ ｉ ｍ The weight for w ｉ ｍ and the manually estimated age y^ ｉ ｏ＿ｊ The weight for w ｉ ｏ＿ｊ In addition, if we do not distinguish between the weights, we write ｉ The automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊIf only one of the two weights is not selected, the sum of the two weights does not have to be 1.0. In this case, arbitrary weights are assigned, and the weighted average is the final estimated age y^ ｉ It may be said that.

[0037] The information related to a person is one or more manually estimated ages y^ ｉ ｏ＿ｊ , estimated gender g^ ｉ ｏ＿ｊ , information indicating the accuracy of age estimation based on the voice of a person (operator o_j) evaluated by a predetermined method (hereinafter referred to as "manual estimation accuracy"); ｉ ｏ＿ｊ a value indicating the degree of confidence of a person (operator o_j) in the estimated gender g^ ｉ ｏ＿ｊ It is a value indicating the degree of confidence of a person (operator o_j) in the

[0038] The information about the machine is, for example, the automatically estimated age y^ ｉ ｍ , information indicating the accuracy of age estimation based on the voice evaluated by a predetermined method for the age estimation model (hereinafter referred to as "model estimation accuracy"); ｉ ｍ The information includes information based on the probability distribution of the gender of the speaker, and a value indicating the degree of confidence that the machine has in its estimation of the speaker's gender.

[0039] The information related to the voice is, for example, the voice S ｉ The acoustic features and speech S ｉ The acoustic feature may be, for example, a jitter ji, which is one of the acoustic features indicating hoarseness. ｉ is.

[0040] The output unit 132 outputs the determination result (w ｉ ｍ and w ｉ ｏ＿ｊ Based on the values ​​of ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ The weighted sum of the above is calculated, and the result is used as the final estimated age y^ｉ The final estimated age y^ is output. ｉ The calculation formula is as follows: ｉ =y^ ｉ ｍ ×w ｉ ｍ +y^ ｉ ｏ＿ｊ ×w ｉ ｏ＿ｊ w ｉ ｍ and w ｉ ｏ＿ｊ If one of them is 1 (in this case, the weight of the other is 0), the output unit 132 essentially outputs the age related to that one as the final estimated age y^ ｉ Therefore, in this case, the output unit 132 may output the one age without calculating the weighted sum.

[0041] Hereinafter, a number of patterns of processing executed by the determining unit 131 and the output unit 132 will be described.

[0042] [First Pattern] The determination unit 131 determines the estimated gender g^ ｉ ｏ＿ｊ and the estimated age of the human being y^ ｉ ｏ＿ｊ is used as input information, and the estimated gender g^ ｉ ｏ＿ｊ Depending on the estimated age y^ ｉ ｏ＿ｊ Applying different thresholds to the weights w ｉ Determine the following.

[0043] FIG. 4 is a flowchart illustrating an example of a processing procedure executed by the determining unit 131 in the first pattern.

[0044] In step S101, the determination unit 131 determines the estimated gender g^ ｉ ｏ＿ｊ Determine whether the person is "Male" or not. ｉ ｏ＿ｊ If the user is "Male" (Yes in S101), the determination unit 131 ｉ ｏ＿ｊ To determine that the age is 1 (i.e., the manually estimated age y^ ｉ ｏ＿ｊ(To recruit) estimated age y^ ｉ ｏ＿ｊ The threshold range for the estimated gender g^ is set to 35 to 45 years old (S102). In FIG. 4, th1 is the lower limit threshold of the range, and th2 is the upper limit threshold of the range. ｉ ｏ＿ｊ If the person is a woman (No in S101), the determination unit 131 determines the range of the threshold value as 35 to 65 years old (S103). ｉ ｏ＿ｊ But the estimated gender is g^ ｉ ｏ＿ｊ If it is within the threshold determined according to (Yes in S104), ｉ ｍ is set to 0, and w ｉ ｏ＿ｊ On the other hand, the determination unit 131 sets the manually estimated age y^ to 1 (S105). ｉ ｏ＿ｊ But the estimated gender is g^ ｉ ｏ＿ｊ If it is outside the range of the threshold determined according to (No in S104), ｉ ｍ Let w be 1. ｉ ｏ＿ｊ is set to 0 (S106).

[0045] The output unit 132 outputs the automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Of these, the weight w ｉ The one with 1 is the final estimated age y^ ｉ Output as

[0046] In the experiments in published papers 1 and 2, it was observed that operators tended to estimate female speakers in their late 40s to late 60s as younger than their actual age, in their late 30s to late 40s. Furthermore, it was also observed that operators tended to estimate male speakers in their late 30s to late 40s as being in their late 20s. On the other hand, this tendency was not observed in the machine learning model. The first pattern is an embodiment based on these findings.

[0047] [Second Pattern] The determination unit 131 receives the model estimation accuracy and the manual estimation accuracy, and determines the accuracy of each w based on the model estimation accuracy and the manual estimation accuracy. ｉ The evaluation scores obtained by conducting an arbitrary evaluation test on the age estimation model and the operator (by evaluating the age estimation accuracy for each of the age estimation model and the operator using a predetermined method) may be used as the model estimation accuracy or the manual estimation accuracy. An example of the evaluation score is the accuracy rate (the probability that the age is correctly estimated from multiple voices used in the evaluation test). Hereinafter, the evaluation score of the operator o_j is referred to as score ｏ＿ｊ and the evaluation score of the age estimation model is denoted as score ｍ It is written as follows.

[0048] FIG. 5 is a flowchart illustrating an example of a processing procedure executed by the determining unit 131 in the second pattern.

[0049] In step S201, the determination unit 131 determines the score ｏ＿ｊ and score ｍ The more the score satisfies the specified conditions, the ｏ＿ｊ Big score ｍ If the condition is satisfied (Yes in S201), the determination unit 131 determines whether or not the condition that w ｉ ｍ is set to 0, and w ｉ ｏ＿ｊ If the condition is not satisfied (No in S201), the determination unit 131 sets w ｉ ｍ Let w be 1. ｉ ｏ＿ｊ is set to 0 (S203). Note that, although Fig. 5 shows an example in which the predetermined condition is, for example, a score difference of 2, this condition is merely an example. For example, since the accuracy of manual estimation for the speech of a specific age group (e.g., elderly people) and the accuracy varies depending on the gender and age of the operator as shown in experimental results in published papers, the determination in step S201 may be made based on such factors alone or in combination.

[0050] The output unit 132 outputs the automatically estimated age y^ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Of these, the weight w ｉ The one with 1 is the final estimated age y^ ｉ Output as

[0051] Alternatively, the determination unit 131 may ｏ＿ｊ and score ｍ Score so that the sum of ｏ＿ｊ and score ｍ When the value of ｏ＿ｊ Wow ｉ ｏ＿ｊ Toshi, score ｍ Wow ｉ ｍ It may also be possible to use the following.

[0052] In this case, the output unit 132 outputs the automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Regarding weight w ｉ The weighted sum based on the final estimated age y^ ｉ Output as

[0053] The contents of Non-Patent Document 1 and the experiments in Papers 1 to 3 reveal that some operators are good at estimating speaker age and others are not, resulting in variations in estimation accuracy. For example, as shown in Table 2 of Paper 1 and the table to be published in Paper 2 (FIG. 13 described below), some operators estimate speaker age with accuracy equal to or higher than that of the machine learning model, while other operators estimate speaker age with even lower accuracy. The second pattern is an embodiment based on this knowledge.

[0054] [Third Pattern] In the third pattern, the determination unit 131 determines whether the audio S ｉ Regarding the jitter ji, which is one of the acoustic features indicating hoarseness, ｉ The value of and automatically estimated age y^ ｉ ｍ and are used as input information, and jitter ji ｉ and the automatically estimated age y^ ｉ ｍ By comparing each weight w with the respective threshold,ｉ Determine the following.

[0055] FIG. 6 is a flowchart illustrating an example of a processing procedure executed by the determining unit 131 in the third pattern.

[0056] In step S301, the determination unit 131 determines the automatically estimated age y^ ｉ ｍ is an elderly person (e.g., 60 years old) or older, and jitter ji ｉ is the threshold th ｊ If the condition is satisfied (Yes in S301), the determination unit 131 determines whether or not the condition that w is less than w ｉ ｍ is set to 0, and w ｉ ｏ＿ｊ If the condition is not satisfied (No in S301), the determination unit 131 sets w ｉ ｍ Let w be 1. ｉ ｏ＿ｊ is set to 0 (S303).

[0057] The output unit 132 outputs the automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Of these, the weight w ｉ The one with 1 is the final estimated age y^ ｉ Output as

[0058] The results of comparative experiments between operators and machine learning models conducted in published papers 1 and 2 showed that operators tended to estimate the voice to be around 35 years old when characteristic voice characteristics such as hoarseness, which are thought to appear with aging, were not detected. On the other hand, no such tendency was observed in the machine learning model. The third pattern is an embodiment based on this finding.

[0059] [Fourth Pattern] In the fourth pattern, the determination unit 131 determines a plurality of manually estimated ages y^ each having a different (non-overlapping) estimation source (operator o_j). ｉ ｏ＿ｊ The determination unit 131 determines the input information based on the manually estimated age y^ ｉ ｏ＿ｊThe number of ｏ ) or more, each weight w ｉ Determine the following.

[0060] FIG. 7 is a flowchart illustrating an example of a processing procedure executed by the determining unit 131 in the fourth pattern.

[0061] In step S401, the determination unit 131 determines whether the input manually estimated age y^ ｉ ｏ＿ｊ The number of ｏ If the condition is satisfied (Yes in S401), the determination unit 131 determines whether or not the condition that w is equal to or greater than w ｉ ｍ is set to 0, and w ｉ ｏ＿ｊ If the condition is not satisfied (S402), the determination unit 131 sets each weight w ｉ For example, as in the first pattern, the manually estimated age y^ ｉ ^o and estimated gender g^ ｉ ^o is taken into account and weight w ｉ may be determined, or the weight w may be determined based on other patterns. ｉ may be determined.

[0062] The output unit 132 is ｉ ｏ＿ｊ If is 1, the input multiple manual estimated age y^ ｉ ｏ＿ｊ The average value of y^ ｉ The calculation result is the final estimated age y^ ｉ It will be output as w ｉ ｍ is 1, the output unit 132 outputs the automatically estimated age y^ ｉ ｍ The final estimated age y^ ｉ Output as

[0063] The results of experiments in published papers have shown that combining the estimation results of multiple operators tends to improve estimation accuracy. The fourth pattern is an embodiment based on this finding.

[0064] [Fifth Pattern] In the fifth pattern, the determination unit 131 determines whether the audio S ｉ The time length is used as input information, and the weights w ｉ Determine the following.

[0065] FIG. 8 is a flowchart illustrating an example of a processing procedure executed by the determining unit 131 in the fifth pattern.

[0066] In step S501, the determination unit 131 ｉ If the condition is satisfied (Yes in S501), the determination unit 131 determines whether the time length of w is less than a threshold value (for example, 30 seconds). ｉ ｍ is set to 0, and w ｉ ｏ＿ｊ If the condition is not met (No in S501), the determination unit 131 sets each weight w ｉ (S503).

[0067] The output unit 132 outputs the automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Of these, the weight w ｉ The one with 1 is the final estimated age y^ ｉ Output as

[0068] It is generally believed that automatic age estimation from voice using machine learning improves accuracy the longer the input voice. This tendency is also confirmed by a study using the data in Published Paper 2, which confirmed that accuracy deteriorates with short voices of about 10 seconds. On the other hand, in the age estimation by an operator in Published Papers 1 and 2, the time required from listening to the voice to answering the gender and age is about 30 seconds per file on average, making estimation possible even from voices of shorter duration than machine learning. The fifth pattern is an embodiment based on this knowledge.

[0069] [Sixth Pattern] In the sixth pattern, the determination unit 131 determines whether the operator o_j himself / herself has a manually estimated age y^ ｉ ｏ＿ｊThe value of the self-confidence score between 0 and 1 (hereinafter referred to as "confidence score c ｉ ｏ＿ｊ ") is used as input information, and the confidence level c ｉ ｏ＿ｊ Based on each weight w ｉ The confidence level c ｉ ｏ＿ｊ The closer to 1, the more confident you are. ｉ ｏ＿ｊ may be determined based on the form of (e.g., manually estimated age y^ ｉ ｏ＿ｊ If the confidence level has a certain range, such as 35±10, the reciprocal of the range may be used as the confidence level.

[0070] FIG. 9 is a flowchart illustrating an example of a processing procedure executed by the determining unit 131 in the sixth pattern.

[0071] In step S601, the determination unit 131 determines the confidence level c ｉ ｏ＿ｊ If the condition is satisfied (Yes in S601), the determination unit 131 determines whether or not the condition that w is less than a threshold value (for example, 0.4) is satisfied. ｉ ｍ Let w be 1. ｉ ｏ＿ｊ If the condition is not satisfied (No in S601), the determination unit 131 sets w ｉ ｍ is set to 0, and w ｉ ｏ＿ｊ is set to 1 (S603).

[0072] In addition, when the age estimation model is a model that estimates the posterior probability for each age, the determination unit 131 determines the standard deviation σ of the posterior probability distribution. ｉ The inverse of the confidence level estimated separately by the age estimation model is the confidence level c ｉ ｍ As a result, the confidence level c ｉ ｏ＿ｊ Instead of or confidence level c ｉ ｏ＿ｊ The determination unit 131 may input the confidence level c ｉｍ If only the confidence level c is entered, the condition in step S601 is set to ｉ ｍ Alternatively, the determination unit 131 may be configured to determine the confidence level c ｉ ｏ＿ｊ and confidence level c ｉ ｍ When both of these are input, the condition of step S601 is set to the confidence level c ｉ ｏ＿ｊ and confidence level c ｉ ｍ and the condition based on (for example, confidence level c ｉ ｏ＿ｊ and confidence level c ｉ ｍ The above may be replaced with a condition in which a threshold value is set for each of the above and the judgment result is determined by comparing each of the above with the threshold value.

[0073] The output unit 132 outputs the automatically estimated age y^ ｉ ｍ and the estimated age of the human being y^ ｉ ｏ＿ｊ Of these, the weight w ｉ The one with 1 is the final estimated age y^ ｉ Output as

[0074] The results of the experiments in published papers 1 and 2 suggest that operators may estimate the age of a person from a voice without clear characteristics to be around 35 years old. The sixth pattern is an embodiment based on this finding.

[0075] [Seventh Pattern] In the seventh pattern, the determination unit 131 determines the estimated gender g^ by the operator o_j himself / herself. ｉ ｏ＿ｊ The input information is a value indicating the degree of confidence in the gender estimation by machine learning (hereinafter referred to as "confidence level"), and a value indicating the degree of confidence in the gender estimation by machine learning (confidence level). ｉ For example, if one of the two confidence levels is equal to or less than a threshold, the determination unit 131 determines w ｉ ｍ Let w be 1. ｉ ｏ＿ｊ Let w be 0 otherwise. ｉｍ is set to 0, and w ｉ ｏ＿ｊ Let be 1.

[0076] The results of the experiment in paper 3 showed that when the gender estimation result is incorrect, the age estimation result by the operator tends to deteriorate. On the other hand, in automatic age estimation using machine learning, age estimation models are trained separately for men and women, and the outputs from each of the two models for men and women are weighted and summed based on the results of gender estimation using machine learning (for example, defining male as 1.0 and female as 0.0 and inferring the posterior probability) to produce the automatically estimated age y^. ｉ ｍ In this method, when the posterior probability of gender estimation is close to 0.5, the error in automatic age estimation tends to be reduced compared to that of a human operator. The seventh pattern is an embodiment based on this finding.

[0077] [Combination of Multiple Patterns] Two or more of the above patterns may be combined. Fig. 10 shows a flowchart for explaining an example of the processing procedure executed by the determination unit 131 when combining the first to sixth patterns. In Fig. 10, the same steps as those in Figs. 4 to 9 are assigned the same step numbers, and their explanations will be omitted.

[0078] Although the seventh pattern is not combined in FIG. 10, the seventh pattern may also be combined as a branch point at any location.

[0079] When a plurality of patterns are combined, the determining unit 131 assigns a weight w ｐ The weights w are calculated using a model trained using machine learning. ｉ In this case, the determining unit 131 may determine the weight w ｉ The result of the judgment is ｐ The weighted sum of the weights w ｉ It may also be possible to use the following.

[0080] As described above, according to this embodiment, it is possible to estimate the speaker's age by taking into consideration the tendency of both human and machine estimations. As a result, it is possible to improve the accuracy of speaker age estimation based on the voice of an utterance compared to age estimation using only one of them.

[0081] Conventionally, when considering replacing subjective age estimation by an operator with automatic age estimation using machine learning, the only decision criterion was to conduct age estimation experiments on several samples and adopt the one with the better evaluation score (for example, the average value of the absolute errors between the true age and the estimated age). It was unclear what types of voices humans and machines are good at and what issues to be aware of when switching from human age estimation to automatic age estimation or not. However, according to the present embodiment, it is possible to appropriately select one or combine the two based on knowledge gained through experiments, etc.

[0082] [Published papers] 1. "Analysis of trends in speaker age estimation by humans and machine learning," Kitagishi Yuki et al., Autumn National Meeting of the Acoustical Society of Japan, 2022. 2. "What are the difference? Comparing DNN and human by their performance and characteristics in speaker age estimation," INTERSPEECH, scheduled for presentation in August 2023. 3. "Analysis of trends in humans and machine learning in age estimation taking speaker gender into account," Kitagishi Yuki et al., Autumn National Meeting of the Acoustical Society of Japan, scheduled for presentation in September 2023. [Experimental results for each published paper] Below are described the experiments and results conducted for published papers 2 and 3, which were unpublished at the time of filing this application. For the experimental results for published paper 1, please refer to published paper 1.

[0083] [Published Paper 2] In the following, "CA" is the speaker's age (Non-Patent Document 2), "PA" is the speaker's age estimated manually (Non-Patent Document 2), and "EA" is the speaker's age estimated by a machine learning model.

[0084] 2.1 Datasets Because DNN models may be trained under mismatched conditions between the training environment and the actual usage environment, we created in-house speech datasets for two different domains.

[0085] The first dataset (hereafter referred to as the "Call Center Dataset") consists of telephone speech from 532 Japanese speakers (305 females and 227 males) aged 11 to 89 years. Each participant was recorded following five scripted call center scenarios. One scenario was a telephone conversation between a customer and an operator, while the others were interactive voice response system utterances. This dataset contains five single-channel calls per speaker recorded indoors using a smartphone, capturing only the speaker's voice. All calls were downsampled by quantizing at 16 bits and converting to an 8 kHz sampling frequency. The dataset was randomly divided into training, validation, and test subsets with no overlapping speakers. Each subset contained 289, 76, and 170 participants, respectively. After removing non-speech periods using voice activity detection (VAD) (Reference 1), the average speech length was 35.25 seconds, with a standard deviation (SD) of 31.85.

[0086] The other dataset (hereafter referred to as the "short sentence dataset") consists of utterances from 3,904 Japanese speakers (2,220 females and 1,684 males) aged 6 to 91 years. This dataset contains 20 single-channel utterances per speaker recorded indoors. All utterances were downsampled by 16-bit quantization and conversion to an 8 kHz sampling frequency. The dataset was randomly divided into training, validation, and test subsets without speaker overlap. The number of participants was 3,297, 355, and 352, respectively. After removing silent periods using VAD, the average speech length was 6.73 seconds with a standard deviation (SD) of 6.61.

[0087] 2.2 Age Estimation Experiments Using Machine Learning We conducted automated experiments on speaker age estimation using a transformer model. The model architecture was determined with reference to Reference 2. That is, this model includes 13 WavLM transformer encoder layers from "WavLMBase+" (Reference 3), five ECAPA-TDNN layers (Reference 4), a careful statistical pooling layer, two dense layers, and an output layer with a softmax function. The hyperparameters of each layer are the same as Reference 2. To optimize the model parameters, β １ and β ２ We used the RAdam optimizer (reference 5) with σ set to 0.9 and 0.999, respectively. The loss σ was 3.6, the mini-batch size was 64, and the L2 regularization weight was 1e-4. Training speech samples were randomly clipped into consecutive 5-second segments after removing non-speech durations using VAD (reference 1) at each epoch to create mini-batches. The model training procedure was divided into two phases, as described in reference 2. First, the pre-trained parameters of the WavLM model were fixed, and other parameters were trained with initial and minimum learning rates of 1e-4 and 1e-6, respectively. Next, all model parameters were fine-tuned with initial and minimum learning rates of 1e-5 and 1e-7, respectively. We used the same learning rate decay strategy and data augmentation method (adding noise and reverberation using simulated room impulse responses) for the training subset as in reference 6. All hyperparameters were determined using the results of the validation subset.

[0088] Two types of training conditions were set: an out-of-domain (OD) condition and an in-domain (ID) condition. In the OD condition, only the short sentence dataset was used in the model training procedure. This is a condition outside the domain of the call center dataset. In the ID condition, both datasets were used in the model training procedure. This is a condition within the domain of the call center dataset.

[0089] The experimental results were evaluated using the mean absolute error (MAE) and the Pearson correlation coefficient ρ between CA and EA. Figure 11 shows the experimental results of age estimation by machine learning (MAE / ρ).

[0090] Both the OD and ID models achieved good estimation accuracy for the short-sentence dataset. However, the accuracy of the OD condition was not better than that of the ID condition for the call center dataset. These accuracy declines are due to domain mismatch between the two datasets. It is known that differences in acoustic characteristics, such as speaking style, can lead to reduced accuracy (References 7, 8).

[0091] Figure 12 shows the effect of test segment length on MAE for a call center dataset. Several studies have reported that estimation accuracy decreases when the input sample is short (References 2, 6, 9, 10). The results of this experiment showed a similar trend. In particular, when the test speech sample length was 5 seconds, the MAE decreased even further. The MAE was more than 10% worse than when the full length was used.

[0092] 2.3 Manual Speaker Age Estimation Experiment We conducted a speaker age estimation experiment with 21 Japanese participants (16 women and 5 men in their 20s to 60s) with experience working in call centers. Details are shown in Figure 13. Each participant listened to calls included in the test subset of the call center dataset using headphones and labeled the speaker's age group and gender. We defined 17 age classes in 5-year increments: 1. Early teens or younger, 2. Late teens, 3. Early twenties, ..., 16. Late 80s, 17. 90 years or older.

[0093] To evaluate the estimation results of these age classes, we define the MAE between the age class numbers of PA and CA, called "Categorical MAE (C-MAE)." The C-MAE is formulated as follows:

[0094] Here, c ｉ and c^ ｉ∈[1,17] is the age class number of CA and PA, and N is the number of test samples. Note that c^ denotes the symbol with ^ above c.

[0095] One point of C-MAE corresponds to five years of MAE. The ρ value between the age class numbers of CA and PA is also used. In addition, the mean standard deviation (MSD) of the estimation error for each speaker is evaluated as the intra-estimator variability. The C-MAE for each speaker (C-MAEs) is defined as follows:

[0096] Here, s is the speaker ID and j is the scenario ID.

[0097] MSD is defined as follows:

[0098] where S is the number of speakers.

[0099] The results are shown in Figure 13. Participants estimated the speaker's age with an error of 1.37 to 3.43 points for C-MAE for female speakers and 1.62 to 2.51 points for C-MAE for male speakers. The average accuracy of PA is also shown in Figure 13, but there were also accuracies that were better than the accuracy of each participant. This is due to a phenomenon similar to ensemble estimation in machine learning. In addition, the MSD was 1.12 to 1.95 points for female speakers and 1.30 to 2.07 points for male speakers. There is no considered abnormal accuracy.

[0100] 2.4 Comparison of machine learning model and human experiment results To compare the categorical age estimation accuracy of the participants, EA converted the age values ​​into age classes in 5-year increments. The C-MAE, MSD, and ρ values ​​between the age class numbers of EA and CA are shown in the bottom two rows of Figure 13.

[0101] The DNN model performed at or above the accuracy of the participants. The DNN model trained in the OD condition outperformed 57% (12 / 21) of the participants for female speakers and 38% (8 / 21) of the participants for male speakers. The DNN model trained in the ID condition outperformed 86% (18 / 21) of the participants for female speakers and all participants for male speakers.

[0102] Figures 14 and 15 show the C-MAE for female and male speakers by CA class. Each bar shows the average C-MAE for PA and the C-MAE for the DNN model trained under ID and OD conditions. Similar to references 11-13, the C-MAE for older speakers using the DNN model was worse than that for younger speakers. In particular, the C-MAE was worse in the OD condition than in the ID condition. This degradation is due to the small number of older speakers included in the training subset and the difficulty of estimating the age of older speakers (reference 11).

[0103] [Published Paper 3] This paper presents the results of an analysis of the tendency of age estimation errors when gender estimation errors occur when speaker age is estimated by humans and DNNs.

[0104] For humans, experienced call center operators simultaneously estimated age (in 5-year increments) and gender. When gender was incorrectly estimated, there was a tendency for age estimation to also be significantly incorrect (3 classes = 15 years or older).

[0105] For the DNN, inference was performed using a model structure that took gender into consideration. The model was trained separately for male and female encoders, and age was estimated by softly taking the posterior probability of gender estimation into consideration. Softly taking the posterior probability of gender estimation into consideration means that, if the probability of male in gender estimation is α and the probability of female is β, the weight for the output of the male model is α and the weight for the output of the female model is β, respectively, and the weighted sum is used as the age estimation result. On the other hand, hard consideration of the posterior probability of gender estimation means that if α > β, only the output of the male model is adopted, and if α < β, only the output of the female model is adopted.

[0106] When the DNN misestimated gender, there were times when it misestimated age, and times when it misestimated age. Among the cases where the gender was misestimated, the number of cases where the age was significantly incorrect (± approximately 15 years or more) was significantly more frequent in humans than in DNN (Fisher's exact test, p = 0.045).

[0107] Furthermore, for DNN, the posterior probability of gender estimation had a negative impact on the age estimation results. When gender estimation was incorrect, there was a tendency for age estimation to be significantly incorrect when the error between the accuracy probability and the estimation probability was large, and when this error was small, there was a tendency for age estimation error to be small. The degree of error in gender estimation by DNN when it mis-estimated gender was defined using the posterior probability. A high correlation (correlation coefficient ρ = 0.865) was observed between the degree of error and age estimation error. In other words, there was a tendency for a large error in gender estimation to also result in a large error in age estimation.

[0108] Furthermore, for DNN, switching to hard consideration of gender estimation results tended to worsen age estimation accuracy. Therefore, it is thought that the mechanism that softly considers gender prevents deterioration of age estimation accuracy when gender estimation is incorrect. When hard consideration of gender estimation results was used, the average absolute error of age estimation when gender estimation was incorrect significantly worsened (p = 0.032 in a paired t-test), supporting this hypothesis.

[0109] The conclusions are as follows: ・Errors in gender estimation have a negative impact on age estimation in both humans and DNNs. ・However, DNNs can softly take gender into account, which may prevent accuracy degradation.

[0110] [References] 1. Z.-H. Tan, A. kr. Aarkar, and N. Dehak, "rVAD: An unsupervised segment-based robust voice activety detection method," Computer Speech and Language, vol. 59, pp. 1-21, 2020. 2. Z. Kang, J. Wang, J. Peng, and J. Xiao, "SVLDL: Improved speaker age estimation using selective variance label distribution learning," in SLT. IEEE, 2022, pp. 1037-1044. 3. S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y. Qian, Y. Qian, J.Wu, M. Zeng, X. Yu, and F.Wei, "WavLM: Large-scale self-supervised pre-training for full stack speech processing," IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505-1518, 2022. 4. B. Desplanques, J. Thienpondt, and K. Demuynck, "ECAPATDNN: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification," in INTERSPEECH, 2020, pp. 3380-3384. 5. L. Liu, H. Jiang, P. He, C.Weizhu, X. Liu, J. Gao, and J.han, "On the variance of the adaptive learning rate and beyond," in ICLR, 2020, 13 pages. 6.P. Ghahremani, P. S. Nidadavolu, N. Chen, J. Villalba, D. Povey, S. Khudanpur, and N. Dehak, "End-to-end deep neural network age estimation," in INTERSPEECH, 2018, pp. 277-281. 7.R. Takeda and K. Komatani, "Age estimation with speech-age model for heterogeneous speech datasets," in INTERSPEECH, 2021, pp. 4164-4167. 8.N. Tawara, A. Ogawa, Y. Kitagishi, H. Kamiyama, and Y. Ijima, "Robust speech-age estimation using local maximum mean discrepancy under mismatched recording conditions," in ASRU. IEEE, 2021. 9.R. Zazo, P. S. Nidadavolu, N. Chen, J. Gonzalez-Rodriguez, and N. Dehak, "Age estimation in short speech utterances based on LSTM recurrent neural networks," IEEE Access, pp. 22 524- 22 530, 2018. 10.S. Si, J.Wang, J. Peng, and J. Xiao, "Towards speaker age estimation with label distribution learning," in ICASSP. IEEE, 2022, pp. 4618-4622. 11.M. Huckvale and A.Webb, "A comparison of human and machine estimation of speaker age," in SLSP, 2015, pp. 111-122. 12. S.S. Waller, M. Eriksson, and P. Sörqvist, "Can you hear my age? Influences of speech rate and speech spontaneity on estimation of speaker age," Frontiers in Psycholagy, vol. 6, p. 978, 2015. 13. H. Goy, M.K. Pichora-Fuller, and P. van Lieshout, "Effects of age on speech and voice quality ratings," The Journal of the Acoustical Society of America, vol. 139, no. 4, pp. 1648-1659, 2016. The following supplementary notes are further disclosed regarding the above embodiments.

[0111] (Supplementary Item 1) A speaker age estimation device comprising: a memory; and at least one processor connected to the memory, wherein the processor determines (or decides) weights for a first age estimated by a human based on the speaker's speech and a second age estimated by a machine using a machine learning model based on one or more of information related to the person, information related to the machine, and information related to the speech; and outputs an estimate of the speaker's age based on the first age, the second age, and the weights.

[0112] (Supplementary Item 2) A recording medium having recorded thereon a program that causes a computer to execute the following steps: a determination procedure (or decision procedure) for determining (or deciding) weights for a first age estimated by a human based on the voice of the speaker's speech and a second age estimated by a machine using a machine learning model, based on one or more of information related to the person, information related to the machine, and information related to the voice; and an output procedure for outputting an estimate of the speaker's age based on the first age, the second age, and the weights.

[0113] In this embodiment, the manually estimated age y^ ｉ ｏ＿ｊ is an example of the first age. ｉ ｍ is an example of the second age.

[0114] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0115] REFERENCE SIGNS LIST 10 Speaker age estimation device 11 Automatic estimation unit 12 Manual estimation unit 13 Age determination unit 14 Analysis unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 Processor 105 Interface device 131 Determination unit 132 Output unit B Bus

Claims

1. A speaker age estimation device comprising: a determination unit configured to determine weights for a first age estimated by a human based on the speaker's speech and a second age estimated by a machine using a machine learning model, based on one or more of information related to the person, information related to the machine, and information related to the speech; and an output unit configured to output an estimate of the speaker's age based on the first age, the second age, and the weights.

2. The speaker age estimation device of claim 1, characterized in that the information related to the person includes the gender of the speaker estimated by the person and the first age, or the information related to the person includes an estimation accuracy of the age based on the voice of the person evaluated in a predetermined manner, and the information related to the machine includes an estimation accuracy of the age based on the voice of the machine learning model evaluated in the predetermined manner, or the information related to the machine includes the second age, and the information related to the voice includes acoustic features of the voice, or the information related to the voice includes a duration of the voice, or the information related to the person includes a value indicating the person's degree of confidence in the first age, or the information related to the machine includes information based on a probability distribution regarding the estimation of the second age output by the machine learning model, or the information related to the person includes the gender of the speaker estimated by the person and a value indicating the person's degree of confidence in the gender, and includes a value indicating the degree of confidence in the gender of the speaker estimated by the machine through machine learning.

3. The speaker age estimation device of claim 1, characterized in that the information related to the person includes a plurality of the first ages, the judgment unit sets a weight for the first age to 1 and a weight for the second age to 0 when the number of the first ages is equal to or greater than a threshold, and the output unit outputs an average value of the plurality of the first ages as the estimated value.

4. The speaker age estimation device of claim 1, characterized in that the judgment unit is configured to select either the first age or the second age based on one or more of information related to the machine, information related to the person, and information related to the voice, and the output unit is configured to output the age selected by the judgment unit.

5. A speaker age estimation method, characterized in that a computer executes the following steps: a determination step of determining weights for a first age estimated by a human based on the speaker's speech and a second age estimated by a machine using a machine learning model, based on one or more of information related to the person, information related to the machine, and information related to the speech; and an output step of outputting an estimate of the speaker's age based on the first age, the second age, and the weights.

6. A program that causes a computer to execute the following steps: a determination step of determining weights for a first age estimated by a human based on the speaker's speech and a second age estimated by a machine using a machine learning model, based on one or more of information related to the person, information related to the machine, and information related to the speech; and an output step of outputting an estimate of the speaker's age based on the first age, the second age, and the weights.