Estimation method, learning method, estimation device, and estimation program

Abstract

An estimation device (10) has: an activity score calculation unit (11) that calculates an activity score representing the degree to which the excitement of a conversation affects the satisfaction of each participant on the basis of on linguistic features of speech in a transcript of a conversation involving a plurality of people; an enthusiasm score calculation unit (12) that divides the transcript of the conversation along a time axis and calculates an enthusiasm score representing the degree to which the enthusiasm of utterance in the conversation by the participants affects the satisfaction of each participant on the basis of a speech count and spoken word count of each participant in each section; an influence score calculation unit (14) that divides the transcript of the conversation and calculates an influence score representing the degree to which the influence of speech in each divided period by an initial speaker and repeated speakers on the process and result of consensus formation affects the satisfaction of each participant; and a conversation assessment estimation unit (15) that estimates a conversation assessment score representing an assessment of the conversation by each participant on the basis of the activity score, the enthusiasm score, and the influence score.

Description

[Technical field]

[0001] The present invention relates to an estimation method, a learning method, an estimation device, and an estimation program. [Background technology]

[0002] As an evaluation technique for group dialogue, there is a technique to estimate leadership and contribution from the number of times of speaking during a dialogue, the frequency of words contained in the spoken sentences, the number of nods analyzed from camera footage, etc. There is also a technique to estimate the performance of the entire group by calculating features by graphing the speech turns of the dialogue. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-242528 A [Non-patent literature]

[0004] [Non-Patent Document 1] Indrani Bhattacharya et al, “A Multimodal-Sensor-Enabled Room for Unobtrusive Group Meeting Analysis”, ICMI'18, October 16-20, 2018. [Non-Patent Document 2] Uriyana Kubasova et al, “Group Performance Prediction with Limited Context”, ICMI '20 Companion, October 25-29, 2020. Summary of the Invention [Problem to be solved by the invention]

[0005] When evaluating a dialogue, the results obtained through the dialogue are often used to evaluate the performance of the group that conducted the dialogue. However, satisfaction with the dialogue may vary among individual participants, even if they belong to the same group.

[0006] For example, there are cases where the satisfaction of a person who can contribute to the formation of a group consensus by actively speaking up increases (case 1), and cases where the satisfaction of a person who can provide a topic that can help promote the formation of a group consensus increases (case 2).

[0007] In the first case, when making active comments, it is necessary to consider the ideas that lead to consensus building or the comments about related topics. In the second case, consensus building is promoted by two types of participants: the "first speaker" who first proposes a new idea or topic, and the "repeated speaker" who sympathizes with an idea or topic proposed by someone else and brings it up in conversation. In this way, participants who make many comments that promote consensus building can be said to have a "high degree of influence," and it is necessary to consider the participants with a high degree of influence and their degree of influence when estimating the evaluation.

[0008] However, even if the above-mentioned proactiveness and influence can liven up the dialogue, they may not directly relate to the outcome, and therefore cannot be taken into account in an evaluation based solely on the results of consensus building through dialogue. In addition, since satisfaction levels differ between proactive and inactive people, and between people with high and low influence, each participant is likely to have a different evaluation of the same dialogue. Furthermore, evaluations of the group as a whole do not express the evaluations of each individual participant.

[0009] For example, Patent Document 1 describes a method for evaluating a dialogue based on information about a specific utterance in the dialogue and the length of the dialogue. However, the method described in Patent Document 1 does not use features that distinguish between proactiveness and speakers of two types of ideas and topics, and it is not possible to obtain an evaluation of each participant that takes proactiveness and influence into consideration.

[0010] Non-Patent Document 1 describes a method of extracting opinions from spoken sentences and camera footage, and scoring leadership and contribution. Non-Patent Document 2 describes a method of creating a graph with linguistic features during a conversation and utterances, speakers, and topics as nodes, and estimating group performance using features calculated from the centrality. The methods described in Non-Patent Documents 1 and 2 use speaker switching and the number of times a topic is spoken as non-linguistic and linguistic features for estimating the evaluation of consensus building.

[0011] However, the methods described in Non-Patent Documents 1 and 2 take into account speaker changes for each utterance, but do not distinguish between "first-time speakers" and "repeated speakers," so they cannot be said to appropriately request evaluation of individual participants that takes into account their proactiveness and influence. Furthermore, the method described in Non-Patent Document 2 evaluates the performance of the entire group as the quality of the results of consensus building as an evaluation of the dialogue, and does not request evaluation of individual participants.

[0012] The present invention has been made in consideration of the above, and aims to provide an estimation method, a learning method, an estimation device, and an estimation program that can estimate each participant's evaluation of a group dialogue. [Means for solving the problem]

[0013] In order to solve the above-mentioned problems and achieve the object, the estimation method of the present invention is characterized by including a first calculation step of receiving at least a transcript of the dialogue as dialogue data between multiple people, and calculating an activity score representing the degree to which the excitement of the dialogue affects the satisfaction of each participant based on linguistic features of the utterances in the transcript of the dialogue; a second calculation step of dividing the transcript of the dialogue along a time axis, and calculating an activeness score representing the degree to which the activeness of the participants' comments in the dialogue affects the satisfaction of each participant based on the number of utterances and number of words spoken by each participant in each section; a third calculation step of dividing the transcript of the dialogue, identifying first-time speakers and repeat speakers for each divided period, and calculating an influence score representing the degree to which the utterances of the first-time speaker and repeat speakers had an influence on the process and results of consensus building on the satisfaction of each participant; and a step of estimating a dialogue evaluation score representing each participant's evaluation of the dialogue based on the activity score, activeness score, and influence score.

[0014] The learning method of the present invention is characterized by including a fourth calculation step of receiving input of at least a transcript of the dialogue as training dialogue data for a plurality of people, and calculating an activity score representing the degree to which the excitement of the dialogue affects the satisfaction of each participant based on the linguistic features of the utterances in the transcript of the dialogue; a fifth calculation step of dividing the transcript of the dialogue along a time axis, and calculating an activeness score representing the degree to which the activeness of the participants' utterances in the dialogue affects the satisfaction of each participant based on the number of utterances and the number of words spoken by each participant in each section; a sixth calculation step of dividing the transcript of the dialogue, identifying first-time speakers and repeat speakers for each divided period, and calculating an influence score representing the degree to which the utterances of the first-time speaker and repeat speakers had an influence on the process and results of consensus building on the satisfaction of each participant; and a creation step of creating a model that estimates a dialogue evaluation score representing each participant's evaluation of the dialogue by machine learning using the activity score, activeness score, and influence score as input, and the satisfaction level based on the participants' subjective evaluation of the training dialogue data as correct answer data. Effect of the Invention

[0015] According to the present invention, it is possible to estimate the evaluation of the group dialogue by each individual participant. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an estimation device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating the feature amounts used in calculating the aggressiveness score. [Diagram 3] FIG. 3 is a diagram for explaining an example of a dialogue turn. [Figure 4] FIG. 4 is a diagram for explaining another example of a dialogue turn. [Diagram 5] FIG. 5 is a flowchart illustrating a processing procedure of the estimation process according to the embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of a processing procedure of the aggressiveness score calculation processing illustrated in FIG. [Figure 7] FIG. 7 is a flowchart illustrating an example of a processing procedure of the influence score calculation processing illustrated in FIG. [Figure 8] FIG. 8 is a diagram illustrating an example of a configuration of a learning device according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing a processing procedure of the learning process in the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a computer that realizes an estimation device and a learning device by executing a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are indicated by the same reference numerals.

[0018] [Embodiment Mode] The estimation method according to the present embodiment estimates each participant's evaluation of a group dialogue using dialogue data that records information about a dialogue between multiple people. The estimation method according to the present embodiment quantifies the group dialogue from three aspects, namely, the participants' activity, proactiveness, and influence, and estimates each participant's evaluation of the dialogue. Thus, in the present embodiment, a score is estimated that is a numerical value of an evaluation that reflects the satisfaction (or achievement) of each participant in a group dialogue of two or more people with respect to the dialogue.

[0019] That is, the estimation method according to the embodiment estimates a dialogue evaluation score, which is an evaluation score of the dialogue for each participant, based on the activity score, proactiveness score, and influence score. The dialogue evaluation score, as described below, is a score that represents the satisfaction level and further the achievement level of each participant. First, the dialogue data used for evaluating the dialogue of each participant in a group dialogue will be described.

[0020] [Dialogue data] The dialogue data is data in which information about a dialogue to be evaluated is recorded. The dialogue data is data in which information about a dialogue is recorded in chronological order.

[0021] The dialogue data includes at least text data (transcript of dialogue) of what each participant said. The dialogue data may include, for example, voice data collected by a microphone.

[0022] The dialogue data may include video data capturing the movements of each participant, vital data such as each participant's heart rate recorded using a device such as a smart watch, and the like.

[0023] The dialogue data may include individuality data representing the individuality of each participant in the dialogue. The individuality data is data representing the character and values ​​of the participants constituting a dialogue group.

[0024] For example, the personal data may include the results of a personality questionnaire, personal data predicted from past data using existing technology, and data on attributes such as age, work history, and job title.

[0025] The individual data is, for example, data based on the results of a questionnaire about personality and attributes. Specifically, the individual data is data based on the result of participants selecting the answer that best suits them from options such as "I think so," "I think a little," "I don't think so much," and "I don't think so at all" in response to the question, "I like talking about myself in front of others." The individual data is also data based on the result of participants selecting the answer that best suits their personality and values ​​from words such as "quiet," "talkative," and "outgoing." Alternatively, the individual data is data based on the result obtained by participants rating each item on, for example, a nine-point scale.

[0026] The individuality data may be based on data on evaluations of each participant by other people. The individuality data may be based on a score obtained by predicting individuality using past dialogue data or the like.

[0027] When using the results of a questionnaire about personality or attributes, if the questionnaire answers are given in graded numbers, the numbers may be used as they are as personal data, or the aggregated values ​​may be used as personal data. If the questionnaire answers are given in Yes or No, the numerical data, such as 1 for Yes and 0 for No, may be used as personal data.

[0028] When answering by multiple choice from non-continuous items such as occupation or preference, a feature for the number of items may be prepared, and the selected items may be converted into a vector with 1 as the selected items and 0 as the rest, and the data may be used as the personal data. In the case of free text, data using the contained words as the feature may be used as the personal data. In addition, in the case of free text, the written content may be categorized, a feature for the number of categories may be prepared, and the selected items may be converted into a vector with 1 as the selected items and 0 as the rest, and the data may be used as the personal data. The personal data may be a mixture of questions with different answering methods, such as questions answered with Yes or No, questions answered on a 7-point scale, and free text questions.

[0029] [Activity score] Next, the liveliness score will be described. The liveliness score is a numerical value that indicates the degree to which the excitement of the conversation affects the satisfaction of each participant. The liveliness score may be a score that numerically indicates how lively the conversation is, or the participants' feelings of elation. In particular, it is assumed that in a conversation between participants who meet for the first time, it is often the case that importance is placed on the overall continuity of speech and the excitement of the situation, rather than on the frequency or quality of one's own remarks. Based on this assumption, in this embodiment, the liveliness score is defined as a numerical value that indicates the degree to which the excitement of the conversation and the participants' feelings of elation affect satisfaction.

[0030] The dialogue activity score is obtained from the dialogue data. For example, if the dialogue data is text data, linguistic features of the speech in the dialogue data, such as the number of times each participant speaks and the number of words spoken, are used as the activity score. If the dialogue data is voice data, the volume and changes in the voice of each participant are used as the activity score. If the dialogue data is video data, the size of gestures, the size of nodding, etc. are used as the activity score. If the dialogue data is vital sign data, the speed and changes in heart rate are used as the activity score.

[0031] Furthermore, when the dialogue data includes personal data, the personal data may be converted into a numerical value that estimates the excitement and elation for each dialogue participant, and used as the activity score. For example, the degree of excitement for a sociable person and the degree of excitement for an introverted person may be scored using a gradient based on the personal data, and used as the activity score. Furthermore, data that quantifies the individuality that represents the personality and values ​​that can be determined from the personal data and data that quantifies the activity score that can be determined from dialogue data other than the personal data may be used as a feature.

[0032] [Positiveness Score] Next, the proactiveness score will be explained. The proactiveness score is a numerical value that indicates the degree to which the proactiveness of participants' comments in a dialogue affects each participant's satisfaction. The proactiveness score is a numerical value that indicates the degree to which whether or not a participant was able to speak proactively in a dialogue affects each participant's satisfaction. Here, proactive comments include not only comments that lead the dialogue, but also questions seen when encouraging the other person to talk or digging deeper, and comments showing empathy. Therefore, comments that have substance, rather than mere backchannel (interjections), are judged to be more proactive, and the proactiveness score increases.

[0033] The proactiveness score of each participant is calculated from the dialogue data. The proactiveness score is based on the number of utterances and the number of words uttered when focusing on each participant's own utterance. In this embodiment, all utterances are not treated equally, but the importance of utterances about ideas for consensus building and related topics is set higher than the importance of other utterances.

[0034] For this reason, when calculating the proactiveness score, backchannels and utterances that are not related to consensus building are weighted low, and conversations that include content related to consensus building are weighted high. Utterances that are not related to consensus building are, for example, utterances related to interactions such as repeating when writing characters on paper or a whiteboard, or checking spelling and kanji.

[0035] [Impact Score] Next, the influence score will be described. The influence score is a numerical value that indicates the degree of satisfaction of each participant due to the influence of the other participants and the process and results of consensus building, i.e., whether the participant was able to liven up the dialogue or make a statement that leads to a result, through their own remarks. The influence score is a numerical value that indicates the degree of satisfaction of each participant due to the influence of the utterances of the first speaker and the repeat speaker, which will be described later, on the process and results of consensus building. The influence score of each participant is calculated based on the dialogue data and the dialogue turns.

[0036] During the consensus building process, various topics and ideas may emerge and the dialogue may continue about them, or the focus of the dialogue may quickly shift to other topics or ideas.

[0037] In this embodiment, a dialogue turn is defined as a section of a dialogue in which a specific topic or idea is being discussed. In an actual dialogue, multiple topics may overlap. Furthermore, in an actual dialogue, a dialogue turn on a broad topic may contain a dialogue turn on a more specific topic.

[0038] For this reason, when determining the dialogue along a chronological order, there is a method of dividing the dialogue so that dialogue turns about a plurality of different topics and ideas overlap, or so that a specific dialogue turn is always uniquely determined.

[0039] In this embodiment, dialogue turns are automatically extracted by determining the contents of utterances. In this embodiment, dialogue turns are extracted from a transcript of a dialogue using key words, which are words that represent ideas and topics. The key words may be words that are included in notes or blackboard notes about ideas and topics during the consensus building process, or in documents such as proposals as a result of consensus building, or in transcripts of oral presentations of results, and the extraction method is not important.

[0040] For example, only nouns are considered to be important words among the words included in the proposal document of the agreement result. Alternatively, the TFIDF (Term Frequency-Inverse Document Frequency) of words included in the proposal document of the agreement result is calculated using proposal documents of other groups, and words with high calculated values ​​are considered to be important words. Note that the extraction of dialogue turns may be manually annotated.

[0041] The influence score is calculated for each dialogue turn. In each dialogue turn, a dialogue about an idea or related topic takes place. Here, consensus building is promoted by two types of participants: the "first speaker" and the "repeated speaker" of the idea or topic. For this reason, it is necessary to identify the first speaker and the repeated speaker for each dialogue turn.

[0042] In this embodiment, an important word used in a dialogue turn is used to identify the first speaker and the repeat speaker. In the entire dialogue, the person who first utters this important word is called the "first speaker," and the person who utters it the second or subsequent time is called the "repeated speaker." The first speaker and the repeat speaker may be the same person.

[0043] Next, the extent to which the conversation became lively as a result of the first and repeated utterances is used as an influence score. Based on how long the conversation about the important word continued, or how much the conversation frequency increased compared to before the important word was spoken, an influence score is added to the first speaker and the repeat speaker of the important word. However, if it was the first time that the important word was spoken, the influence score is added only to the first speaker.

[0044] [Interaction evaluation score] Next, the dialogue evaluation score will be described. The dialogue evaluation score is calculated by combining the activity score, proactiveness score, and influence score obtained based on the dialogue data. The dialogue evaluation score is a score that represents the achievement and satisfaction of the participants in the dialogue, and is given to each participant. It is also possible to express the score of the entire group by taking the average of the scores of each participant, for example.

[0045] In this embodiment, for example, a model is used to obtain a dialogue evaluation score. The model receives the liveness score, proactiveness score, and influence score calculated for each of the dialogue data to be estimated, and performs various calculation processes such as weighting each score to obtain and output the dialogue evaluation score.

[0046] The model is trained to estimate the dialogue evaluation score by machine learning using the activation score, proactiveness score, and influence score calculated for each of the training dialogue data as input and the satisfaction level based on the subjective evaluation of the training dialogue data as correct answer data. In an embodiment, the trained model is used to calculate the dialogue evaluation score for new dialogue data in the same manner and estimate the agreement evaluation of each participant.

[0047] [Estimation device] Specifically, an estimation device according to an embodiment will be described. Here, an estimation device is described that estimates a dialogue evaluation score of each participant for new dialogue data by using a model trained using a plurality of dialogue data and dialogue evaluation scores as correct answer data, and estimates the evaluation of the dialogue by each participant in a group dialogue. Fig. 1 is a diagram showing a schematic example of a configuration of an estimation device according to an embodiment.

[0048] The estimation device 10 shown in FIG. 1 is realized, for example, by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The estimation device 10 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network or the like. As shown in FIG. 1, the estimation device 10 has an activity score calculation unit 11 (first calculation unit), an activeness score calculation unit 12 (second calculation unit), a dialogue turn extraction unit 13, an influence score calculation unit 14 (third calculation unit), and a dialogue evaluation estimation unit 15 (estimation unit).

[0049] The estimation device 10 receives new dialogue data and a participant list as input. The participant list lists k participants in a dialogue, h1, . . . , h k The dialogue data includes at least a transcript of what was said by each participant.

[0050] The estimation device 10 may use, as the dialogue data, data that can determine the state of the dialogue, such as speech, video, and biosensor information. The estimation device 10 may also use, as the dialogue data, personal data that represents personal characteristics, such as attributes, such as age, work history, and job title, and survey results asking about values, experiences, and preferences. The estimation device 10 does not care about the number of types of dialogue data, so long as it is the dialogue data described above.

[0051] Then, the estimation device 10 calculates the dialogue evaluation scores F1, . . . , F k The dialogue evaluation score is a score representing the degree of satisfaction of each participant with respect to the dialogue, and is a score that indicates whether each participant is satisfied with the dialogue.

[0052] [Activity score calculation section] The activity score calculation unit 11 receives the dialogue transcript and the participant list, which are dialogue data, and calculates and outputs an activity score for each participant of the dialogue based on the linguistic features of the utterances in the dialogue data. k The total speech time UT = (u1, ,u k ), the number of back channels for each participant is BC=(b1, ,b k ), the number of times each participant spoke a proper noun W = (w1, , w k ), personality and attribute score PNT = (p1, ,p k ) is extracted as a feature.

[0053] Here, the backchannel can be extracted from the dialogue data based on a definition such as an utterance that does not include verbs, nouns, adjectives, numerals, etc. The number of proper noun utterances is calculated by performing morphological analysis on the transcribed utterances in advance and counting the number of proper nouns among the words. The personality and attribute scores are calculated by aggregating the results of a questionnaire and converting them into numerical values ​​using classification methods such as the Big Five personality traits or existing scales.

[0054] The feature used for the activity score may be a numerical value that represents the overall state of the conversation. Therefore, the estimation device 10 may use, as the feature used for the activity score, other linguistic feature, vital data such as movements, facial expressions, and heart rate obtained from a video or a sensor, and / or non-linguistic feature based on personal data.

[0055] Each feature of each participant may be a scalar value or a vector value. For example, the estimation device 10 estimates the number of times a proper noun is spoken W j =(w´ j1 ,···,w´ jc ) may be used as a feature quantity, i.e., the number of occurrences of each of the c proper nouns. j =(p´ j1 ,···,p´ jc ) d types of personality trait factors or attributes may be used as features.

[0056] Furthermore, the estimation device 10 may extract and use only the feature of the participant corresponding to the dialogue evaluation. The estimation device 10 may use aggregated values ​​such as the average, maximum and minimum values, and variance of all participants. The estimation device 10 may use, as the feature, numerical values ​​that can be extracted or calculated from the dialogue data, such as the average or maximum volume of the microphone, the number of actions such as nodding, or the number of times the maximum or average heart rate was exceeded. The estimation device 10 may classify attributes such as age and job title from data representing individuality, and use a number that represents the group to which the participant belongs.

[0057] As an example, the estimation device 10 calculates the activity score for all participants in the dialogue using four types of feature amounts, namely, the total speech time, the number of back channels, the number of times proper nouns are spoken, and the personality / attribute score, which are calculated individually. In this case, the activity score A can be expressed as formula (1) using the feature amounts (UT, BC, W, PNT).

[0058]

number

[0059] [Positiveness score calculation section] The proactiveness score calculation unit 12 calculates and outputs a proactiveness score for each participant in the dialogue based on the dialogue data and the participant list. Here, the proactiveness score calculation unit 12 separates the dialogue data into utterances of each participant and focuses only on the utterances of the target participant for which the proactiveness score is to be calculated.

[0060] There are two types of utterances: "content utterances" that promote consensus building or provide topics of conversation to stimulate discussion, and "empathic utterances" that express listening to the conversation of other participants, such as nodding.

[0061] The proactiveness score is higher for participants who speak more and whose speech is content-based. The proactiveness score is lower for participants who speak less and who speak more empathetic than content-based. Therefore, expressing one's own opinions a lot or talking about topics related to the theme of the conversation will contribute to a high proactiveness score. Even if one is listening to what others say, asking questions to expand the conversation or summarizing and organizing the main points of the conversation will also contribute to a high proactiveness score, as they are actively participating in the conversation.

[0062] For example, an example will be described in which scoring is performed based on the number of utterances and the parts of speech of words included in the utterances as the processing of the proactive score calculation unit 12. Note that the method of calculating the proactive score is not limited to this, and any score that represents the characteristics of the proactive score described above may be used.

[0063] The proactiveness score calculation unit 12 divides the transcript of the dialogue to be estimated along a time axis, and calculates the proactiveness score based on the number of utterances and the number of words uttered by each participant in each section. i The conversation is divided into m minutes. The proactiveness score calculation unit 12 calculates the proactiveness score of the participant h i The transcript of the dialogue spoken by the user is subjected to morphological analysis and divided into words.

[0064] FIG. 2 is a diagram explaining the feature values ​​used to calculate the proactiveness score. As shown in FIG. 2, i The number of words in the utterance in a certain zone z is cnt_word iz The number of words that fall into the category of nouns, verbs, or adjectives is called cnt_kw. iz , cnt_ow is the number of words other than nouns, verbs, or adjectives iz For each section, participant h i Interval aggressiveness score Ps´ iz can be expressed as equation (2). In equation (2), α, β, and γ represent the weights of each word count.

[0065]

number

[0066] The method of calculating the section aggressiveness score is not limited to this, and any score may be used as long as content utterances have a positive effect and empathy utterances have a negative effect. Therefore, the aggressiveness score may be calculated based on the number of utterances, not just the number of words, or may be calculated based on actions or behaviors that can distinguish content utterances from empathy utterances.

[0067] The proactiveness score may be a relative value to other participants. Specifically, the proactiveness score calculation unit 12 may use, as the proactiveness score, a value obtained by dividing the section proactiveness score of a target participant by the sum of the section proactiveness scores of all participants for a certain section.

[0068] The aggressiveness score calculation unit 12 calculates the section aggressiveness score for the entire section, and calculates the aggressiveness score Ps i The average, variance, maximum value, minimum value, etc. of the interval aggressiveness score in the entire interval are used as the aggressiveness score Ps i can be expressed as a vector shown in equation (3).

[0069]

number

[0070] The aggressiveness score may be a value calculated based on the interval aggressiveness score and may be assigned to each participant. The aggressiveness score may be a scalar value or a vector value.

[0071] [Dialogue turn extraction section] The dialogue turn extraction unit 13 divides the dialogue data into dialogue turns based on the dialogue data and the participant list, and outputs the start and end points of each dialogue turn and key words at those times.

[0072] The start and end points of a dialogue turn may be anything that expresses where in the dialogue data the start and end of the dialogue turn fall. The start and end points of a dialogue turn may be, for example, the time elapsed from the start of the dialogue, or information that uniquely identifies the utterances corresponding to the start and end. The dialogue turn extraction unit 13 may also divide the dialogue data into time zones at specific times, such as every 5 minutes or every minute, and output information that uniquely identifies the time zone that includes the start and end utterances.

[0073] The dialogue turn extraction unit 13 extracts dialogue turns using the key word list. As an example, a case will be described in which a key word list based on a report summarizing the results of consensus building by each dialogue group is used.

[0074] The report contains ideas that each group agreed upon. In the embodiment, a morphological analysis is performed on the sentences in the report, and only nouns are extracted from the divided words. In this embodiment, other parts of speech such as verbs and adjectives may also be extracted.

[0075] The extracted word group includes words that express ideas, commonly used words, or words that are used by any group, such as topics for consensus-building dialogue.

[0076] For example, when forming a consensus on a travel plan, words such as "travel," "movement," and "meal" are common and do not represent the ideas of this group. Therefore, it is necessary to remove such words. There is no particular method for removing such words, and they can be manually selected.

[0077] It is also possible to use reports from other groups, treat each report as a single document, calculate TFIDF or IDF (Inverse Document Frequency), and select words that are unique to that group. DF (Document Frequency) is the number of documents in which a word appears, and TFIDF and IDF use the inverse of this as the score. For this reason, the more a word appears in reports from many groups, the lower its score tends to be, and the more a word appears only in that group, the higher its score tends to be. By using TFIDF or IDF, or scores with similar trends to TFIDF and IDF, it is possible to identify words that represent ideas in the target dialogue group.

[0078] If multiple groups are not engaged in dialogue, similar effects can be achieved by using other types of dialogue data or by using a variety of general document sets, such as general news articles, blog articles, and articles from social networking services (SNS).

[0079] The dialogue turn extraction unit 13 uses the group of words thus obtained as a list of important words for that dialogue group. The list of important words may include synonyms, related words, and the like of the words by using external data, etc. Also, the same list of important words may be commonly used by a plurality of dialogue groups. Hereinafter, the list of important words IW=(iw1, , iw u ,···,iw i )

[0080] The dialogue turn extraction unit 13 then chronologically tracks the transcript of the dialogue and extracts, from the transcript of the dialogue, a spoken sentence in which a word (key word) included in the key word list appears as a dialogue turn. Figure 3 is a diagram for explaining an example of a dialogue turn.

[0081] In the case of Fig. 3, h1 makes an utterance including the important word iw1. At this point, since there is no other participant who has uttered this important word iw1, the dialogue turn extraction unit 13 judges the participant h1 to be a "first-time speaker." The dialogue turn extraction unit 13 considers that the utterances of all the participants after that utterance are continuing the dialogue regarding the important word iw1 uttered by the participant h1.

[0082] After a while of dialogue, participant h3 makes an utterance including the important word iw2. If the important word iw1 is not included in the utterances of all the participants after this, it is considered that the dialogue regarding iw1 ends here and the topic of the dialogue has shifted to iw2. Therefore, the dialogue turn extraction unit 13 considers the period from the utterance including the important word iw1 by the first participant h1, represented by a black circle, to the utterance immediately before the utterance including the important word iw2 by this participant h3, to be the dialogue turn regarding the important word iw1. Here, this is called turn 1.

[0083] Similarly, the dialogue turn extraction unit 13 also finds the dialogue turn about the important word iw2 uttered by the participant h3, and sets this as turn 2. After turn 2, another dialogue turn about the important word iw1 similar to turn 1 begins. The dialogue turn extraction unit 13 sets the participant h2, who first spoke about the important word iw1 in this dialogue turn, as the "repeat speaker." A dialogue turn is similarly found for turn 3.

[0084] In this way, the topic that is the focus of the dialogue may change from one to the next, but there may also be cases where another topic is mixed in midway and the focus gradually changes, or where a topic unrelated to consensus building is mixed in. Therefore, the dialogue turn extraction unit 13 eliminates these effects by setting an upper limit on the number of dialogues or an upper limit on the dialogue time.

[0085] Fig. 4 is a diagram for explaining another example of a dialogue turn. Fig. 4 shows an example in which a participant h1 speaks about an important word iw1, but then the topic turns to something unrelated to consensus building that does not include any of the words in the important word group.

[0086] In this case, if the dialogue turn of the important word iw1 is determined to be the period until the next important word appears, it may have a negative effect on the estimation of the evaluation. Therefore, if an utterance containing the important word or its related word does not appear within a specific number of dialogues or within a certain dialogue time, the dialogue turn extraction unit 13 separates the dialogue turn with the previous utterance. Note that related words are words that are closely related to the important word, such as co-occurring words in an utterance with the important word, or synonyms or similar words that can be extracted using external data.

[0087] 4 shows an example in which the upper limit of the dialogue count is set to four. Participant h1 makes an utterance including an important word iw1, and then participant h2 makes an utterance including a word related to the important word iw1. After that, the four utterances, which are the upper limit of the dialogue count, do not include an important word or a related word. Therefore, the dialogue turn extraction unit 13 extracts up to the utterance by participant h2 just before that as the dialogue turn (turn 4).

[0088] [Impact score calculation section] The influence score calculation unit 14 divides the transcript of the dialogue to be estimated, identifies the first speaker and the repeat speaker for each divided period, and calculates the influence score. The influence score calculation unit 14 calculates and outputs the influence score using each dialogue turn extracted by the dialogue turn extraction unit 13 and the dialogue data. The influence score calculation unit 14 uses the same important word list as the important word list used by the dialogue turn extraction unit 13.

[0089] First, there is a dialogue turn x About the important words of the dialogue turn u , key word iw u The first speaker of fp u , the repeat speaker in that dialogue turn is rp u Here, fp u =rp u In other words, the initial speaker and the repeat speaker may be the same person. Also, if the key word iw1 appears for the first time in this dialogue turn, there is no repeat speaker.

[0090] This dialogue turnx Repeated speaker rp u Key words by iw u The dialogue is triggered by an utterance that includes the word iw. u is the first speaker fp u Therefore, the impact score calculation unit 14 scores the impact and calculates the u and fp u The feature is assigned as

[0091] The influence score may be calculated using information on the dialogue carried out in the corresponding dialogue turn, and the details are not important. For example, the influence score calculation unit 14 may calculate the influence score using information on the dialogue carried out in the corresponding dialogue turn. x The number of utterances and / or number of words spoken by all participants in x The frequency of all participants' utterances and dialogue turns x Important words included in iw x The influence score is calculated based on the number of times and / or the number of people who have spoken the word and its related words.

[0092] In addition, the influence score calculation unit 14 calculates the dialogue turn x The dialogue turn immediately before x-1 Or, dialogue turn x For example, the last minute or five minutes before a conversation starts. x For example, the influence score calculation unit 14 may calculate the influence score by comparing the state of a dialogue turn with the state of a predetermined period before the start of the dialogue turn in order to see the change in the number of utterances of all participants. x From the number of utterances in x-1 The impact score is calculated using the value obtained by subtracting the number of utterances in

[0093] For example, the influence score calculation unit 14 calculates the dialogue turn x Time f x1 , the sum of the number of utterances by all participants f x2 Dialogue turn x Important words included in iw u and the number of utterances of its related words fx3 The feature amount is not limited to this, and may be any value that can be calculated for each turn from the dialogue data.

[0094] The influence score calculation unit 14 employs, as the influence score of each participant, the sum, variance, maximum value, minimum value, etc. of the feature amount of the turn in which the participant is the first speaker or the repeat speaker. At this time, the influence score calculation unit 14 may change the weight of the feature amount depending on whether the participant is the first speaker or the repeat speaker. For example, assume that participant h1 is the first speaker in dialogue turn turn1 and the repeat speaker in dialogue turn turn3.

[0095] If the sum of features is used as the influence score, then participant h i Impact score F i can be expressed, for example, as in equation (4).

[0096]

number

[0097] In this case, θ is a weight between the first speaker and the repeat speaker, and 0<θ≦. Note that in formula (4), the influence score is expressed by the sum of the feature amounts for the first speaker and the repeat speaker, but as described above, this is not limiting.

[0098] For example, the influence score calculation unit 14 may use, as the influence score, feature amounts that are separated into the cases of a first speaker and a repeated speaker.

[0099] For example, the feature quantity for the first speaker's turn is (Sum f ,Var f ,Max f ,Min f ) and the feature quantity in the repeated utterance turn is (Sum r ,Var r ,Max r ,Min r ), the impact score Fi can be expressed as a multidimensional vector shown in equation (5).

[0100]

number

[0101] [Dialogue Evaluation Calculation Part] The dialogue evaluation estimation unit 15 estimates a dialogue evaluation score based on the liveliness score, proactiveness score, and influence score calculated for the dialogue data of the estimation target.

[0102] The dialogue evaluation estimation unit 15 inputs the feature quantities of the activity score, proactiveness score, and influence score calculated for the dialogue data to be estimated into a trained model, and obtains the dialogue evaluation score output from the model. The model is configured by a neural network or the like, and is trained in a learning device 20, which will be described later. Note that the dialogue evaluation estimation unit 15 may estimate the dialogue evaluation score using not only the trained model, but also a mathematical formula or a threshold value.

[0103] [Estimation process] Next, a description will be given of the estimation process executed by the estimation device 10. Fig. 5 is a flowchart showing the procedure of the estimation process according to the embodiment.

[0104] As shown in FIG. 5, when the estimation device 10 receives input of dialogue data and a participant list to be estimated for the dialogue evaluation score (step S1), the estimation device 10 outputs the received dialogue data and participant list to the activity score calculation unit 11, the proactiveness score calculation unit 12, and the dialogue turn extraction unit 13.

[0105] The activity score calculation unit 11 performs an activity score calculation process to calculate and output an activity score for each participant of the dialogue based on the dialogue data and the participant list (step S2). The proactive score calculation unit 12 receives the dialogue data and the participant list and performs an proactive score calculation process to calculate and output an proactive score for each participant of the dialogue (step S3).

[0106] The dialogue turn extraction unit 13 performs a dialogue turn extraction process that divides the dialogue data into dialogue turns based on the dialogue data and the participant list, and outputs the start and end points of each dialogue turn and key words at that time (step S4).The influence score calculation unit 14 performs an influence score calculation process that calculates and outputs an influence score using each dialogue turn extracted by the dialogue turn extraction unit 13 and the dialogue data (step S5).

[0107] The dialogue evaluation estimation unit 15 performs a dialogue evaluation estimation process to estimate a dialogue evaluation score for each participant based on the feature values ​​of the activeness score, proactiveness score, and influence score calculated for the dialogue data to be estimated (step S6).The dialogue evaluation estimation unit 15 then outputs the estimated dialogue evaluation score (step S7) and ends the process.

[0108] [Positiveness score calculation process] Next, the procedure of the aggressiveness score calculation process (step S3) will be described. Fig. 6 is a flowchart showing an example of the procedure of the aggressiveness score calculation process shown in Fig. 5.

[0109] As shown in FIG. 6, the proactiveness score calculation unit 12 calculates the proactiveness score of a participant h i The dialogue is divided into m parts (step S11). The proactiveness score calculation unit 12 determines whether or not there is an unprocessed dialogue among the divided dialogues (step S12).

[0110] If there is an unprocessed dialogue among the divided dialogues (step S12: Yes), the proactiveness score calculation unit 12 calculates the number of participants h in each section for the unprocessed dialogue among the divided dialogues. i The speech (transcript of the conversation between the speaker and the speaker) is subjected to morphological analysis (step S13) and divided into words. The proactiveness score calculation unit 12 counts the total number of words, the number of nouns, verb forms, and adjectives, and the number of other words for each divided word (step S14).

[0111] The proactiveness score calculation unit 12 applies the counts of the total number of words, the number of nouns, verbs, and adjectives, and the number of other words to, for example, formula (2) for the divided dialogue to calculate the section proactiveness score of the participant (step S15). After processing step S15, the proactiveness score calculation unit 12 proceeds to step S12.

[0112] If there is no unprocessed dialogue among the divided dialogues (step S12: No), the proactiveness score calculation unit 12 calculates the proactiveness score of the participant h i Based on the aggressiveness scores of each interval, participant h i The proactiveness score of the user is calculated (step S16).

[0113] [Impact score calculation process] Next, the procedure of the influence score calculation process (step S5) will be described below. Fig. 7 is a flowchart showing an example of the procedure of the influence score calculation process shown in Fig. 5.

[0114] 7, the influence score calculation unit 14 divides the dialogue data to be subjected to influence score calculation into dialogue turns based on each dialogue turn and the dialogue data extracted by the dialogue turn extraction unit 13 (step S21). The influence score calculation unit 14 determines whether or not there is an unprocessed dialogue turn among the divided dialogue turns (step S22).

[0115] If there is an unprocessed dialogue turn among the divided dialogue turns (step S22: Yes), the influence score calculation unit 14 judges whether the participant is a first-time speaker or a repeat speaker (step S23). If the participant is not a first-time speaker or a repeat speaker (step S23: No), the influence score calculation unit 14 returns to step S22.

[0116] If the participant is a first-time speaker or a repeat speaker (step S23: Yes), the influence score calculation unit 14 performs morphological analysis on the dialogue that took place in the dialogue turn to be processed (step S24). The influence score calculation unit 14 calculates the sum of the number of utterances, the average number of utterances by the number of participants, and the number of utterances of important and its related words (step S25). After processing step S25, the influence score calculation unit 14 returns to step S22.

[0117] If there are no unprocessed dialogue turns among the divided dialogue turns (step S22: No), the influence score calculation unit 14 applies the sum of the number of utterances, the average number of utterances by the number of people, and the number of utterances of the important word and its related words to, for example, formula (4) or formula (5) to calculate the influence score (step S26).

[0118] [Learning device] Next, a description will be given of a learning device that trains a model used by the estimation device 10. Fig. 8 is a diagram illustrating an example of the configuration of a learning device according to an embodiment.

[0119] The learning device 20 shown in Fig. 8 is realized, for example, by loading a predetermined program into a computer or the like including a ROM, a RAM, a CPU, etc., and causing the CPU to execute the predetermined program. The learning device 20 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc. As shown in Fig. 8, the learning device 20 has an activity score calculation unit 21, an activeness score calculation unit 22, a dialogue turn extraction unit 23, an influence score calculation unit 24, and a dialogue evaluation model creation unit 25.

[0120] The learning device 20 receives, as input, dialogue data for training, a participant list, and a satisfaction evaluation value. The input of the learning device 20 is dialogue data for training and a participant list (h1,...,h k ), satisfaction evaluation value (S1, ,S k ) is a collection of sets of

[0121] The dialogue data includes at least a transcript of what each participant said, and may also include personal data of each participant obtained by a questionnaire survey. The satisfaction evaluation value is, for example, a value obtained by having the participant subjectively evaluate whether or not they were satisfied with the dialogue using an n-point scale in a questionnaire. In this case, the questionnaire may be conducted with multiple evaluation axes, such as proactiveness, influence, enjoyment of the dialogue, and satisfaction with the agreement, and the aggregated value may be used as the satisfaction evaluation value. In addition, an objective evaluation by a third party looking at the dialogue or dialogue data may be used as the satisfaction evaluation value.

[0122] The activity score calculation unit 21 has the same functions as the activity score calculation unit 11. The activity score calculation unit 21 calculates and outputs an activity score for each participant of the dialogue based on the inputted dialogue data for training and the participant list.

[0123] The proactiveness score calculation unit 22 has the same functions as the proactiveness score calculation unit 12. The proactiveness score calculation unit 22 calculates and outputs an proactiveness score for each participant in a dialogue based on the dialogue data for training and the participant list.

[0124] The dialogue turn extraction unit 23 has the same function as the dialogue turn extraction unit 13. Based on the training dialogue data and the participant list, the dialogue turn extraction unit 23 divides the training dialogue data into dialogue turns, and outputs the start and end points of each dialogue turn and key words at that time.

[0125] The influence score calculation unit 24 has the same function as the influence score calculation unit 14. The influence score calculation unit 24 calculates and outputs an influence score using each dialogue turn extracted by the dialogue turn extraction unit 23 and the training dialogue data.

[0126] The dialogue evaluation model creation unit 25 creates and outputs a model for estimating a dialogue evaluation using the activity score, proactiveness score, and influence score calculated using the training dialogue data. Here, an example of creating a model using machine learning is shown. The dialogue evaluation model creation unit 25 receives as input the activity score for each participant calculated by the activity score calculation unit 21, the proactiveness score calculated by the proactiveness score calculation unit 22, and the influence score calculated by the influence score calculation unit 24, and creates a model for estimating a dialogue evaluation score by machine learning using the satisfaction level based on the subjective evaluation of the participants for the training dialogue data as correct answer data.

[0127] The dialogue evaluation model creation unit 25 trains the model using a method such as Random Forest, SVM (Support Vector Machine), or Deep Learning. Any machine learning method may be used as the model training method as long as it is a method of weighting features using correct answer data. The dialogue evaluation model creation unit 25 appropriately uses correct answer data to tune the parameters of the model. The model output as a training result is used in the dialogue evaluation estimation unit 15 of the estimation device 10.

[0128] Note that the dialogue evaluation model creation unit 25 does not necessarily have to use machine learning to train the model. The dialogue evaluation model creation unit 25 may, for example, take correlations between the features of various scores and correct answer data, and create a model that calculates the dialogue evaluation score using only highly correlated features, or by calculating weights of the features according to the correlation. For example, the dialogue evaluation model creation unit 25 may create a model that estimates the level of the dialogue evaluation score based on whether a specific feature exceeds a threshold value or whether the weighted sum of the feature values ​​exceeds a threshold value.

[0129] The dialogue evaluation model creation unit 25 may weight the feature amounts in three ways: all feature amounts of the activity score, all feature amounts of the proactive score, and all feature amounts of the influence score, or may weight each feature amount of each score.i is expressed as equation (6).

[0130]

number

[0131] ω A ,ω p ,ω F If each score is a scalar value, it takes a scalar value, and if it is a vector value, it can be either a scalar value or a vector value. Also, for example, if it is known in advance from the personality traits or interviews of the participants that they place particular importance on any of the activity level, proactiveness, and influence level, adjustments such as increasing the weighting of that level can be made here.

[0132] In this embodiment, an example has been shown in which the dialogue evaluation score is a scalar value, but it may also be a vector value, and the dialogue evaluation may be determined by the magnitude of the vector, or the elements of each vector may be factors in the dialogue evaluation, such as "degree of agreement," "level of activity," and "level of influence."

[0133] [Learning process] Next, a description will be given of the learning process executed by the learning device 20. Fig. 9 is a flowchart showing the processing procedure of the learning process in the embodiment.

[0134] As shown in FIG. 9, the learning device 20 receives, as input, dialogue data for training, a participant list, and a satisfaction level evaluation value (step S31).

[0135] The activity score calculation unit 21 performs an activity score calculation process (step S32) to calculate and output an activity score for each participant in the dialogue based on the dialogue data for training and the participant list, similar to the process of step S2 shown in Fig. 5. The proactive score calculation unit 22 performs an proactive score calculation process (step S33) to calculate and output an proactive score for each participant in the dialogue, using the dialogue data for training and the participant list as input, similar to the process of step S3 shown in Fig. 5.

[0136] The dialogue turn extraction unit 23 performs a dialogue turn extraction process (step S34) in which the dialogue data is divided into dialogue turns based on the training dialogue data and the participant list, and the start and end points of each dialogue turn and key words at that time are output, similar to the process of step S4 shown in Fig. 5. The influence score calculation unit 24 performs an influence score calculation process (step S35) in which the influence score is calculated and output using each dialogue turn extracted by the dialogue turn extraction unit 13 and the training dialogue data, similar to the process of step S5 shown in Fig. 5.

[0137] The dialogue evaluation model creation unit 25 performs a dialogue evaluation model creation process to create a model for estimating a dialogue evaluation using the activity score, proactiveness score, and influence score calculated using the training dialogue data (step S36). The dialogue evaluation model creation unit 25 outputs the created model to the estimation device 10 (step S37) and ends the process.

[0138] [Effects of the embodiment] In this way, the estimation device 10 according to the embodiment accepts input of at least a transcript of the dialogue to be estimated as dialogue data, and calculates an activity score representing the degree to which the excitement of the dialogue affects the satisfaction of each participant based on the linguistic features of the utterances in the dialogue data. The estimation device 10 divides the transcript of the dialogue to be estimated along a time axis, and calculates an activeness score representing the degree to which the activeness of the participants' utterances in the dialogue affects the satisfaction of each participant based on the number of utterances and the number of words uttered by each participant in each section. The estimation device 10 divides the transcript of the dialogue to be estimated, identifies the first speaker and the repeat speaker for each divided period, and calculates an influence score representing the degree to which the utterances of the first speaker and the repeat speaker affect the process and results of consensus building and the satisfaction of each participant. The estimation device 10 estimates a dialogue evaluation score representing the evaluation of the dialogue by each participant based on the activity score, activeness score, and influence score.

[0139] The evaluation of a dialogue in a group of several people is influenced by how actively participants take part in the dialogue, whether their comments sparked an interesting conversation, and whether they promoted consensus building.

[0140] The estimation device 10 calculates an activeness score and an influence score for these, and further estimates a dialogue evaluation score to be combined with an activity score calculated from features of utterances and behaviors in the dialogue data. This enables the estimation device 10 to appropriately estimate the evaluation of the dialogue by each participant in a group dialogue, which reflects the satisfaction of each dialogue participant with the dialogue. Furthermore, the estimation device 10 identifies the first speaker and the repeat speaker, and estimates a dialogue evaluation score that appropriately reflects the influence that the utterances between the first speaker and the repeat speaker had on the process and results of consensus building.

[0141] In addition, the estimation device 10 calculates an activity score using non-verbal features based on the vital data of each participant and / or personal data representing the individuality of each participant, thereby enabling the estimation device 10 to estimate a dialogue evaluation score that better reflects the overall state of the dialogue.

[0142] Furthermore, the estimation device 10 can extract an utterance sentence in which a key word appears as a dialogue turn, and use the key word to more accurately identify a first speaker and a repeat speaker for each dialogue turn.

[0143] Furthermore, the estimation device 10 estimates the dialogue evaluation score using a trained model. This model is appropriately trained to estimate the dialogue evaluation score by machine learning using the activeness score, proactiveness score, and influence score calculated in the learning device 20 for each piece of training dialogue data as input and participants' subjective satisfaction with the training dialogue data as correct answer data, so that it is possible to realize accurate estimation of the dialogue evaluation score.

[0144] Then, using the dialogue evaluation score estimated by the estimation device 10, it is possible to estimate the satisfaction and achievement of the dialogue taking into consideration the proactiveness of the participants' comments and the promotion of consensus building through the content of their comments.

[0145] Furthermore, by using the evaluation estimated by the estimation device 10, it is possible to consider the formation of groups that will increase the satisfaction of the participants, or to intervene in the way the dialogue proceeds. In addition, by estimating the dialogue evaluation of the other party using the estimation device 10 after the dialogue, it is possible to consciously take necessary steps to increase the dialogue evaluation of the participant in the next dialogue, or to reflect them in materials. Alternatively, by virtually simulating a dialogue and using the estimation device 10 to observe changes in the dialogue evaluation of the dialogue participants in the simulation, it is possible to devise a strategy for what kind of dialogue should be conducted and what kind of intervention should be made to promote consensus building.

[0146] [System configuration of the embodiment] Each component of the estimation device 10 and the learning device 20 is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the estimation device 10 and the learning device 20 is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0147] Furthermore, all or any part of the processes performed in the estimation device 10 and the learning device 20 may be realized by a CPU, a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU. Furthermore, each process performed in the estimation device 10 and the learning device 20 may be realized as hardware using wired logic.

[0148] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the process procedures, control procedures, specific names, and information including various data and parameters described above and shown in the drawings can be changed as appropriate unless otherwise specified.

[0149] [program] 10 is a diagram showing an example of a computer in which a program is executed to realize the estimation device 10 and the learning device 20. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0150] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0151] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define each process of the estimation device 10 and the learning device 20 are implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing processes similar to the functional configurations of the estimation device 10 and the learning device 20 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0152] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary, and executes the program.

[0153] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0154] Although the embodiment of the invention made by the present inventor has been described above, the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to the present embodiment. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are all included in the scope of the present invention. [Explanation of symbols]

[0155] 10 Estimation device 11,21 Activity score calculation section 12,22 Aggression score calculation section 13,23 Dialogue turn extraction unit 14,24 Impact score calculation section 15 Dialogue evaluation estimation unit 20 Learning Device 25 Dialogue evaluation model creation department

Claims

1. a first calculation step of receiving at least a transcript of the dialogue as dialogue data of a plurality of people, and calculating an activity score representing a degree to which the enthusiasm of the dialogue affects the satisfaction of each participant based on linguistic features of utterances in the transcript of the dialogue; a second calculation step of dividing the transcript of the dialogue along a time axis, and calculating an assertiveness score representing the degree to which the assertiveness of the participants' comments in the dialogue affects the satisfaction of each participant based on the number of utterances and the number of words uttered by each participant in each section; a third calculation step of dividing the transcript of the dialogue, identifying first-time speakers and repeat speakers for each divided period, and calculating an influence score representing the degree to which the influence of the utterances of the first-time speaker and the repeat speaker on the process and results of consensus building has on the satisfaction level of each participant; estimating a dialogue evaluation score representing an evaluation of the dialogue by each participant based on the liveness score, the proactiveness score, and the influence score; The estimation method according to claim 1, further comprising:

2. The estimation method according to claim 1, characterized in that the first calculation step accepts input of vital data of each participant and / or personal data representing the individuality of each participant as the dialogue data, and calculates the activity score using non-verbal features based on the vital data of each participant and / or the personal data of each participant.

3. extracting, from the transcript of the dialogue, a dialogue turn that contains a key word that represents an idea or topic; 3. The method according to claim 1, wherein the third calculation step includes identifying the first speaker and the repeated speaker for each dialogue turn by using the key words.

4. the third calculation step includes obtaining the interaction evaluation score by using a model that estimates the interaction evaluation score by inputting the liveness score, the proactiveness score, and the influence score; The estimation method according to any one of claims 1 to 3, characterized in that the model is trained to estimate the dialogue evaluation score by machine learning using the activity score, the proactiveness score, and the influence score calculated for training dialogue data by a plurality of people as input, and satisfaction levels based on subjective evaluations of the participants with respect to the training dialogue data as correct answer data.

5. a fourth calculation step of receiving at least a transcript of the dialogue as training dialogue data for a plurality of people, and calculating an activity score representing the degree to which the enthusiasm of the dialogue affects the satisfaction of each participant based on linguistic features of utterances in the transcript of the dialogue; a fifth calculation step of dividing the transcript of the dialogue along a time axis, and calculating an assertiveness score representing the degree to which the assertiveness of the participants' comments in the dialogue affects the satisfaction of each participant based on the number of utterances and the number of words uttered by each participant in each section; a sixth calculation step of dividing the transcript of the dialogue, identifying a first speaker and a repeat speaker for each divided period, and calculating an influence score representing the degree to which the influence of the utterances of the first speaker and the repeat speaker on the process and result of the consensus building has on the satisfaction level of each participant; a creation step of creating a model for estimating a dialogue evaluation score representing an evaluation of the dialogue by each participant by machine learning using the activity score, the proactiveness score, and the influence score as input and using the satisfaction level based on the subjective evaluation of the training dialogue data as correct answer data; A learning method comprising:

6. a first calculation unit that receives at least a transcript of the dialogue as dialogue data of a plurality of people, and calculates an activity level score that indicates a degree to which the enthusiasm of the dialogue affects the satisfaction of each participant based on linguistic features of utterances in the transcript of the dialogue; a second calculation unit that divides the transcript of the dialogue along a time axis, and calculates an assertiveness score that indicates the degree to which the assertiveness of the participants' comments in the dialogue affects the satisfaction of each participant based on the number of utterances and the number of words uttered by each participant in each section; a third calculation unit that divides the transcript of the dialogue into sections, identifies a first speaker and a repeat speaker for each section, and calculates an influence score that indicates the degree to which the influence of the utterances of the first speaker and the repeat speaker on the process and result of the consensus building has on the satisfaction level of each participant; an estimation unit that estimates a dialogue evaluation score representing an evaluation of the dialogue by each participant based on the liveliness score, the proactiveness score, and the influence score; An estimation device comprising:

7. a first calculation step of receiving at least a transcript of the dialogue as dialogue data of a plurality of people, and calculating an activity score representing a degree to which the enthusiasm of the dialogue affects the satisfaction of each participant based on linguistic features of utterances in the transcript of the dialogue; a second calculation step of dividing the transcript of the dialogue along a time axis, and calculating an assertiveness score representing the degree to which the assertiveness of the participants' comments in the dialogue affects the satisfaction of each participant based on the number of utterances and the number of words uttered by each participant in each section; a third calculation step of dividing the transcript of the dialogue, identifying first-time speakers and repeat speakers for each divided period, and calculating an influence score representing the degree to which the influence of the utterances of the first-time speaker and the repeat speaker on the process and results of consensus building has on the satisfaction level of each participant; estimating a dialogue evaluation score representing an evaluation of the dialogue by each participant based on the liveness score, the proactiveness score, and the influence score; An estimation program for causing a computer to execute the above.

Images