Persuasiveness estimation device, persuasiveness estimation method, and persuasiveness estimation program

Abstract

To estimate persuasive power of an attendant at a group conference.SOLUTION: An input information generation part 16 uses a first activation function to generate input information including a feature quantity of an action in a conversation of a conversation attendant based upon voice information, language information, video information, and talk structure information on all conference attendants at a conference. An estimation part 17 uses a second activation function for the input information generated by the input information generation part 16 to acquire an evaluation value of persuasive power, and generates a class of order of persuasive power among conversation attendants based upon the acquired evaluation value.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a persuasiveness estimation device, a persuasiveness estimation method, and a persuasiveness estimation program.

Background Art

[0002] Conventionally, as a technique for estimating the persuasiveness of participants in a group meeting, a technique has been proposed in which persuasiveness is defined as the degree to which a viewer who has watched a monologue of a speaker has been persuaded, and persuasiveness is estimated according to that definition.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] ​However, in a group meeting with multiple participants, the persuasiveness of each participant is evaluated by comparing their persuasiveness with that of the other participants. Therefore, simply defining persuasiveness based on the degree to which viewers who listened to the narrator's monologue were persuaded does not take into account interpersonal comparisons, making it difficult to estimate the persuasiveness of each participant in a group meeting.

[0005] The present invention has been made in view of the above, and has as its object to enable estimation of the persuasiveness of participants in a group conference. [Means for solving the problem]

[0006] To solve the above-mentioned problems and achieve the object, an inference control device includes an input information generation unit and an estimation unit. The input information generation unit generates input information including behavioral features of the conversation participants in a conference using a first activation function based on audio information, linguistic information, video information, and discourse structure information of all conversation participants. The estimation unit obtains a persuasiveness evaluation value for the input information generated by the input information generation unit using a second activation function, and generates a persuasiveness ranking class among the conversation participants based on the obtained evaluation value. [Effects of the Invention]

[0007] According to the present invention, it is possible to estimate the persuasiveness of participants in a group meeting. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of a persuasiveness estimation device according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an outline of the process executed by the persuasiveness estimation device. [Figure 3] FIG. 3 is a flowchart of the persuasiveness estimation process according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a computer that executes a persuasiveness estimation program. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of a persuasiveness estimation device, a persuasiveness estimation method, and a persuasiveness estimation program disclosed in the present application will be described in detail with reference to the drawings. Note that the persuasiveness estimation device, the persuasiveness estimation method, and the persuasiveness estimation program disclosed in the present application are not limited to the following embodiment.

[0010] [Configuration of the persuasiveness estimation device] The configuration of the persuasive power estimation device 1 will be described with reference to FIG. 1. FIG. 1 is a block diagram of the persuasive power estimation device according to the embodiment. The persuasive power estimation device 1 is an information processing device such as a server. The persuasive power estimation device 1 is a device that estimates the persuasive power of a person to be estimated who participated in a conversation in a group meeting in which multiple members participate. As shown in FIG. 1, the persuasive power estimation device 1 is connected to a data input device 2.

[0011] The data input device 2 acquires data on the estimated target, who is a conversation participant who participated in the conversation in the group conference for one minute, and other participants, who are other participants, and transmits the data to the persuasiveness estimation device 1. For example, the data input device 2 has a camera and a microphone. The data input device 2 transmits one minute of image data of the group conference captured by the camera and one minute of audio data collected by the microphone to the persuasiveness estimation device 1. In addition, the data input device 2 may have a function to edit image data and audio data in response to instructions from a user.

[0012] 1, the persuasiveness estimation device 1 includes a data acquisition unit 11, a video information processing unit 12, a voice information processing unit 13, a language information processing unit 14, a discourse structure information generation unit 15, an input information generation unit 16, an estimation unit 17, and an output unit 18. The following describes an example in which a group conference has a total of four conversation participants.

[0013] The data acquisition unit 11 acquires from the data input device 2 one minute of image data and voice data for four people in a group conference.

[0014] The data acquisition unit 11 then acquires facial feature values ​​of each of the four conversation participants in a one-minute segment from the image data. In this embodiment, the facial feature values ​​are six-dimensional data including information on the head position and head rotation on the video expressed in three-dimensional space. For example, the data acquisition unit 11 can obtain the facial feature values ​​as output values ​​of a video-based face tracker based on the image data. The data acquisition unit 11 then outputs video information including the facial feature values ​​of each of the four conversation participants in a one-minute segment to the video information processing unit 12.

[0015] The data acquisition unit 11 also acquires mel spectrograms, which are acoustic features of each of the four conversation participants, from the speech data in a one-minute section. In this embodiment, the mel spectrograms are 64-dimensional data with a frame rate of 50 fps, i.e., 3000 frames per minute, which is the analysis unit. The data acquisition unit 11 then outputs speech information including the mel spectrograms of each of the four conversation participants in the one-minute section to the speech information processing unit 13.

[0016] The data acquisition unit 11 also acquires linguistic information obtained by transcribing the utterances of each of the four conversation participants in a one-minute section from the audio data.The data acquisition unit 11 then outputs the linguistic information obtained by transcribing the utterances of each of the four conversation participants in a one-minute section to the linguistic information processing unit 14.

[0017] The data acquiring unit 11 also acquires time interval information of utterances of each of the four conversation participants in a one-minute period from the voice data, and outputs the time interval information of utterances of each of the four conversation participants in a one-minute period to the discourse structure information generating unit 15.

[0018] The video information processing unit 12 has a video information encoder that vectorizes the video information. The video information encoder is implemented, for example, by a gated recurrent unit (GRU).

[0019] The video information processor 12 receives input video information, which is six-dimensional data representing head position and head rotation in three-dimensional space, from the data acquisition unit 11. Next, the video information processor 12 averages the input video information every 180 frames (180 frames corresponds to six seconds). Next, the video information processor 12 inputs the averaged frames to a two-layer bidirectional GRU, which is an encoder for the video information. The video information processor 12 then obtains the final hidden state of the second layer of the GRU as the output of the encoder, and generates it as video vector data. The video vector data has a size of 6x2. The video information processor 12 then outputs the video vector data for each of the four conversation participants to the input information generation unit 16.

[0020] The audio information processing unit 13 has an audio information encoder that vectorizes the audio information. The audio information encoder is implemented by, for example, a GRU.

[0021] The speech information processing unit 13 receives input of speech information, which is 64-dimensional data, from the data acquisition unit 11. Next, the speech information processing unit 13 averages the input speech information every 300 frames (300 frames correspond to 6 seconds). Next, the speech information processing unit 13 inputs the averaged frames to a two-layer bidirectional GRU, which is an encoder for speech information. Then, the speech information processing unit 13 obtains the final hidden state of the second layer of the GRU as the output of the encoder and uses it as speech vector data. The speech vector data has a size of 64 x 2. Thereafter, the speech information processing unit 13 outputs the speech vector data of each of the four conversation participants to the input information generation unit 16.

[0022] The language information processing unit 14 has a language information encoder that vectorizes the language information. The language information encoder is implemented by, for example, a GRU.

[0023] The language information processing unit 14 receives input of language information transcribed from each of the four conversation participants over a one-minute period from the data acquisition unit 11. Next, the language information processing unit 14 generates 768-dimensional vector data for each utterance generated over the one-minute period using a natural language processing model such as BERT (Bidirectional Encoder Representations from Transformers). Next, the language information processing unit 14 inputs the 768-dimensional vector data corresponding to each generated utterance into a two-layer bidirectional GRU, which is an encoder for language information. The language information processing unit 14 then obtains the final hidden state of the second layer of the GRU as the output of the encoder, and uses this as language vector data. The language vector data has a size of 768 × 2. Thereafter, the language information processing unit 14 outputs the language vector data for each of the four conversation participants to the input information generation unit 16.

[0024] The discourse structure information generation unit 15 receives input of time interval information on utterances made by each of the four conversation participants in a one-minute period from the data acquisition unit 11. Next, the discourse structure information generation unit 15 acquires the number of utterances made by each of the four conversation participants in one minute and the length of each utterance from the time interval information. Next, the discourse structure information generation unit 15 generates discourse structure information, which is vector data representing the number of utterances and the length of each utterance made by each of the four conversation participants in one minute.

[0025] For example, the discourse structure information generation unit 15 prepares eight bins based on utterance length. The eight bins are, for example, a bin for utterance lengths of 0 to less than 1 second, a bin for utterance lengths of 1 to less than 2 seconds, a bin for utterance lengths of 2 to less than 3 seconds, a bin for utterance lengths of 3 to less than 5 seconds, a bin for utterance lengths of 5 to less than 8 seconds, a bin for utterance lengths of 8 to less than 10 seconds, a bin for utterance lengths of 10 to less than 15 seconds, and a bin for utterance lengths of 10 to less than 15 seconds. Next, the discourse structure information generation unit 15 inputs utterances generated within one minute into each bin. Thereafter, the discourse structure information generation unit 15 counts the number of utterances included in each bin. This allows the discourse structure information generation unit 15 to generate discourse structure information, which is eight-dimensional vector data.

[0026] Thereafter, the discourse structure information generating unit 15 outputs the generated discourse structure information for each of the four conversation participants to the input information generating unit 16.

[0027] The input information generation unit 16 has a fully connected layer whose activation function is ReLU (Rectified Linear Unit). The input information generation unit 16 receives input of video vector data of each of the four conversation participants from the video information processing unit 12. The input information generation unit 16 also receives input of audio vector data of each of the four conversation participants from the audio information processing unit 13. The input information generation unit 16 also receives input of language vector data of each of the four conversation participants from the language information processing unit 14. The input information generation unit 16 also receives input of discourse structure information of each of the four conversation participants from the discourse structure information generation unit 15.

[0028] The input information generation unit 16 generates input information for the estimated subject as described below. The input information generation unit 16 concatenates the video vector data, audio vector data, language vector data, and discourse structure information of the estimated subject to create a multimodal tensor for the estimated subject. Next, the input information generation unit 16 inputs the generated multimodal tensor for the estimated subject to a fully connected layer whose activation function is ReLU, and obtains a 100-dimensional estimated subject vector.

[0029] The input information generation unit 16 also generates input information regarding other participants as described below. For each other participant, the input information generation unit 16 concatenates video vector data, audio vector data, language vector data, and discourse structure information to create a multimodal tensor for each other participant. Next, the input information generation unit 16 inputs the generated multimodal tensors for each other participant into a fully connected layer whose activation function is ReLU, thereby obtaining a 100-dimensional other participant vector for each other participant. The input information generation unit 16 then concatenates the other participant vectors for three participants to create a multiparty tensor. Next, the input information generation unit 16 inputs the multiparty tensor into a fully connected layer whose activation function is ReLU, thereby obtaining a 100-dimensional other participant combined vector. Here, the fully connected layer that inputs the multiparty tensor may have the same structure as the fully connected layer that inputs the multimodal tensor, or may have a different structure.

[0030] As described above, the input information generation unit 16 generates input information including behavioral features of the conversation participants in the conversation using a first activation function that is ReLU, based on the speech information, linguistic information, video information, and discourse structure information of all the conversation participants in the conference. More specifically, the input information generation unit 16 generates first input information for the presumed target person based on the speech information, linguistic information, video information, and discourse structure information of the presumed target person, with one of the conversation participants being considered as a presumed target person and the other conversation participants being considered as other participants, and generates second input information summarizing the other participants based on the speech information, linguistic information, video information, and discourse structure information of all the other participants.

[0031] Thereafter, the input information generation unit 16 outputs the 100-dimensional estimation target vector and the 100-dimensional other participant combined vector to the estimation unit 17.

[0032] The estimation unit 17 has a fully connected layer whose activation function is softmax. The estimation unit 17 receives an input of a 100-dimensional estimation target vector and a 100-dimensional other participant connection vector from the input information generation unit 16.

[0033] Next, the estimation unit 17 concatenates the estimation target person vector and the other participant combined vector to generate combined data, which is 200-dimensional vector data. Next, the estimation unit 17 inputs the combined data to a fully connected layer whose activation function is softmax. Thereafter, the estimation unit 17 obtains the output from the fully connected layer as the estimation result of the persuasiveness of the estimation target person. The estimation unit 17 estimates persuasiveness as an n-value classification problem (n is a natural number greater than or equal to 2), and obtains the likelihood of the persuasiveness class label as the output of the fully connected layer.

[0034] For example, a case will be described in which evaluators assign persuasiveness ratings using a number ranging from 1 to 7. In this case, the estimation unit 17 classifies the numbers using the following method. For example, the estimation unit 17 obtains the persuasiveness ratings of all participants in a one-minute segment and converts the numbers into rankings. For example, if the persuasiveness ratings of four participants are 5, 7, 3, and 1, respectively, the estimation unit 17 ranks the participants as second, first, third, and fourth, respectively. Next, the estimation unit 17 classifies the rankings. For example, the estimation unit 17 classifies the first and second places as high persuasiveness and the others as low persuasiveness. In this case, the estimation unit 17 outputs the likelihood of two types of persuasiveness class labels as estimation results.

[0035] As described above, the estimation unit 17 generates classes of persuasiveness rankings among conversation participants using the second activation function, which is softmax, for the input information generated by the input information generation unit 16. More specifically, the estimation unit 17 estimates to which class the person to be estimated belongs, using the first input information of the person estimating and the second input information of all other participants. For example, the estimation unit 17 obtains an evaluation value of the persuasiveness of each conversation participant using the second activation function for the input information, and generates the classes based on the obtained evaluation values.

[0036] The output unit 18 acquires the estimation result from the estimation unit 17. Then, the output unit 19 displays the estimation result on an output device such as a monitor, and notifies the user of the estimation result.

[0037] Here, the functions of the persuasive power estimation device 1 will be summarized with reference to Fig. 2. Fig. 2 is a diagram showing an outline of the processing executed by the persuasive power estimation device. Block 100 in Fig. 2 represents processing of the behavior of the person to be estimated. Blocks 110, 120, and 130 represent processing of the behavior of three other participants, the first to third participants.

[0038] In block 100, video information processing unit 12 receives input of video information 101, which is six-dimensional data representing the head position and head rotation of the person to be estimated in three-dimensional space, from data acquisition unit 11. Next, video information processing unit 12 averages video information 101 every 180 frames and inputs it to GRU 105, a two-layer bidirectional GRU. Then, video information processing unit 12 acquires video vector data of the person to be estimated, which is 6×2-dimensional data, using the final hidden state of the second layer of the GRU as the output of the encoder.

[0039] Similarly, in block 100, the speech information processing unit 13 receives input of speech information 102, which is 64-dimensional data of the person to be estimated, from the data acquisition unit 11. Next, the speech information processing unit 13 averages the speech information 102 every 300 frames and inputs it to GRU 106, a two-layer bidirectional GRU. Then, the speech information processing unit 13 acquires speech vector data of the person to be estimated, which is 64 × 2-dimensional data, using the final hidden state of the second layer of the GRU as the output of the encoder.

[0040] Similarly, in block 100, the linguistic information processing unit 14 acquires linguistic information 103, which is a transcription of the utterances of the target person in a one-minute section, and generates 768-dimensional vector data for each utterance using a natural language processing model. While FIG. 2 illustrates the linguistic information 103 with a 768-dimensional footnote for convenience, this actually represents the dimensions after the linguistic information 103 is converted into vector data. Next, the linguistic information processing unit 14 inputs the 768-dimensional vector data corresponding to each generated utterance to GRU 107, a two-layer bidirectional GRU. The linguistic information processing unit 14 then acquires the linguistic vector data of the target person in the form of 768 x 2-dimensional data, using the final hidden state of the second layer of the GRU as the output of the encoder.

[0041] Similarly, in block 100, the discourse structure information generation unit 15 acquires the number of utterances per minute by the estimated subject and the length of each utterance from the audio data and image data. Next, the discourse structure information generation unit 15 generates discourse structure information 104, which is vector data representing the number of utterances per minute and the length of each utterance by the estimated subject.

[0042] Similarly, in block 100, the input information generation unit 16 concatenates the video vector data, audio vector data, language vector data, and discourse structure information of the person to be estimated to create a multimodal tensor 108 of the person to be estimated. Next, the input information generation unit 16 inputs the generated multimodal tensor 108 of the person to be estimated to a fully connected layer 109 whose activation function is ReLU, and obtains the person to be estimated as 100-dimensional data.

[0043] Also, in block 110, video information processor 12 receives input of video information 111, which is six-dimensional data representing the head position and head rotation of the first other participant in three-dimensional space, from data acquisition unit 11. Next, video information processor 12 averages video information 111 every 180 frames and inputs it to GRU 115, a two-layer bidirectional GRU. Then, video information processor 12 acquires video vector data of the first other participant, which is 6x2-dimensional data, using the final hidden state of the second layer of the GRU as the output of the encoder. Here, GRU 115 may be the same as GRU 105.

[0044] Similarly, in block 110, the audio information processing unit 13 receives input of audio information 112, which is 64-dimensional data of the first other participant, from the data acquisition unit 11. Next, the audio information processing unit 13 averages the audio information 112 every 300 frames and inputs it to the GRU 116, which is a two-layer bidirectional GRU. Then, the audio information processing unit 13 obtains the audio vector data of the first other participant, which is 64 x 2-dimensional data, using the final hidden state of the second layer of the GRU as the output of the encoder. Here, the GRU 116 may be the same as the GRU 106.

[0045] Similarly, in block 110, the language information processing unit 14 acquires language information 113, which is a transcription of the utterances of the first other participant, and generates 768-dimensional vector data for each utterance using a natural language processing model. Next, the language information processing unit 14 inputs the 768-dimensional vector data corresponding to each generated utterance to GRU 117, which is a two-layer bidirectional GRU. Then, the language information processing unit 14 acquires the language vector data of the first other participant, which is 768 x 2-dimensional data, using the final hidden state of the second layer of the GRU as the output of the encoder. Here, GRU 117 may be the same as GRU 107.

[0046] Similarly, in block 110, the discourse structure information generation unit 15 acquires the number of utterances per minute by the first other participant and the length of each utterance from the audio data and image data. Next, the discourse structure information generation unit 15 generates discourse structure information 114, which is vector data representing the number of utterances per minute and the length of each utterance by the first other participant.

[0047] Similarly, in block 110, the input information generation unit 16 concatenates the video vector data, audio vector data, language vector data, and discourse structure information of the first other participant to create a multimodal tensor 118 of the first other participant. Next, the input information generation unit 16 inputs the generated multimodal tensor 118 of the person to be estimated to a fully connected layer 119 whose activation function is ReLU, and obtains an other participant vector of the first other participant as 100-dimensional data. Here, the fully connected layer 119 may be the same as the fully connected layer 109.

[0048] The input information generating unit 16 also performs the same process on the blocks 120 and 130 as on the block 110, and acquires the other participant vectors of the second and third other participants of the 100-dimensional data, respectively.

[0049] Next, the input information generation unit 16 concatenates the other participant vectors of the first to third other participants, which are 100-dimensional data, output from the blocks 110, 120, and 130, to create a multi-party tensor 201. Next, the input information generation unit 16 inputs the multi-party tensor 201 to a fully connected layer 202 whose activation function is ReLU, and acquires the other participant joint vector, which is 100-dimensional data.

[0050] The estimation unit 17 generates linked data 301, which is 200-dimensional vector data, by linking the estimation target person vector and the other participant linked vector. Next, the estimation unit 17 inputs the linked data 301 to a fully connected layer 302 whose activation function is softmax. Thereafter, the estimation unit 17 obtains an output from the fully connected layer 302 as a persuasiveness estimation result 303 of the estimation target person.

[0051] [Persuasiveness estimation processing] Next, the overall flow of the persuasive power estimation process performed by the persuasive power estimation device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart of the persuasive power estimation process according to the embodiment.

[0052] The video information processing unit 12, the audio information processing unit 13, the language information processing unit 14, and the discourse structure information generating unit 15 acquire data on the person to be estimated (step S1).

[0053] The video information processing unit 12 inputs the video information into a two-layer bidirectional GRU on average every 180 frames, and performs video processing to obtain video vector data of the person to be estimated using the final hidden state of the second layer of the GRU as the output of the encoder (step S2).

[0054] The audio information processing unit 13 averages audio information every 300 frames and inputs it into a two-layer bidirectional GRU, and performs audio processing to obtain audio vector data of the person to be estimated using the final hidden state of the second layer of the GRU as the output of the encoder (step S3).

[0055] The language information processing unit 14 inputs the 768-dimensional vector data generated for the language information using a natural language processing model into a two-layer bidirectional GRU, and performs language processing to obtain the language vector data of the person to be estimated using the final hidden state of the second layer of the GRU as the output of the encoder (step S4).

[0056] The discourse structure information generation unit 15 acquires the number of utterances per minute by the estimated subject and the length of each utterance, and generates discourse structure information, which is vector data representing the number of utterances per minute and the length of each utterance by the estimated subject (step S5).

[0057] Thereafter, the input information generating unit 16 creates a multimodal tensor of the estimated subject by linking the video vector data, audio vector data, language vector data, and discourse structure information of the estimated subject (step S6).

[0058] Next, the input information generating unit 16 inputs the generated multimodal tensor of the person to be estimated to a fully connected layer whose activation function is ReLU, and acquires a vector of the person to be estimated (step S7).

[0059] Furthermore, the video information processing unit 12, the audio information processing unit 13, the language information processing unit 14, and the discourse structure information generating unit 15 acquire data on the other participants (step S8).

[0060] The video information processing unit 12 inputs the video information into a two-layer bidirectional GRU on average every 180 frames, and performs video processing to obtain the video vector data of other participants using the final hidden state of the second layer of the GRU as the output of the encoder (step S9).

[0061] The audio information processing unit 13 averages audio information every 300 frames and inputs it into a two-layer bidirectional GRU, and performs audio processing to obtain audio vector data of other participants using the final hidden state of the second layer of the GRU as the output of the encoder (step S10).

[0062] The language information processing unit 14 inputs 768-dimensional vector data generated for the language information using a natural language processing model into a two-layer bidirectional GRU, and performs language processing to obtain the language vector data of other participants using the final hidden state of the second layer of the GRU as the output of the encoder (step S11).

[0063] The discourse structure information generation unit 15 acquires the number of utterances per minute and the length of each utterance by other participants, and generates discourse structure information, which is vector data representing the number of utterances per minute and the length of each utterance by other participants (step S12).

[0064] Thereafter, the input information generating unit 16 creates a multimodal tensor for the other participants by linking the video vector data, audio vector data, language vector data, and discourse structure information of the other participants (step S13).

[0065] Next, the input information generating unit 16 inputs the generated multimodal tensors of the other participants to a fully connected layer whose activation function is ReLU, and acquires the other participant vectors (step S14).

[0066] Next, the input information generation unit 16 determines whether or not the other participant vectors of all other participants have been acquired (step S15). If there is another participant whose other participant vector has not yet been acquired (step S15: No), the input information generation unit 16 returns to step S8.

[0067] On the other hand, if the other participant vectors of all other participants have been acquired (step S15: Yes), the input information generation unit 16 concatenates all other participant vectors to create a multi-party tensor (step S16).

[0068] Next, the input information generation unit 16 inputs the multi-party tensor to a fully connected layer whose activation function is ReLU, and acquires other participant connection vectors (step S17).

[0069] The estimation unit 17 generates linked data, which is 200-dimensional vector data, by linking the estimation target vector and the other participant linked vector (step S18).

[0070] Next, the estimation unit 17 inputs the connected data to a fully connected layer whose activation function is a softmax function, and acquires an output from the fully connected layer as a persuasiveness estimation result of the estimation target person (step S19).

[0071] [Effects of persuasiveness estimation device and persuasiveness estimation process] As described above, the persuasiveness estimation device according to this embodiment receives as input information audio information such as mel spectrograms of all conversation participants, linguistic information such as transcripts of all conversation participants' utterances, video information such as output values ​​of a video-based face tracker, and discourse structure information such as the amount of speech of each participant. The persuasiveness estimation device then estimates the persuasiveness of the person being estimated from the input information using an estimator employing deep learning technology. Furthermore, in estimating persuasiveness, the persuasiveness estimation device redefines the persuasiveness evaluation value as a ranking class of persuasiveness among the participants through preprocessing in order to take into account implicit relative evaluations between participants when an evaluator evaluates the persuasiveness of each participant. Specifically, the ranking class of persuasiveness among the participants means replacing the persuasiveness of all participants with a ranking and classifying the persuasiveness into a high persuasive class and a low persuasive class based on the ranking.

[0072] This improves the performance of persuasiveness estimation for a group compared to directly estimating persuasiveness values ​​or classifying persuasiveness values ​​without considering the rankings of participants, thereby enabling appropriate estimation of the persuasiveness of each participant. Here, directly estimating persuasiveness values ​​means, for example, when the evaluator's evaluation of persuasiveness is expressed as a number from 1 to 7, directly estimating the numerical value of the persuasiveness of the participant (e.g., "6"). Furthermore, classifying persuasiveness values ​​without considering the rankings of participants means, for example, when the evaluator's evaluation of persuasiveness is expressed as a number from 1 to 7, classifying 1 to 3 points as a low class, 4 points as a medium class, and 5 to 7 points as a high class.

[0073] In addition, in the past, when evaluating the persuasiveness of a specific speaker, estimation was performed using data on that specific speaker without using data on other speakers. In contrast, the persuasiveness estimation device according to the present embodiment evaluates the persuasiveness of a specific speaker using data on all conversation participants, including the speaker to be evaluated and the other speakers, and therefore can also take into account the reactions of others to the speaker's utterances, thereby improving the performance of persuasiveness estimation.

[0074] [System configuration, etc.] Furthermore, the components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0075] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0076] [program] In one embodiment, the persuasive power estimation device 1 can be implemented by installing a persuasive power estimation program that executes the above information processing as package software or online software on a desired computer. For example, by executing the above persuasive power estimation program on an information processing device, the information processing device can function as the persuasive power estimation device 1. The information processing device referred to here includes desktop and notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handy-phone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).

[0077] The persuasive power estimation device 1 can also be implemented as a persuasive power estimation server device that treats a terminal device used by a user as a client and provides the client with services related to the persuasive power estimation process. For example, the persuasive power estimation device is implemented as a server device that receives data of each participant in a group conference as input and provides a persuasive power estimation service that performs persuasive power estimation processing. In this case, the persuasive power estimation server device may be implemented as a web server or as a cloud that provides services related to the persuasive power estimation process through outsourcing.

[0078] 4 is a diagram showing an example of a computer that executes a persuasiveness estimation program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0079] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0080] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a persuasiveness estimation program that defines each process of the persuasiveness estimation device 1, which has the same functions as the persuasiveness estimation device 1, is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing processes similar to the functional configuration of the persuasiveness estimation device 1 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0081] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0082] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070. [Explanation of symbols]

[0083] 1. Persuasiveness Estimation Device 2 Data input device 11 Data Acquisition Section 12 Video information processing section 13 Audio information processing section 14 Language Information Processing Section 15 Discourse structure information generation unit 16 Input information generation unit 17 Estimation part 18 Output section

Claims

1. an input information generation unit that generates first input information including behavioral features in a conversation of the estimated target person using a first activation function based on speech information, linguistic information, video information, and discourse structure information of the estimated target person, where one of all conversation participants in a conference is an estimated target person and the other conversation participants are other participants, and generates second input information including behavioral features in a conversation of the estimated target person using the first activation function based on the speech information, linguistic information, video information, and discourse structure information of all the other participants; an estimation unit that generates linked data by linking the first input information and the second input information generated by the input information generation unit, inputs the linked data to a fully connected layer whose activation function is a second activation function, and estimates the class of the ranking of persuasiveness between the conversation participants to which the estimation target belongs, using the likelihood of a label of the class of the ranking of persuasiveness between the conversation participants output from the fully connected layer; A persuasiveness estimation device comprising:

2. The persuasiveness estimation device according to claim 1, characterized in that the estimation unit obtains an evaluation value of persuasiveness for each of the conversation participants using the second activation function for the linked data, and generates the class of persuasiveness ranking among the conversation participants based on each of the obtained evaluation values.

3. The persuasiveness estimation device according to claim 1, characterized in that the input information generation unit generates behavioral features in the conversation for each of the other participants using the first activation function based on the speech information, the linguistic information, the video information, and the discourse structure information of each of the other participants, and generates the second input information using the first activation function for data concatenating the behavioral features in the conversation for each of the other participants.

4. the input information generation unit uses ReLU as the first activation function, The estimation unit uses Softmax as the second activation function.

4. The persuasiveness estimation device according to claim 1, wherein:

5. a video information processing unit that generates video vector data based on the facial feature values ​​of the video information of each of the conversation participants; a voice information processing unit that generates voice vector data based on mel spectrograms, which are the voice information of each of the conversation participants; a language information processing unit that generates language vector data based on transcriptions of utterances, which are the language information of each of the conversation participants; a discourse structure information generating unit that generates vector data representing the number and length of utterances of each of the conversation participants as the discourse structure information, The input information generation unit generates the first input information and the second input information based on the video vector data generated by the video information processing unit, the audio vector data generated by the audio information processing unit, the language vector data generated by the language information processing unit, and the discourse structure information generated by the discourse structure information generation unit.

5. The persuasiveness estimation device according to claim 1, wherein:

6. A persuasiveness estimation method executed by a persuasiveness estimation device, a generation step of generating first input information including behavioral features in the conversation of the estimated target person using a first activation function based on speech information, linguistic information, video information, and discourse structure information of the estimated target person, where one of all conversation participants in the conference is an estimated target person and the other conversation participants are other participants, and generating second input information including behavioral features in the conversation of the estimated target person using the first activation function based on the speech information, linguistic information, video information, and discourse structure information of all the other participants; an estimation step of generating linked data by linking the first input information and the second input information, inputting the linked data to a fully connected layer whose activation function is a second activation function, and estimating the class of the ranking of persuasiveness between the conversation participants to which the estimation target belongs, using the likelihood of a label of the class of the ranking of persuasiveness between the conversation participants output from the fully connected layer; A method for estimating persuasiveness, comprising:

7. A persuasiveness estimation program for causing a computer to function as the persuasiveness estimation device according to any one of claims 1 to 5.

Images