Analysis device, analysis method, and program

Abstract

To provide a technique for investigating and analyzing effects of various conditions related to work environments and workers on a process of constructing a common foundation, when constructing the common foundation in dialogue between humans.SOLUTION: An analysis device that analyzes effects on a process of constructing a common foundation including knowledge shared between participants in dialogue includes: an acquisition unit that acquires corpus data related to dialogue contents of the participants in executing a task; a statistical data extraction unit that outputs, from the corpus data, statistical data classified into a set of conditions related to modality of the dialogue and social relationships of the participants; a task pattern classification unit that classifies the statistical data into a task success pattern or a task failure pattern and outputs a classification result; an analysis unit that generates visualization information obtained by visualizing the classification result by analyzing the classification result; and an output unit that outputs the visualization information.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a technique for visualizing and chronologically analyzing the construction status of a common foundation, which is information including knowledge shared among participants in a dialogue using a dialogue system. [Background technology]

[0002] In recent years, in dialogue systems, humans can interact with computers such as smart speakers to obtain various information and fulfill requests. In addition to dialogue systems that accomplish specific tasks, such as AI dialogue-based customer service systems, there are also dialogue systems such as stuffed animal-type robots that engage in everyday conversations (casual conversations). These systems enable humans to achieve mental stability, satisfy their desire for recognition, and build trusting relationships. Types of dialogue systems are described in Non-Patent Document 1.

[0003] Furthermore, Non-Patent Document 2 states that even with the use of recent large-scale language models, long-term communication is difficult, resulting in a problem of low user satisfaction. As a means for solving this problem, there is a method of providing a dialogue system with information called a common foundation. A "common foundation" refers to information such as knowledge and beliefs shared between participants in a dialogue, and is sometimes called a "mutual belief." In order for a computer to process the contents of a dialogue, the common foundation handled by the computer has been considered to be one of the important concepts, but Non-Patent Document 3 discloses that there has been little research analyzing the process by which a common foundation is constructed.

[0004] In addition, Non-Patent Document 4 discloses research content in which text chats between two workers completing a task were collected and analyzed as an attempt to build a common foundation. Non-Patent Document 5 discloses research content in which video data was collected and analyzed of the process in which two workers completed a task. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] "Natural Language Processing Series 7 Dialogue Systems", edited by Manabu Okumura, co-authored by Mikio Nakano, Kazunori Komatani, Kotaro Funakoshi, and Yukiko Nakano, published 2015.2.13 [Non-Patent Document 2] Sparks of Artificial General Intelligence: Early experiments with GPT-4<https: / / arxiv.org / abs / 2303.12712> [Non-Patent Document 3] "Understanding Processing in Dialogue Systems", Mikio Nakano, SIG-SLUD-B901-01, Language, Speech Understanding and Dialogue Processing, 2019. [Non-Patent Document 4] "Analysis of the process of constructing a common ground for dialogue in a collaborative shape layout task," Wataru Mitsuda, Ryuichiro Higashinaka, Yuhei Ohga, Tetsuya Kinebuchi, Proceedings of the 27th Annual Conference of the Association for Natural Language Processing (NLP2021), March 2021 [Non-Patent Document 5] Furuya, Yuki, Saito, Koki, Ogura, Kosuke, Mitsuda, Koh, Higashinaka, Ryuichiro and Takashio, Kazunori, Dialogue Corpus Construction Considering Modality and Social Relationships in Building Common Ground, Language Resources and Evaluation Conference (LREC2022), 2022. Summary of the Invention [Problem to be solved by the invention]

[0006] However, in order for robots and virtual agents to communicate naturally with humans, it is essential to build a common ground with the user. There are few cases where the details of the process of building a common ground in human-to-human dialogue have been analyzed, and no method has been established to investigate and analyze the effects of various conditions related to the work environment and the worker on the process of building a common ground.

[0007] The present invention has been made in consideration of the above-mentioned points, and aims to provide a method for investigating and analyzing the effects of various conditions related to the work environment and workers on the process of building a common foundation for human dialogue. [Means for solving the problem]

[0008] In order to solve the above problem, the invention of claim 1 is an analysis device that analyzes the influence on the process of constructing a common foundation including knowledge shared between participants in a dialogue, the analysis device having an acquisition unit that acquires corpus data regarding the content of the dialogue of the participants when a task is performed, a statistical data extraction unit that outputs statistical data classified from the corpus data into a set of conditions related to the modality of the dialogue and the social relationships of the participants, a task pattern classification unit that classifies the statistical data into a task success pattern or a task failure pattern and outputs a classification result, an analysis unit that generates visualization information that visualizes the classification result by analyzing the classification result, and an output unit that outputs the visualization information. Effect of the Invention

[0009] As described above, the present invention has the effect of providing a method for investigating and analyzing the influence of various conditions related to the work environment and workers on the process of building a common foundation for human dialogue. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a functional configuration diagram of an analysis device according to an embodiment. [Diagram 2]Figure 1 shows the task tools used in the collaborative shape placement task. [Diagram 3] FIG. 2 is a diagram showing a part of corpus data, and an example of corpus data of collaborative graphic layout in an arbitrary dialogue. [Figure 4] FIG. 13 is a diagram showing an example dialogue (example utterance) portion in the corpus data of collaborative graphic layout. [Diagram 5] FIG. 13 is a diagram showing data regarding operations on graphical objects and utterances, generated from a portion of the corpus data. [Figure 6] FIG. 13 is a diagram showing the classification results of seven types of task patterns for each of four conditions, generated from a portion of the corpus data. [Figure 7] FIG. 13 is a diagram showing an example of visualization information, illustrating the average value of the distance between graphic layouts of task success patterns in four conditions. [Figure 8] FIG. 13 is a diagram showing an example of visualization information, illustrating the average value of the distance between graphic layouts of task failure patterns in four conditions. [Figure 9] FIG. 13 shows the results of an analysis of non-verbal behavior, which is an example of statistical information, and is a diagram showing the results of a comparison of turn-taking timing for each of four conditions, taking into account modality and social relationships. [Figure 10] FIG. 13 shows the results of an analysis of linguistic behavior, which is an example of statistical information, and is a diagram showing a comparison of the percentage of dialogues in which a specific reply was given to a question accompanied by a confirmatory dialogue act. [Figure 11] FIG. 13 shows the results of an analysis of non-verbal behavior, which is an example of statistical information, and is a diagram showing the presence or absence of eye contact in response to a question depending on the content of the utterance. [Figure 12] FIG. 2 is a diagram illustrating an electrical hardware configuration of an analysis device according to an embodiment. [Figure 13] 4 is a flowchart showing a process or operation of the analysis device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [Overview of the embodiment] In this embodiment, four conditions are set by combining the condition of audio or video with audio as the dialogue modality and the condition of first meeting or acquaintance as the social relationship, and if dialogues in which a common foundation is built between speakers can be collected and both verbal and non-verbal behaviors in the dialogue can be treated as time-series data and their progress analyzed, it will be possible to verify whether a wider variety of modalities and deeper social relationships accelerate the building of a common foundation. The collected corpus data and the results of the analysis are also useful for research and development of dialogue systems and communication robots.

[0013] First, here, a case will be described in which two workers independently set a joint graphic arrangement task for arranging each graphic object such as a circle, triangle, square, or icon, and the corresponding graphic arrangement is regarded as a common base, and the dialogue and the common base are recorded as a joint graphic arrangement corpus. The joint graphic arrangement task is a task in which two workers independently arrange the arrangement of multiple randomly arranged figures while consulting with each other without looking at each other's screen (see Non-Patent Document 4). The arrangement of graphic objects is a general task common to the creation of furniture arrangements, the arrangement of symbols on a map, or the creation of designs, and covers a variety of tasks in the real world. The analysis device 10 shown in FIG. 1 is a device capable of such analysis.

[0014] Figure 2 shows the display of each worker's tool (seven geometric objects arranged in random locations) at the start of the task in the collaborative geometric arrangement task. Figure 2(a) shows the screen displayed by the task tool on the display terminal of worker A, and Figure 2(b) shows the screen displayed by the task tool on the display terminal of worker B.

[0015] During the task, workers A and B discuss and align the arrangements of graphical objects that are placed in different locations without looking at the other worker's display screen. Workers A and B can move the position of the graphical objects displayed on each screen by dragging the mouse. When both parties agree on the arrangement, they finish the task by pressing the "End" button at the top of the tool. Operations on the graphical objects and changes in status ("Start", "End", etc.) are recorded in the log file of each display terminal along with a timestamp. The degree to which the graphical arrangements between workers A and B match is considered to be a common ground in human dialogue, making it possible to analyze the process of building a common ground.

[0016] Here, we will explain an example in which the conditions for the work environment are (1-1) audio or (1-2) audio and video, and the conditions for the workers are (2-1) a pair meeting for the first time or (2-2) a pair of acquaintances. Audio or audio and video are conditions related to the modality in the work environment, and a pair meeting for the first time or a pair of acquaintances are conditions related to the social relationship between the workers.

[0017] For example, as a condition related to modality in the work environment, under the condition where the video of the video chat tool is turned off (hereafter referred to as "Voice"), only the voice is heard by the other party. Under the condition where the video is turned on (hereafter referred to as "Voice+Video"), the other party sees the image in addition to the voice, and non-verbal behavior such as facial expressions and gestures are also communicated. On the other hand, under the condition of social relationships, we assume that the impact on the construction of a common foundation will be analyzed by using as examples a conversation in which the two workers are acquainted with each other before the experiment (hereafter referred to as "Friend") and a conversation in which they are meeting for the first time (hereafter referred to as "First meet").

[0018] [Functional configuration of the analysis device] The functional configuration of the analysis device 10 will be described with reference to Fig. 1. Fig. 1 is a functional configuration diagram of the analysis device according to the embodiment. The analysis device 10 is a device that receives corpus data as input and outputs, as statistical data, data on actions and speech related to tasks, task completion rates for four conditions, and the progress status of construction of a common infrastructure. By manually analyzing the final output (visualized information, statistical information), it becomes possible to consider whether the set conditions accelerate the construction of a common infrastructure and what conditions are necessary for the construction of a common infrastructure.

[0019] The analysis device 10 includes an acquisition unit 11, a statistical data extraction unit 12, a task pattern classification unit 13, an analysis unit 14, and an output unit 15. Each of these units has a function realized by an instruction from a CPU 101 in FIG. 12 based on a program.

[0020] The acquisition unit 11 receives input of conditions relating to the work environment and the worker, which are analysis conditions, and corpus data of one or more joint graphic layouts to be analyzed.

[0021] The acquisition unit 11 also receives input of label data indicating which of the seven classifications described below the data of the graphic layout image derived from the corpus data is paired with the data as input data for machine learning.

[0022] The corpus data includes the following items (a) to (d). All of the data can be handled in chronological order using timestamps.

[0023] (a) Conditional Data This data indicates the conditions under which workers A and B interact with each other, and includes, for example, the above four conditions.

[0024] (b) Collaborative figure placement task task data for each worker This data indicates who placed (moved) which graphic object to what position, when (at what timing) and at what time.

[0025] (c) Speech data of each worker This data indicates who said what, when (at what timing), etc. This data is the data shown in Figs. 3 and 4 described later.

[0026] (d) Annotation data on non-verbal behavior This data is data in which actions other than language (utterances) are labeled, for example, data in which a label is attached that worker A turned to the right at a predetermined time. One method of creating this data is for a creator to manually assign a label such as "turn to the right for a predetermined time" to 10:00:00-10:00:05 while viewing a video of workers A and B working as shown in the video area 210 of FIG. 3 described below. Another method of creating the data is to automatically assign a label such as "turn to the right for a predetermined time" to 10:00:00-10:00:05 by estimating the direction of the face through image recognition processing.

[0027] FIG. 3 is a diagram showing a part of the corpus data, and an example of the corpus data of the collaborative graphic layout in an arbitrary dialogue. The corpus data of the collaborative graphic layout is data shown on a screen 200 as shown in FIG. 3. The screen 200 includes a video area 210, a setting area 220, an adjustment area 230, a waveform area 240, and an utterance content area 250. Among these, the video area 210 is an area for displaying a video in which the workers A and B arranged the graphic objects on the display screen shown in FIG. 2. The setting area 220 is an area for setting the playback volume, playback speed, etc. The waveform area 240 is an area for displaying a voice waveform related to the utterance content while the workers A and B were arranging the graphic objects on the display screen shown in FIG. 2.

[0028] Furthermore, the utterance content area 250 is an area for showing text related to the utterance content made by workers A and B while they were placing each of the graphic objects on the display screen shown in FIG. 2. L_utterance shows the utterance content made by worker A on the left side of the video area 210, and R_utterance shows the utterance content made by worker B on the right side of the video area 210. The text related to the utterance content is shown separately in FIG. 4. FIG. 4 is a diagram showing an example dialogue (example utterance) portion in the corpus data of collaborative graphic layout. This utterance data corresponds to (c) above. Note that ID is an utterance identifier, P is the participant (workers A and B), <o> and< / o> The space between the two indicates the overlapping part of the speech (the content of worker A's speech while worker B is speaking).

[0029] ID U 82 Now, we have confirmed that the trapezoid is the last shape whose position has not yet been determined. 87 and U 88 In this case, the participants use gestures to confirm the order of the shapes, including the placement of the geometric objects that have been decided so far. 90 In this case, a statement is made to seek the other person's agreement, and a final common ground is established.

[0030] The statistical data extraction unit 12 extracts and outputs statistical data classified into four conditions from the corpus data acquired from the acquisition unit 11. The statistical data includes data related to the four conditions, and is extracted in the relationship shown in FIG. 5. FIG. 5 is a diagram showing data related to the operation and speech of graphic objects generated from a part of the corpus data. The top part of FIG. 5 shows the four conditions. "Number of dialogues" indicates the number of dialogues acquired under each condition. The values ​​below "average number of graphic operations" indicate the average value per dialogue in 20 dialogues. "Average number of graphic operations" indicates the average total number of times that participants (workers A and B) operated (moved) each graphic object in the task. The time until the completion of the collaborative graphic arrangement task indicates the average number of seconds from when the "start" button of the task tool is pressed to when the "end" button is pressed. "Total number of turn takes" is the count of turn takes, which are the switching of speakers. In this case, a turn indicates the time (section) during which a certain worker is speaking with initiative. For example, in FIG. 4 described later, each ID is one turn. Note that if a worker speaks for a short time and then stops, that time may not be included in the turn in this case. "Average speaking time" is the average of the total speaking time of each speaker. The total number of turn-takes and average speaking time are values ​​found by analysis using a general voice analysis program (e.g. Julius) after workers A and B have finished their dialogue. "Average task time" is the average of the total task time of each speaker.

[0031] In the example of Figure 5, we can see that the total number of turn-takes, average speaking time, and average task time were the highest in the Voice+Video Friend condition. However, the average number of graphic operations was the lowest in the same condition, which suggests that in the Voice+Video Friend condition, participants were able to talk to each other a lot and operate graphic objects efficiently.

[0032] In order to compare the final task completion rate (i.e., the rate of matching of the figures), the task pattern classification unit 13 classifies the final figure layout images into seven types of task patterns based on the statistical data extracted by the statistical data extraction unit 12, and classifies these into two types of task success patterns and task failure patterns. This classification is similar to a prior example (see Non-Patent Document 4) that targets text chat. Since the details are shown in Non-Patent Document 4, only the main points will be explained below.

[0033] As shown below, the final geometric layout is roughly divided into two task patterns (divided into seven in more detail). Each task pattern is divided into seven task patterns for each of the four specified conditions, as shown in FIG. 6. FIG. 6 is a diagram showing the classification results of seven types of task patterns for each of the four conditions, generated from a portion of the corpus data. For example, in the case of video on and acquaintance (Voice+Video / Friend) conditions, 60% were successful. That is, in the case of 20 conversations as shown in FIG. 5, FIG. 6 shows that success was achieved in 12 conversations.

[0034] In this case, for example, the relationship between the two final geometric layouts created by workers A and B in Figure 1 is shown. Here, the task success patterns are (1-1) and (1-2), in which case it is considered that a common foundation has been completely established between the participants (workers). Furthermore, the task failure patterns are (2-1), (2-2), (2-3), (2-4), and (2-5), in which case it is considered that a common foundation has not been established between the participants (workers).

[0035] (1-1) Exact match: The shapes match perfectly This means that a certain range of mismatches will be considered a match, and a 100% match is not required.

[0036] (1-2) Origin offset: The position of the entire figure is different This refers to the case where the absolute positions of one final graphic layout image and the other final graphic layout image are different, but the layout of the specified graphics is the same. Therefore, such a case is considered successful. (2-1) Scale Mismatch: The size of the entire graphics is different. For example, when three graphic objects b2, b3, and b4 are arranged around one graphic object b1, the three graphic objects b2, b3, and b4 are arranged close to the one graphic object b1 in the final graphic arrangement image by worker A, while the same three graphic objects b2, b3, and b4 are each arranged relatively far away from the one graphic object b1 in the final graphic arrangement image by worker B. Note that in this embodiment, the graphic objects themselves are not scaled or enlarged.

[0037] (2-2) Symmetrical deviation: The position of the entire figure is symmetrically different This shows a case where the final graphic layout image by worker A and the final graphic layout image by worker B are so-called mirror-inverted.

[0038] (2-3) Same-figure confusion: Same-figure objects are in different positions This refers to a case where, for example, there are geometric objects of the same type, such as a large triangle and a small triangle, and the positions of the large triangle and the small triangle are swapped between the final geometric layout image by worker A and the final geometric layout image by worker B.

[0039] (2-4) Different-shape objects: Different types of shape objects are in different positions This refers, for example, to the case where different geometric objects are arranged at the same position in the final geometric layout image by worker A and the final geometric layout image by worker A.

[0040] (2-5) Complete mismatch: The positions of the figures are completely different This shows a case where the final graphic layout image by worker A and the final graphic layout image by worker B end up with completely different graphic layouts.

[0041] In addition, the task pattern classification unit 13 has a trained machine learning model (e.g., BERT), and during machine learning, it re-trains (fine-tunes) the trained machine learning model using input data that is a pair of data of a predetermined figure arrangement image that can be derived from statistical data and label data indicating which of the above seven classifications it falls into, obtained from the acquisition unit 11.

[0042] On the other hand, the task pattern classification unit 13 uses the trained machine learning model during inference, inputs data of a predetermined figure arrangement image that can be derived from statistical data, and outputs the classification results of the task patterns, thereby automatically performing classification. Note that classification may also be performed manually.

[0043] First, the analysis unit 14 calculates the distance between geometric objects in the joint geometric object layout task, which is the sum of the distances of the differences between vectors defined between any two geometric objects in the joint geometric object layout task, as a quantitative index for constructing a common foundation, and generates visualization information that visualizes the transition patterns and time transition patterns in operation units in the joint geometric object layout task for each of the four conditions. Examples of the visualization information are shown in Figs. 7 and 8.

[0044] Fig. 7 is an example of visualization information, showing the average value of the distance between graphic layouts in the task success patterns in four conditions. Note that, in Fig. 5, one dialogue includes, for example, 124.80 utterances (number of turn-takes). The "distance between graphic arrangements" indicates the distance between the position of a specific graphic object on the side of worker A and the position of the same (corresponding) specific graphic object placed by worker B on the side of worker B, assuming that the specific graphic object on the side of worker A is displayed on the side of worker B. In this example, the average value of the distance between graphic arrangements of all corresponding graphic objects in the task success pattern in the four conditions is shown. The vertical axis of the graph indicates the distance between graphic arrangements, and the horizontal axis indicates the number of steps in which both workers A and B manipulate the graphic objects in the joint graphic arrangement task. The number of steps on the horizontal axis is normalized to 100 steps using a linear transformation. From this example, it can be seen that in all conditions, the distance between graphic arrangements eventually converges in a direction that becomes smaller around 100 steps, and the graphic arrangements match. As a result, it becomes possible to obtain analysis results such as the fact that in the case of Voice / First meet (dashed line), the construction of a common foundation stagnates in the early stages, whereas in the case of Voice+Video / Friend (thick dashed line), the construction of a common foundation proceeds smoothly from start to finish. In addition, in the First Meet condition, there was a lot of initial confirmation behavior and moving of graphical objects while trying to figure out the other person's intentions, and the building of a common foundation in the early stages of the task was gradual, suggesting that it is easier to build a common foundation in Friends than in First Meet, and that work progresses more easily in Voice+Video than in Voice.

[0045] Moreover, FIG. 8 is an example of visualization information, showing the average distance between geometric arrangements of failure patterns in four conditions. In this example, the average value of the distance between geometric arrangements of failure patterns in four conditions is shown. The vertical axis of the graph represents the distance between geometric arrangements, and the horizontal axis represents the number of steps in which workers A and B manipulate geometric objects in the collaborative geometric arrangement task. The number of steps on the horizontal axis has been normalized to 100 steps using a linear transformation. From this example, it can be seen that there is not much difference between the conditions in terms of the transition of the distance between geometric arrangements.

[0046] Secondly, the analysis unit 14 analyzes the classification results output from the task pattern classification unit 13, and outputs each piece of statistical information shown in FIG. 9 to FIG.

[0047] Figure 9 shows the results of an analysis of non-verbal behavior, which is an example of statistical information, and is a diagram showing the comparison results of turn-taking timing for each of the four conditions, taking modality and social relationship into consideration. Bold text indicates the largest value in each row. * indicates that the comparison result with the video-on and acquaintance (Voice+Video / Friend) condition is p<0.05. ** indicates that the comparison result with the video-on and acquaintance (Voice+Video / Friend) condition is p<0.01, which is a significant difference. It can be seen that the turn-taking latency in the dialogue in the joint figure layout task is longer in the video-on and acquaintance (Voice+Video / Friend) condition compared to the other three conditions. This result suggests that in conditions with rich modalities and close social relationships, dialogues take a long time to build a common foundation.

[0048] FIG. 10 shows the results of an analysis of linguistic behavior, which is an example of statistical information, and is a diagram showing a comparison result of the percentage of dialogues in which a specific reply was made to a question accompanied by a dialogue act of confirmation. Note that a dialogue act indicates a semantic expression that expresses a speech intention. It is automatically classified by a machine learning method from data of utterances to which a dialogue act is assigned. FIG. 10 is a diagram showing a comparison result of the percentage of dialogues in which a specific reply was made to a question accompanied by a dialogue act of confirmation.

[0049] In Figure 10, the percentage of conversations in which one speaker (worker) made a confirmatory dialogue act and the other speaker gave a specific reply (presentation of information or self-disclosure of a fact) is compared across four conditions that take modality and social relationship into consideration. In the video-on and acquaintance (Voice+Video / Friend) condition, there is a significant tendency that a higher percentage of people gave specific replies to questions accompanied by a confirmatory dialogue act from the other speaker. In the video-on and acquaintance (Voice+Video / Friend) condition, where the process of building a common ground is smooth, the turn-over latency was longer than in the other three conditions, and there were more specific replies to the dialogue partner's questions. This shows that taking the time to give a thoughtful, specific reply to the dialogue partner's questions is one of the factors that facilitate the building of a common ground.

[0050] FIG. 11 shows the results of an analysis of non-verbal behavior, which is an example of statistical information, and is a diagram showing the presence or absence of gaze in response to a question depending on the content of the utterance. Specifically, it shows a part of the results of an analysis of non-verbal behavior during a joint figure arrangement task, and shows an example of a comparison of the proportion of speakers who make specific utterances or avoid specific utterances depending on whether or not they look at the other speaker's image when one speaker (worker) asks a question. The result shows that there is a tendency for the other speaker to look at the other speaker's image when making a specific reply to the other speaker's question, suggesting that when one speaker asks a question, the other speaker looks at the other speaker to complement the linguistic information and make a specific reply.

[0051] The output unit 15 outputs the visualization information or statistical information acquired from the analysis unit 14. Examples of the output method include displaying on the display 107, outputting to a printer and printing, and transmitting to another device (such as a PC).

[0052] As described above, by using the analysis device 10 of this embodiment, corpus data is input, and as described above, data on behavior and speech related to the task, task achievement rates for the four conditions, and the progress of construction of a common foundation are output as statistical data. By manually analyzing these outputs, it is possible to consider whether the set conditions accelerate the construction of a common foundation and what conditions are necessary for the construction of a common foundation.

[0053] [Hardware configuration of the analysis device] Next, the electrical hardware configuration of the analysis device 10 will be described with reference to Fig. 12. Fig. 12 is a diagram showing the electrical hardware configuration of the analysis device.

[0054] The analysis device 10 is a computer that includes a CPU 101, a ROM 102, a RAM 103, an SSD 104, an external device connection I / F (Interface) 105, a network I / F 106, a display 107, an input device 108, a media I / F 109, and a bus line 110.

[0055] Of these, the CPU 101 controls the overall operation of the analysis device 10. The ROM 102 stores programs such as IPL used to drive the CPU 101. The RAM 103 is used as a work area for the CPU 101.

[0056] The SSD 104 reads and writes various data under the control of the CPU 101. Instead of the SSD 104, a HDD (Hard Disk Drive) may be used.

[0057] The external device connection I / F 105 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB memory, and a printer.

[0058] The network I / F 106 is an interface for performing data communication via the communication network 100 .

[0059] The display 107 is a type of display means, such as a liquid crystal display or an organic EL (Electro Luminescence) display, that displays various images.

[0060] The input device 108 is a keyboard, a pointing device, or the like, and is an example of an input means for accepting input operations of characters, numerical values, various instructions, and the like.

[0061] The media I / F 109 controls reading and writing (storing) of data from and to a recording medium 109m such as a flash memory. The recording medium 109m includes DVDs and Blu-ray Discs (registered trademarks).

[0062] The bus line 110 is an address bus, a data bus, or the like for electrically connecting the various components such as the CPU 101 shown in FIG.

[0063] [Processing or Operation of the Embodiment] Next, the processing or operation of this embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the processing or operation of the analysis device according to the embodiment.

[0064] S11: The acquisition unit 11 acquires corpus data from outside the analysis device 10.

[0065] S12: The statistical data extraction unit 12 outputs statistical data classified into four conditions, as shown in FIG.

[0066] S13: The task pattern classification unit 13 classifies the final graphic layout image of the task into a task success pattern or a task failure pattern based on the statistical data extracted by the statistical data extraction unit 12. This is an example of a classification result.

[0067] S14: First, the analysis unit 14 calculates a geometric arrangement distance, which is the sum of the distances of the differences of vectors defined between any two geometric objects in the task (joint geometric arrangement task), as a quantification index for constructing a common foundation for each classification of task success pattern or task failure pattern, and generates visualization information that visualizes the transition pattern and time transition pattern in operation units in the joint geometric arrangement task for each of the four conditions. Second, the analysis unit 14 analyzes the classification results output from the task pattern classification unit 13, and outputs at least one of the statistical information shown in Figs. 9 to 11.

[0068] S15: The output unit 15 outputs the results of the analysis performed by the analysis unit 14 (visualized information, each piece of statistical information).

[0069] [Main Effects of the Embodiments] As described above, according to the embodiment, it is possible to provide a method for investigating and analyzing the influence that various conditions related to the work environment and the worker have on the process of building a common foundation for human dialogue.

[0070] More specifically, by setting conditions related to the work environment and the worker, and collecting and analyzing the dialogue in which a common foundation is built between speakers, it becomes possible to consider whether the set conditions accelerate the construction of a common foundation and what conditions are necessary for the construction of a common foundation. In this way, in order to analyze the influence of conditions related to the work environment and the worker on the process of building a common foundation, two workers carry out a task of building a common foundation under the conditions to be investigated (any task can be used as long as the commonness to be built can be quantified), and a corpus is built that records the time progress of the construction of the common foundation. The analysis device 10 of this embodiment inputs corpus data, and visualizes and outputs, as statistical data, data related to actions and utterances related to the task, task completion rates under four conditions, and the progress of the construction of the common foundation within a specified range. By manually analyzing the output, it becomes possible to consider whether the set conditions accelerate the construction of a common foundation, what conditions are necessary for the construction of a common foundation, and what actions and utterances in the four conditions affect the construction of the common foundation.

[0071] As a result, by visualizing the process of constructing a common platform at a fine granularity as transition patterns in operation units or time transition patterns, and in addition outputting feature information on actions and speech during dialogue as statistical data, it becomes possible to manually analyze the effects on task achievement of the working environment and worker conditions when multiple workers cooperate through dialogue to accomplish a task (effect of visualized information).In addition, it becomes possible to obtain knowledge useful for research and development of dialogue systems and communication robots, such as feedback on areas where the common platform has stagnated (effect of statistical information).

[0072] 〔supplement〕 The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations).

[0073] (1) The analytical device 10 can be realized by a computer and a program, and this program can be recorded on a (non-temporary) recording medium or provided via a communication network such as the Internet.

[0074] (2) The CPU 101 may be multiple, not just single. [Explanation of symbols]

[0075] 10 Analyzer 11 Acquisition Department 12 Statistical Data Extraction Section 13 Task Pattern Classification Unit 14 Analysis Department 15 Output section

Claims

1. An analytical device that analyzes the impact on the process of building a common ground, including shared knowledge, between the participants of a dialogue, an acquisition unit that acquires corpus data regarding dialogue contents of the participants when the tasks are executed; a statistical data extraction unit that outputs statistical data classified into a set of conditions related to modality of dialogue and social relationships of the participants from the corpus data; a task pattern classification unit that classifies the statistical data into a task success pattern or a task failure pattern and outputs a classification result; an analysis unit that generates visualization information by analyzing the classification result; an output unit that outputs the visualization information; An analytical device having

2. The analysis device according to claim 1 , wherein the condition of the modality of the dialogue is a condition indicating audio or video with audio, and the condition of the social relationship of the participants is a condition indicating a first meeting or an acquaintance.

3. 3. The analysis device according to claim 2, wherein the task pattern classification unit classifies the statistical data into a task success pattern or a task failure pattern based on the statistical data extracted from the statistical data extraction unit using a machine-learned learning model that pairs data of a graphic layout image at the time of execution of the task in the statistical data with label data indicating success or failure of the task.

4. The analysis device of claim 3, wherein the analysis unit calculates a geometric arrangement distance, which is the sum of distances of vector differences defined between any two geometric objects in a collaborative geometric arrangement task as the task, and generates the visualization information that visualizes a transition pattern and a time transition pattern in operation units in the collaborative geometric arrangement task for each of the conditions.

5. 5. The analysis device according to claim 1, wherein the analysis unit analyzes the classification results by the task pattern classification unit and outputs a comparison result of the timing of turn-taking by the participants, a comparison result of the percentage of conversations in which a specific response was made to a question accompanied by a dialogue act of confirmation by the participants, or statistical information indicating whether or not the participants gazed at each other depending on the content of their utterances in response to the question.

6. An analysis method implemented by an analysis device for analyzing an impact on a process of building a common ground including knowledge shared between participants in a dialogue, comprising: The analysis device comprises: An acquisition process for acquiring corpus data regarding the dialogue content of the participants when performing a task; A statistical data extraction process for outputting statistical data classified into a set of conditions related to modality of dialogue and social relationships of the participants from the corpus data; A task pattern classification process for classifying the statistical data into a task success pattern or a task failure pattern and outputting a classification result; an analysis process for generating visualized information by analyzing the classification result; an output process for outputting the visualization information; The analytical method to perform.

7. A program causing a computer to execute the method according to claim 6.

Images