Visual analysis method for classroom group discussion

Abstract

The present invention discloses a visual analysis method for classroom group discussions, including obtaining and preprocessing the group discussion dialogue data to be analyzed; extracting the dialogue reply relationship in the processed group discussion dialogue data; visually analyzing the relationship between the interaction between students in the group discussion over time; calculating the relationship between the speech and the topic and performing a visual analysis; analyzing and visually displaying the performance of students during the discussion and assisting teachers in the analysis. The visual analysis method for classroom group discussions provided by the present invention extracts useful information from the course group discussion data through an innovative technical solution, and performs a visual analysis and display on the extracted information, so as to better assist teachers in conducting visual analysis of classroom group discussions; and the method of the present invention has high reliability, good accuracy, and is scientific and reasonable.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing, and in particular relates to a visual analysis method for classroom group discussion. Background Art

[0002] Group discussion teaching refers to the process in which students communicate and explore designated issues in groups under the guidance of teachers. Group discussion teaching practice shows that the degree to which teachers grasp the progress of the discussion will directly affect the subsequent teaching work. Therefore, teachers need to track and compare the progress of the topics, member performance and communication and interaction of each discussion group, and analyze the cooperation model and discussion process.

[0003] At present, it is generally very difficult for teachers to analyze the group discussion process: first, it is very unrealistic for individual teachers to pay attention to all group discussion teaching and conduct real-time analysis, so they can only conduct post-analysis; then, when conducting post-analysis, it is very time-consuming to directly browse the lengthy discussion recordings for analysis; next, it is also very difficult for individual teachers to summarize the evolution of the interaction of multiple people in the group over time; finally, how teachers obtain effective information during the lengthy discussion process and conduct personal evaluation is also very complicated and difficult. Summary of the invention

[0004] The purpose of the present invention is to provide a visual analysis method for classroom group discussion which is highly reliable, accurate and scientifically reasonable.

[0005] The visual analysis method for classroom group discussion provided by the present invention comprises the following steps:

[0006] S1. Obtain the group discussion dialogue data to be analyzed and perform preprocessing;

[0007] S2. Use deep learning models to extract and process the dialogue response relationships in the group discussion dialogue data;

[0008] S3. Based on the dialogue reply relationship obtained in step S2, an edge-binding-based interactive relationship view is used to visualize the relationship between students' interactions over time in the group discussion;

[0009] S4. According to the dialogue reply relationship obtained in step S2, a topic segmentation algorithm based on a reply relationship tree is used to calculate the relationship between the speech and the topic, and a visual analysis is performed using an enhanced calendar diagram;

[0010] S5. Use the student performance characteristic model to analyze and visualize students’ performance during discussions, thereby assisting teachers in their analysis.

[0011] The step S1 of obtaining the group discussion dialogue data to be analyzed and preprocessing the data specifically includes the following steps:

[0012] Speech data preprocessing, used to extract conversation text from speech data:

[0013] The DeepSpeech speech recognition module is used for speech recognition; the speech of the participants in the speech data is recognized and converted into text data, and the start and end time of the sentence are marked; finally, the structured text data is saved;

[0014] Speaker Recognition:

[0015] First, obtain a set length of audio data of each participant in the group discussion; obtain the audio features of each participant based on the obtained audio data, and generate a voice signature belonging to each participant; finally, compare the voice in the voice data with the voice signature to identify the speaker corresponding to each voice segment;

[0016] Video data preprocessing is used to extract key frame images and students' head postures from video data:

[0017] Key frame image processing: OpenCV image processing module is used to extract key frame images from video data; the input time array T is set to T = [t1, t2, ..., t k ], t i is the time point of the i-th key frame image to be extracted in the video; the OpenCV image processing module is used to parse the target video and traverse the time array T in turn, so as to calculate each item t in the array i The corresponding frame index index is index=t i *fps, fps is the frame rate of the video; then the frame index is used to set the position of video decoding, and then the frame image extracted from the decoding position is output and saved;

[0018] Head posture recognition algorithm: From the video data of the group discussion, extract the changes of head posture of each person in the group over time; extract key frame images from the video using a downsampling algorithm, and then use the OpenPose model to identify the coordinate values ​​of the target person's head key points from the key frame images; the head key points are the set left eye key points, eye key points, left ear key points and right ear key points; finally, use the following formula to calculate the head posture of the i-th frame:

[0019]

[0020] In the formula, up means the head posture is raised; is the Y-axis coordinate value of the right eye key point in the i-th frame; is the Y-axis coordinate value of the right ear key point in the i-th frame; is the Y-axis coordinate value of the left eye key point in the i-th frame; is the Y-axis coordinate value of the left ear key point in the i-th frame; T is the set threshold; down means the head posture is down; miss means leaving; NaN means the value is missing;

[0021] Text data processing, used to clean and structure text data:

[0022] Case conversion: convert all letters in English text to uppercase or lowercase;

[0023] Remove special characters: Remove special characters from text and keep only letters and numbers;

[0024] Remove stop words: Remove stop words from the text according to the stop word list;

[0025] Lemmatization: The open source text processing Python library Gensim WordNetLemmatizer is used to restore the lemma of words.

[0026] The dialogue reply relationship in the group discussion dialogue data extracted and processed by the deep learning model in step S2 is specifically extracted and processed by the reply relationship recognition algorithm based on the MPC-BERT pre-trained deep learning model; specifically includes the following steps:

[0027] A. Model pre-training: Use the conversation records of the Ubuntu online technical exchange forum to pre-train the deep learning model:

[0028] After being encoded by MPC-BERT, each [CLS] token already contains the semantic context; after each [CLS] token is passed through a layer of nonlinear transformation and normalization, we get a U representing each individual sentence. i The vector v i ,and is a d-dimensional vector space; the following formula is used to calculate the statement U i The matching score m with all previous utterances ij :

[0029] m ij =softmax(v i T ·A·v j )

[0030] Where softmax() is a normalized exponential function; A is a linear transformation; m ij For statement Ui The matching score with all previous utterances is used to represent U j It was U i The matching degree of the responses;

[0031] B. Model fine-tuning training: Perform small sample fine-tuning training on the pre-trained model obtained in step A on the classroom group discussion dataset to make the model adapt to the classroom group discussion scenario;

[0032] During training, the cross entropy loss function L is calculated using the following formula: rur :

[0033]

[0034] Where N is the total number of sentences; y ij is an indicator variable, and if U j It was U i The reply statement, then y ij =1, otherwise y ij =0; 1≤j<i;

[0035] C. Model inference: Use the trained model to make inferences and predictions on the processed group discussion dialogue data obtained in step S1, so as to mark the reply relationship between dialogues:

[0036] The reasoning process of the model is as follows: input the i-th sentence and the k sentences before the i-th sentence, where k is a hyperparameter; the model output is a k-dimensional matching score vector S = [m i-k ,...,m j ,...,m i-1 ], indicating the matching score between the first k statements and the i-th statement, m j The sentence j is the matching score of the sentence i replying to it; the sentence with the largest matching score is taken as the sentence replied to by sentence i; if the largest matching scores of k sentences are all lower than the preset threshold m t , then it is determined that the i-th sentence has no reply to any statement and is regarded as opening a new session.

[0037] According to the dialogue reply relationship obtained in step S2, step S3 uses an edge-binding-based interactive relationship view to visualize the relationship between students' interactions over time in the group discussion, which specifically includes the following steps:

[0038] a. Calculate the coordinates and node radius of the speaking node in the view according to the speaking role and time;

[0039] b. Calculate the edges linking the nodes according to the reply relationship between the nodes that speak;

[0040] c. Use edge binding algorithm to optimize edge crossing problem in layout;

[0041] d. In the interactive relationship view, show how student interactions change over time.

[0042] The step a of calculating the coordinates and node radius of the speaking node in the view according to the speaking role and time specifically includes the following steps:

[0043] a1. Use the following formula to calculate the X-axis coordinate x of node i i :

[0044]

[0045] In the formula, x start is the X coordinate of the starting point of the X axis; end is the X coordinate of the end point of the X axis; t start is the discussion start time; t end End time for discussion; is the starting time of node i; is the end time of node i;

[0046] a2. Use the following formula to calculate the Y-axis coordinate y of node i i :

[0047]

[0048] Where y start is the Y coordinate of the starting point of the Y axis; end is the Y coordinate of the end point of the Y axis; n is the total number of people in the group; p i is the speaker number corresponding to node i; offset is the set offset value;

[0049] a3. Use the following formula to calculate the radius r of node i i :

[0050]

[0051] In the formula is the X-axis coordinate corresponding to the start time of node i; is the X-axis coordinate corresponding to the end time of node i.

[0052] The step b of calculating the edge linking node i according to the reply relationship between the speech nodes specifically includes the following steps:

[0053] For node i, the MPC-BERT model trained in step S2 is used to predict the target node j that node i replies to. The coordinates of node i (x i ,y i ) and node j coordinate (x j ,yj ) is connected to form edge E ij .

[0054] The edge binding algorithm described in step c is used to optimize the edge crossing problem in the layout, which specifically includes the following steps:

[0055] According to student S A , S B Y-axis coordinate y of the speaking node A ,y B Two data axes A and B parallel to the X axis are obtained. Virtual binding axes A' and B' are placed between the data axes A and B respectively; the distance between the data axis and the corresponding virtual binding axis is the adjustment parameter d, and d < |y A -y B |; Then, calculate the binding point X on the virtual binding axis A' and X B' , so that the lines connecting all the data points on the data axes A and B are respectively on X A' , X B' Aggregation, the X-axis coordinate of the binding point is the average of the X-coordinates of all the speaking nodes on the corresponding data axis; next, the cubic Bezier spline curve is used to smooth the binding lines of the three parts, and the Y-axis coordinate of the curve control point is the middle value of the two end points; finally, all the speaking nodes are merged into a rounded rectangle, edge binding details diagram.

[0056] The interactive relationship view described in step d specifically includes a timeline and an interactive detail diagram;

[0057] The timeline uses a combination of line graphs and area graphs to show how the number of student interactions changes over time during the discussion process. The horizontal axis encodes time, the vertical axis encodes the number of student interactions, the line color encodes different students, and the size of the area represents the total number of interactions of all students. The timeline is used to visualize the overall interaction distribution overview, and also provides a swipe function, allowing users to freely swipe time intervals.

[0058] The interaction details graph is used to display the interaction details of a time interval, showing the distribution of each student's speech and how the interaction relationship between students changes over time; the horizontal axis of the interaction details graph encodes time, and each horizontal line on the vertical axis represents a student participating in the discussion; the length of the ellipse on the horizontal line encodes the time period when the student participated in the speech, and the color of the ellipse encodes the topic to which it belongs. The number and content of the topics can be modified through the panel at the top; the line between the ellipses encodes the interaction between the two students, and the number on the line shows the specific number of interactions.

[0059] Step S4, based on the dialogue reply relationship obtained in step S2, uses a topic segmentation algorithm based on a reply relationship tree to calculate the relationship between the speech and the topic, and uses an enhanced calendar chart to perform visual analysis, specifically including the following steps:

[0060] (1) Based on the obtained dialogue reply relations, a reply-relation-based topic segmentation algorithm is used to identify the topics to which different sentences belong;

[0061] (2) Divide each group discussion into set time intervals and calculate the keywords and the amount of speech of each student in each time interval;

[0062] (3) Use enhanced calendar charts to visualize the progress of agenda items.

[0063] According to the obtained dialogue reply relationship, the topic segmentation algorithm based on the reply relationship is used to identify the topics to which different sentences belong, which specifically includes the following steps:

[0064] (1)-1. Calculate the topic vector:

[0065] Given topic A, use the word2vec model to convert the N words contained in topic A into N M-dimensional word vectors, where M is a parameter set by yourself; add the N word vectors and calculate the average to get the topic vector V A for in is the word vector corresponding to the i-th word in topic A, and N is the total number of words contained in topic A;

[0066] (1)-2. Calculate a session S i Related topics:

[0067] All n statements generated in a group discussion are divided into h independent conversations [S1,...,S i ,...,S h ]; if statement i is a reply to statement j, then statements i and j belong to the same session; a statement that does not reply to any other statement is considered to have opened a new session; cosine similarity is used to calculate the session S i With the given topic A j SimilaritySim i,j for Where D t belongs to session S i The sentence vector corresponding to the tth sentence of S i The number of statements in the last selection and S i The topic with the largest similarity value is taken as the topic to which the conversation belongs.

[0068] Step (2) of dividing each group discussion into set time intervals and calculating the keywords and the amount of speech of each student in each time interval specifically includes the following steps:

[0069] (2)-1. Calculate the keywords for a given time interval d:

[0070] Suppose a certain time interval is I[t i ,t i+d ], count the words contained in all sentences whose speech start time falls within interval I, and sort them from high to low according to word frequency; retain the first k words with the largest word frequency;

[0071] (2)-2. Count the number of member A's speeches during time interval I:

[0072] If member A speaks a total of k sentences in interval I, then the number of speeches by A in time interval I is L A is the sum of the number of words contained in k sentences.

[0073] The enhanced calendar chart described in step (3) is used to visualize the progress of the topic, which is specifically used to assist teachers in understanding the discussion situation of the group; the enhanced calendar chart is used to represent the progress of the group discussion topic and the distribution of hot keywords in the discussion; the view divides a discussion into several time intervals in order from top to bottom and from left to right, and each interval encodes the distribution of the topic, the student participation in the speech, and several keywords whose mention times are greater than the set value. The time length of the interval can be freely adjusted through the top controller to support different analysis granularities; in order to display the complex information of each interval, the information is encoded in a circular manner; the bar chart outside the circle encodes the number of speeches of each student in the current time interval, and the color of the circle itself encodes the corresponding topic; the angle of the circle encodes the ratio of the discussion content to the topic, and the larger the angle of the topic, the more discussions on the topic are in the current interval; the hot keywords of the speeches in the time interval are displayed inside the circle, and the size of the keyword encodes the number of times it is mentioned.

[0074] The step S5 uses the student performance feature model to analyze and visualize the student's performance during the discussion, thereby assisting the teacher in the analysis, and specifically includes the following steps:

[0075] 1) Based on head posture data, structured text data and dialogue response relationships, a feature model is used to calculate five performance characteristics of students; the five performance characteristics include activity, contribution, leadership, collaboration and confusion;

[0076] 2) The five performance characteristics of each student and the data content related to the characteristics are displayed in real time using the student performance view.

[0077] The step 1) uses a feature model to calculate the five performance features of students based on the head posture data, structured text data and dialogue response relationship, and specifically includes the following steps:

[0078] 1)-1. Calculate contribution:

[0079] It is defined that only when a speech receives a positive response from the majority of participants, it is considered that the speech has contributed to the topic under discussion; the following formula is used to calculate the contribution of student A in the discussion:

[0080] Contribution = c / n

[0081] In the formula, c is the number of sentences in student A's speech that are recognized by the majority, and n is the total number of student A's speeches. When counting the number, the affirmative vocabulary is used to identify the recognized sentences;

[0082] 1)-2. Calculate activity:

[0083] Activity is used to reflect the students' involvement and enthusiasm during the discussion. The activity of student A during the discussion is defined as the ratio of student A's speaking time to the total discussion time. The activity of student A is calculated using the following formula:

[0084]

[0085] Where, t i is the duration of student A’s speech in the i-th time, k is the total number of speeches made by student A, and t start The starting time of the first speech in the discussion; t end The end time of the last statement in the discussion;

[0086] 1)-3. Calculate the degree of collaboration:

[0087] Collaboration is used to reflect students' social communication skills. The collaboration of student A is defined as the ratio of the number of statements that student A actively responds to to the total number of speeches made by A. The collaboration of student A is calculated as Cooperation = col / n, where col is the number of statements recognized by student A and n is the total number of speeches made by student A. When calculating, the affidavit word list is used to judge the affidavit-type sentences.

[0088] 1)-4: Calculate the perplexity:

[0089] Define the perplexity of student A as the ratio of the number of questions asked to the total number of speeches; calculate the perplexity of student A as Perplexity = p / n, where p is the number of questions asked by student A and n is the total number of speeches made by student A;

[0090] 1)-5. Computational leadership:

[0091] Define student A's leadership as the ratio of the number of new conversations opened to the total number of speeches; calculate student A's leadership as Leadership = l / n, where l is the number of new conversations opened by student A, and n is the total number of speeches made by student A.

[0092] The student performance view described in step 2) is specifically a radar chart that visualizes the data of students participating in the group discussion; the radar chart is a radial layout chart consisting of a set of coordinates and a number of concentric circles; the five axes of the radar chart respectively encode the five performance characteristics of the students; there is a circle on each numerical axis to highlight the specific value of the corresponding characteristic; the view is divided into two parts, the left part shows an overview of the characteristics of unselected students, and the right part is the characteristic information of the selected students and the real-time visualization display results of the specific characteristics.

[0093] The visual analysis method for classroom group discussions provided by the present invention extracts useful information from course group discussion data through an innovative technical solution, and performs visual analysis and display on the extracted information, thereby better assisting teachers in performing visual analysis of classroom group discussions; and the method of the present invention has high reliability, good accuracy, and is scientific and reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 The figure is a schematic diagram of the method flow of the present invention.

[0095] Figure 2 This is a schematic diagram of key points of head posture when extracting head posture according to the method of the present invention.

[0096] Figure 3 The figure is a flow chart of the student interaction view layout process of the method of the present invention.

[0097] Figure 4 Schematic diagram of edge binding of the method of the present invention.

[0098] Figure 5 It is a schematic diagram of the interactive relationship view of the method of the present invention.

[0099] Figure 6 It is a schematic diagram of the progress of the topic of the method of the present invention.

[0100] Figure 7 The present invention is a schematic diagram of a data example diagram of an embodiment of the method of the present invention.

[0101] Figure 8 A schematic diagram of a student interaction overview case diagram of an embodiment of the method of the present invention.

[0102] Fig. 9A schematic diagram of a case diagram of student interaction details of an embodiment of the method of the present invention.

[0103] Fig.10 A schematic diagram of a topic progress case diagram of an embodiment of the method of the present invention.

[0104] Fig.11 A schematic diagram of a student performance case diagram of an embodiment of the method of the present invention.

[0105] Fig.12 A schematic diagram of a collaborative case diagram of an embodiment of the method of the present invention.

[0106] Fig.13 A schematic diagram of a head posture case diagram of an embodiment of the method of the present invention. DETAILED DESCRIPTION

[0107] Figure 1 The method flow chart of the method of the present invention is as follows: The visual analysis method of classroom group discussion provided by the present invention comprises the following steps:

[0108] S1. Obtain the group discussion dialogue data to be analyzed and perform preprocessing; specifically, the steps include:

[0109] Group discussion data is multimodal unstructured data. The data sources include audio, video, and metadata related to the class, such as student names, discussion topics, etc. Voice, video, and text data processing technologies are used to structure the raw data and extract the required features.

[0110] Speech data preprocessing is used to extract conversation text from speech data; the main purpose of speech recognition (Automatic Speech Recognition, ASR) is to convert the given discussion audio record into conversation text through the ASR component, and then use it for subsequent text analysis and discussion content visualization:

[0111] The DeepSpeech speech recognition module is used for speech recognition; the speech of the participants in the speech data is recognized and converted into text data, and the start and end time of the sentence are marked; finally, the structured text data is saved;

[0112] Speaker recognition: Group discussions involve multi-party conversations (MPC) with three or more people, and the speaker recognition (SR) component is needed to match different voices with specific participants. The SR component used in the group discussion visual analysis system is Microsoft Azure Cognitive Services:

[0113] First, obtain a set length of audio data of each participant in the group discussion; obtain the audio features of each participant based on the obtained audio data, and generate a voice signature belonging to each participant; finally, compare the voice in the voice data with the voice signature to identify the speaker corresponding to each voice segment;

[0114] Video data preprocessing is used to extract key frame images and students' head postures from video data:

[0115] Key frame image processing: OpenCV image processing module is used to extract key frame images from video data; the input time array T is set to T = [t1, t2, ..., t k ], t i is the time point of the i-th key frame image to be extracted in the video; the OpenCV image processing module is used to parse the target video and traverse the time array T in turn, so as to calculate each item t in the array i The corresponding frame index index is index=t i *fps, fps is the frame rate of the video; then the frame index is used to set the position of video decoding, and then the frame image extracted from the decoding position is output and saved;

[0116] Head posture recognition algorithm: Extract the changes of head posture of each person in the group over time from the video data of the group discussion; use the downsampling algorithm from the video (for example, when the frame rate is 25 per second, set the sampling rate to The key frame image is extracted by downsampling, which can greatly reduce the computational cost without losing information; then the OpenPose model is used to identify the coordinate values ​​of the head key points of the target person from the key frame image. The head key points are the set left eye key points, eye key points, left ear key points and right ear key points (such as Figure 2 As shown); finally, the head posture of the i-th frame is calculated using the following formula:

[0117]

[0118] In the formula, up means the head posture is raised; is the Y-axis coordinate value of the right eye key point in the i-th frame; is the Y-axis coordinate value of the right ear key point in the i-th frame; is the Y-axis coordinate value of the left eye key point in the i-th frame; is the Y-axis coordinate value of the left ear key point in the i-th frame; T is the set threshold; down means the head posture is down; miss means leaving; NaN means the value is missing;

[0119] Text data processing is used to clean and structure text data. For text data extracted from speech, due to the large amount of data, noise interference and inherent errors in the algorithm during multi-party conversations, there may be some duplication and errors in the text data converted from speech. These duplications and errors will affect the performance of subsequent text data analysis models. In the data preprocessing stage, the collected text data needs to be preprocessed. Text data preprocessing can be divided into the following steps: case conversion, removal of special characters, removal of pause words, and word form restoration:

[0120] Case conversion: convert all letters in English text to uppercase or lowercase;

[0121] Remove special characters: Remove special characters from text and keep only letters and numbers;

[0122] Remove stop words: Remove stop words from the text according to the stop word list;

[0123] Lemmatization: Use the open source text processing python library Gensim WordNetLemmatizer to restore the lemma of words;

[0124] S2. The dialogue reply relationship in the group discussion dialogue data after the deep learning model extraction is used; specifically, the dialogue reply relationship in the group discussion dialogue data after the reply relationship recognition algorithm based on the MPC-BERT pre-trained deep learning model is extracted; specifically, the following steps are included:

[0125] A. Model pre-training: Use the conversation records of the Ubuntu online technical exchange forum to pre-train the deep learning model:

[0126] After being encoded by MPC-BERT, each [CLS] token already contains the semantic context; after each [CLS] token is passed through a layer of nonlinear transformation and normalization, we get a U representing each individual sentence. i The vector v i ,and is a d-dimensional vector space; the following formula is used to calculate the statement U i The matching score m with all previous utterances ij :

[0127] m ij =softmax(v i T ·A·v j )

[0128] Where softmax() is a normalized exponential function; A is a linear transformation; m ij For statement Ui The matching score with all previous utterances is used to represent U j It was U i The matching degree of the response;

[0129] B. Model fine-tuning training: Perform small sample fine-tuning training on the pre-trained model obtained in step A on the classroom group discussion dataset to make the model adapt to the classroom group discussion scenario;

[0130] During training, the cross entropy loss function L is calculated using the following formula: rur :

[0131]

[0132] Where N is the total number of sentences; y ij is an indicator variable, and if U j It was U i The reply statement, then y ij =1, otherwise y ij =0; 1≤j<i;

[0133] C. Model inference: Use the trained model to make inferences and predictions on the processed group discussion dialogue data obtained in step S1, so as to mark the reply relationship between dialogues:

[0134] The reasoning process of the model is as follows: input the i-th sentence and the k sentences before the i-th sentence, where k is a hyperparameter; the model output is a k-dimensional matching score vector S = [m i-k ,...,m j ,...,m i-1 ], indicating the matching score between the first k statements and the i-th statement, m j The sentence j is the matching score of the sentence i replying to it; the sentence with the largest matching score is taken as the sentence replied to by sentence i; if the largest matching scores of k sentences are all lower than the preset threshold m t , then it is determined that the i-th sentence has no reply to any statement and is considered to be a new session;

[0135] S3. Based on the dialogue reply relationship obtained in step S2, an edge-binding-based interactive relationship view is used to visualize the relationship between students' interactions over time in the group discussion; specifically, the following steps are included:

[0136] a. Calculate the coordinates and node radius of the speaking node in the view according to the speaking role and time; specifically, the steps include:

[0137] a1. Use the following formula to calculate the X-axis coordinate x of node i i :

[0138]

[0139] In the formula, x start is the X coordinate of the starting point of the X axis; end is the X coordinate of the end point of the X axis; t start is the discussion start time; t end End time for discussion; is the starting time of node i; is the end time of node i;

[0140] a2. Use the following formula to calculate the Y-axis coordinate y of node i i :

[0141]

[0142] Where y start is the Y coordinate of the starting point of the Y axis; end is the Y coordinate of the end point of the Y axis; n is the total number of people in the group; p i is the speaker number corresponding to node i; offset is the set offset value;

[0143] a3. Use the following formula to calculate the radius r of node i i :

[0144]

[0145] In the formula is the X-axis coordinate corresponding to the start time of node i; is the X-axis coordinate corresponding to the end time of node i.

[0146] b. Calculate the edges of the link nodes according to the reply relationship between the speech nodes; specifically, the steps include:

[0147] For node i, the MPC-BERT model trained in step S2 is used to predict the target node j that node i replies to. The coordinates of node i (x i ,y i ) and node j coordinate (x j ,y j ) is connected to form edge E ij .

[0148] c. Use edge binding algorithm to optimize the edge crossing problem in layout; specifically include the following steps:

[0149] According to student S A , S B Y-axis coordinate y of the speaking node A ,y B We get two data axes A and B parallel to the X axis, such as Figure 3-a; virtual binding axes A' and B' are placed between data axes A and B respectively; the distance between the data axis and the corresponding virtual binding axis is the adjustment parameter d, and d<|y A -y B |; Then, calculate the binding point X on the virtual binding axis A' and X B' , so that the lines connecting all the data points on the data axes A and B are respectively on X A' , X B' Aggregation, the X-axis coordinate of the binding point is the average of the X-coordinates of all speaking nodes on the corresponding data axis, such as Figure 3 -b; next, Figure 3 -c, a cubic Bezier spline curve is used to smooth the binding lines of the three parts, and the Y-axis coordinate of the curve control point is the middle value of the two end points; finally, as shown in Figure 3 -d, all speech nodes are merged into a rounded rectangle, and the edge binding details are shown in the figure; Figure 4 As shown;

[0150] d. In the interactive relationship view, show how student interactions change over time;

[0151] The interactive relationship view (such as Figure 5 ), including a timeline and interactive details;

[0152] The timeline uses a combination of line graphs and area graphs to show how the number of student interactions changes over time during the discussion process. The horizontal axis encodes time, the vertical axis encodes the number of student interactions, the line color encodes different students, and the size of the area represents the total number of interactions of all students. The timeline is used to visualize the overall interaction distribution overview, and also provides a swipe function, allowing users to freely swipe time intervals.

[0153] The interactive relationship view consists of two main parts. One is the timeline, which uses a combination of line graph and area graph to show how the number of student interactions changes over time during the discussion process. The horizontal axis encodes time, the vertical axis encodes the number of student interactions, the line color encodes different students, and the gray area represents the total number of interactions of all students. The timeline is mainly used to visualize the overall interaction distribution overview, and also provides a swipe function, so that users can swipe time intervals freely. The other is the interaction detail graph, which displays the interaction details of the time interval, and is used to specifically display the distribution of each student's speech and the changes in the interactive relationship between students over time. The horizontal axis of the view encodes time, and each horizontal line on the vertical axis represents a student participating in the discussion. The length of the ellipse on the horizontal line encodes the time period when the student participated in the speech, and the color of the ellipse encodes the topic to which it belongs. The number and content of the topic can be modified through the top panel. The line between the ellipses encodes the interaction between the two students, and the numbers on the line show the specific number of interactions.

[0154] S4. According to the dialogue reply relationship obtained in step S2, a topic segmentation algorithm based on a reply relationship tree is used to calculate the relationship between the speech and the topic, and a visual analysis is performed using an enhanced calendar diagram; specifically, the following steps are included:

[0155] According to the obtained dialogue reply relationship, the topic segmentation algorithm based on the reply relationship is used to identify the topics to which different sentences belong, which specifically includes the following steps:

[0156] (1)-1. Calculate the topic vector:

[0157] Given topic A, use the word2vec model to convert the N words contained in topic A into N M-dimensional word vectors, where M is a parameter set by yourself; add the N word vectors and calculate the average to get the topic vector V A for in is the word vector corresponding to the i-th word in topic A, and N is the total number of words contained in topic A;

[0158] (1)-2. Calculate a session S i Related topics:

[0159] All n statements generated in a group discussion are divided into h independent conversations [S1,...,S i ,...,S h ]; if statement i is a reply to statement j, then statements i and j belong to the same session; a statement that does not reply to any other statement is considered to have opened a new session; cosine similarity is used to calculate the session S i With the given topic A j SimilaritySim i,j for Where D t belongs to session S i The sentence vector corresponding to the tth sentence of S i The number of statements in the last selection and S i The topic with the largest similarity value is taken as the topic to which the conversation belongs;

[0160] Step (2) of dividing each group discussion into set time intervals and calculating the keywords and the amount of speech of each student in each time interval specifically includes the following steps:

[0161] (2)-1. Calculate the keywords for a given time interval d:

[0162] Suppose a certain time interval is I[t i ,ti+d ], count the words contained in all sentences whose speech start time falls within interval I, and sort them from high to low according to word frequency; retain the first k words with the largest word frequency;

[0163] (2)-2. Count the number of member A's speeches during time interval I:

[0164] If member A speaks a total of k sentences in interval I, then the number of speeches by A in time interval I is L A is the sum of the number of words contained in k sentences;

[0165] (3) Use enhanced calendar charts to visualize the progress of topics (e.g. Figure 6 ); specifically used to assist teachers in understanding the discussion of the group; an enhanced calendar chart is used to represent the progress of group discussion topics and the distribution of hot keywords in the discussion; the view divides a discussion into multiple time intervals in order from top to bottom and from left to right, and each interval encodes the distribution of topics, student participation and the most frequently mentioned keywords. The length of the interval is a parameter that can be freely adjusted through the top controller to support different analysis granularities; in order to display the complex information of each interval, this paper uses a ring-based design when encoding information; the bar chart outside the ring encodes the number of speeches made by each student in the current time interval, and the color of the ring itself encodes the corresponding topic, where gray indicates that the content of the discussion does not belong to any topic; the angle of the ring encodes the proportion of the discussion content corresponding to the topic, and the larger the angle of the topic, the more the topic is discussed in the current interval; considering that teachers also need to understand the changes in the discussion content semantically to observe whether the discussion is focused, the hot keywords of the speech in the time interval are displayed inside the ring, and the size of the keyword encodes the number of times it is mentioned;

[0166] S5. Use the student performance feature model to analyze and visualize the student's performance during the discussion, so as to assist the teacher in the analysis; the specific steps include:

[0167] The step 1) uses a feature model to calculate the five performance features of students based on the head posture data, structured text data and dialogue response relationship, and specifically includes the following steps:

[0168] 1)-1. Calculate contribution:

[0169] It is defined that only when a speech receives a positive response from the majority of participants, it is considered that the speech has contributed to the topic under discussion; the following formula is used to calculate the contribution of student A in the discussion:

[0170] Contribution = c / n

[0171] In the formula, c is the number of sentences in student A's speech that are recognized by the majority, and n is the total number of student A's speeches. When counting the number, the affirmative vocabulary is used to identify the recognized sentences;

[0172] 1)-2. Calculate activity:

[0173] Activity is used to reflect the students' involvement and enthusiasm during the discussion. The activity of student A during the discussion is defined as the ratio of student A's speaking time to the total discussion time. The activity of student A is calculated using the following formula:

[0174]

[0175] Where, t i is the duration of student A’s speech in the i-th time, k is the total number of speeches made by student A, and t start The starting time of the first speech in the discussion; t end The end time of the last statement in the discussion;

[0176] 1)-3. Calculate the degree of collaboration:

[0177] Collaboration is used to reflect students' social communication skills. The collaboration of student A is defined as the ratio of the number of statements that student A actively responds to to the total number of speeches made by A. The collaboration of student A is calculated as Cooperation = col / n, where col is the number of statements recognized by student A and n is the total number of speeches made by student A. When calculating, the affidavit word list is used to judge the affidavit-type sentences.

[0178] 1)-4: Calculate the perplexity:

[0179] Define the perplexity of student A as the ratio of the number of questions asked to the total number of speeches; calculate the perplexity of student A as Perplexity = p / n, where p is the number of questions asked by student A and n is the total number of speeches made by student A;

[0180] 1)-5. Computational leadership:

[0181] Define student A's leadership as the ratio of the number of new conversations opened to the total number of speeches; calculate student A's leadership as Leadership = l / n, where l is the number of new conversations opened by student A and n is the total number of speeches made by student A;

[0182] 2) The five performance characteristics of each student and the data content related to the characteristics are displayed in real time using the student performance view;

[0183] The student individual performance view uses a radar chart to visualize the individual performance of students participating in group discussions. This view helps teachers analyze the performance characteristics of students of interest during the discussion and the reasons behind the performance. For the five characteristics of student performance (activity, contribution, collaboration, leadership, and confusion), a radar chart is first used to provide an overview and comparison. Then, based on the analysis needs of experts for different characteristics, three types of detailed views are designed to help further analyze and reason about the reasons for student performance.

[0184] The radar chart is a radial layout chart composed of a set of coordinates and multiple concentric circles. It can display the analysis and comparison of multiple indicators in the same coordinate system. The radar chart is particularly suitable for making global and holistic evaluations of objects described by multi-attribute system structures. The five axes of the radar chart encode five student performance characteristics respectively. There is a gray circle on each value axis to highlight the specific value of the characteristic. When the mouse hovers over the circle, the value will be automatically displayed. The upper left corner displays the name of the corresponding student to distinguish and compare the performance of different students. Considering data privacy, all student names in the figure are replaced by uppercase letters. The entire view is divided into two parts. The left part shows an overview of the characteristics of unselected students, and the right side shows the characteristic information of selected students and the details of a specific characteristic. Users can click any student in the list on the left or the tab at the top to switch to the student whose characteristic details need to be displayed, and the panel information on the right will be updated synchronously.

[0185] Three types of charts are designed on the right side of the view to show the details of five different features. Users can freely switch the features they want to display through the feature tabs.

[0186] The method of the present invention is further described below in conjunction with an embodiment:

[0187] Research question: Quality assessment of group discussions on winter survival tasks.

[0188] The winter survival task requires team members to prioritize given items in order to survive a winter plane crash. The winter survival task is one of the most used task scenarios in research on group performance, decision-making, and leadership. In this data, each group contains four students. The students first rank 12 items individually, and then as a team, they have a 15-minute group discussion to decide the final item ranking. The four students sit in pairs at both ends of a long table for face-to-face discussions. The audio recording is recorded by a commercial microphone array placed in the middle of the table, and the audio sample rate is 16kHz. Figure 7 As shown, the video is recorded by two portable webcams, and the frame rate of the video is 30fps.

[0189] S1. Import data and perform data preprocessing operations.

[0190] S2. Extract reply relations in conversations using the MPC-BERT model.

[0191] S3. Use the Student Interactions view to analyze the evolution of student interactions over time to discover interesting interaction patterns. First, view the Student Interactions view to get an overall overview. Figure 8 As shown in the figure, from the line graph on the timeline and the interaction details graph below, it can be seen that the students' overall interaction is relatively active. Although the interaction between students K and L is more frequent, there is no situation where students are passive in the interaction. The interaction is more frequent in the two time intervals of 200-300s and 600-900s. Then explore the conversations in several time periods with more interactions: Fig. 9 As shown in part a, in the early stage of the discussion, the main content of the group discussion was to exchange each other's understanding of the task background and avoid deviations between their own cognition and others, so a large number of interactive behaviors were generated in a short period of time. In the later stage of the discussion, due to time constraints, the conclusions must be quickly unified, so the interaction was relatively intensive, such as Fig. 9 Part b and part c show the discussion contexts of the group determining the fifth, sixth, and twelfth items, respectively. In addition, it can be found that the group has a large number of conversations, and there are often multiple conversation threads in a short period of time. Through exploratory analysis, it was found that this was due to the intense discussion, and the students participating in the discussion were more inclined to express their own opinions rather than continue the discussion by taking on the opinions of others, which led to the frequent opening of new conversations. This interaction mode often appears in some open-ended topics without standard answers.

[0192] S4. By viewing the progress of the discussion, we can roughly understand the students' priorities for the four different types of items. In order to display the progress of the topic in a more granular manner, E2 first sets the time interval size of the topic progress view to the minimum value of 50s. Fig.10 As shown in the figure, the discussion can be divided into three parts according to the progress of the topic. First, the time range of 0-150s, such as Fig.10 -a, the main topic of discussion is orange, which represents "cold-proof supplies". From the keywords minus, ten, winter, clothes, canvas, etc., we can see that everyone believes that in a cold environment of minus ten degrees, keeping warm and maintaining body temperature should be the first priority; the second part is the time range of 150-600s, such as Fig.10-b, the discussion mainly revolved around the green and blue topics representing "food" and "survival tools". The first few keywords that appeared in 150-250s, such as maps, seventy kilometres, and two days, meant that after ensuring cold-proof measures, the group members began to discuss the distance they needed to walk to reach a safe shelter, the time it took, and the important survival tool - the map. Secondly, in the interval of 250-500s, the group's main discussion was the source of food. From the keywords such as chocolate, pistol, and kill animals, we can see that the group members discussed the possibility of obtaining food, including the existing chocolate and using pistols to hunt animals; the last part is the period of 600-900s, such as Fig.10 -c shows that the last topic discussed by the group is "miscellaneous items", which means that they think "miscellaneous items" are the lowest priority items. From the keywords whisky, lighter, newspaper, fire, etc., it can be seen that miscellaneous items are mainly discussed as materials for making fire. At the same time, it can be observed that even if the range of each time interval has been set to the minimum value, the arc outside each ring used to encode the number of students speaking is still of a large length, which means that the group discussion is relatively lively and the four members of the group are highly involved in this group discussion.

[0193] S5. Explore the performance and characteristics of different students during the discussion. First, use the radar chart in the individual performance view to overview and compare the five indicators related to individual performance of the four students. Fig.11 As shown in the figure, through comparative analysis, it can be found that the four students have little difference in activity. The most active students are students K and L, which shows that all four students actively participated in the discussion. Considering that the content of the group's discussion does not involve complex knowledge but focuses on the ability of communication and teamwork, the analysis focuses on the two indicators of collaboration and leadership. It is observed that the most active students K and L belong to different types.

[0194] Student K has the highest degree of collaboration in the discussion. By looking at the collaboration view (such as Fig.12 ) found that K's number of agreeing speeches accounted for the largest proportion among all students, and the agreeing speeches were evenly distributed among the other three. This can to a certain extent indicate that student K is more inclined to collaborate in discussions. He is not only good at listening to others' opinions and able to give positive responses in a timely manner, but also can take care of every speaker equally.

[0195] L is the student with the highest leadership in the group, but at the same time he has the lowest degree of collaboration. Fig.13As shown in the figure, by analyzing his speeches in the leadership view, it can be found that L made a lot of summary speeches, and often guided others to speak by giving his own conclusions and explaining the reasons or soliciting opinions. Timely summarization and guiding speeches are manifestations of high leadership. Such behavior is conducive to ensuring the rhythm of the discussion, stimulating others to participate in the discussion when the discussion is deadlocked, and quickly reaching conclusions, but it is easy to interrupt others when the group discussion is more active. In addition, it can be found that L, the student with the highest leadership index, bowed his head the least, while M, the student with the lowest leadership index, bowed his head almost half of the time during the entire discussion. Fig.13 As shown, such performance differences are consistent with people's cognition that people with strong leadership are likely to appear more confident.

[0196] Combining the above five steps, in general, systematic analysis can more objectively evaluate the quality of group discussions. Follow a hierarchical exploration structure of first overview, then filtering, and finally viewing details. Help teachers visualize and analyze from multiple perspectives such as topic progress, student interaction evolution, and student individual performance.

Claims

1. A visual analysis method for classroom group discussion, comprising the following steps: S1. Obtain the group discussion dialogue data to be analyzed and perform preprocessing; S2. Use deep learning models to extract and process the dialogue response relationships in the group discussion dialogue data; S3. Based on the dialogue reply relationship obtained in step S2, an edge-binding-based interactive relationship view is used to visualize the relationship between students' interactions over time in the group discussion; specifically, the following steps are included: Step a: Calculate the coordinates and node radius of the speaking node in the view according to the speaking role and time, which specifically includes the following steps: a1. Use the following formula to calculate the X-axis coordinate x of node i i : In the formula, x start is the X coordinate of the starting point of the X axis; end is the X coordinate of the end point of the X axis; t start is the discussion start time; t end End time for discussion; is the starting time of node i; is the end time of node i; a2. Use the following formula to calculate the Y-axis coordinate y of node i i : Where y start is the Y coordinate of the starting point of the Y axis; end is the Y coordinate of the end point of the Y axis; n is the total number of people in the group; p i is the speaker number corresponding to node i; offset is the set offset value; a3. Use the following formula to calculate the radius r of node i i : In the formula is the X-axis coordinate corresponding to the start time of node i; is the X-axis coordinate corresponding to the end time of node i; Step b: According to the reply relationship between the speech nodes, calculate the edge linking node i, which specifically includes the following steps: For node i, the MPC-BERT model trained in step S2 is used to predict the target node j that node i replies to. The coordinates of node i (x i ,y i ) and node j coordinate (x j ,y j ) is connected to form edge E ij ; Step c: Use edge binding algorithm to optimize the edge crossing problem in the layout, which specifically includes the following steps: According to student S A , S B Y-axis coordinate y of the speaking node A ,y B Get two data axes A and B parallel to the X axis; place virtual binding axes A' and B' between the data axes A and B respectively; the distance between the data axis and the corresponding virtual binding axis is the adjustment parameter d, and d<|y A -y B |; Then, calculate the binding point X on the virtual binding axis A' and X B' , so that the lines connecting all the data points on the data axes A and B are respectively on X A' , X B' Aggregation, the X-axis coordinate of the binding point is the average value of the X-coordinates of all speech nodes on the corresponding data axis; Next, the three parts of the binding line are smoothed using a cubic Bezier spline curve, and the Y-axis coordinate of the curve control point is the middle value of the two end points; Finally, all the speech nodes are merged into a rounded rectangle, edge binding details diagram; Step d: interactive relationship view, including timeline and interactive details diagram; The timeline uses a combination of line graphs and area graphs to show how the number of student interactions changes over time during the discussion process. The horizontal axis encodes time, the vertical axis encodes the number of student interactions, the line color encodes different students, and the size of the area represents the total number of interactions of all students. The timeline is used to visualize the overall interaction distribution overview, and also provides a swipe function, allowing users to freely swipe time intervals. The interaction details graph is used to display the interaction details of a time interval, showing the distribution of each student's speech and how the interaction relationship between students changes over time; the horizontal axis of the interaction details graph encodes time, and each horizontal line on the vertical axis represents a student participating in the discussion; the length of the ellipse on the horizontal line encodes the time period when the student participated in the speech, and the color of the ellipse encodes the topic to which it belongs. The number and content of the topics can be modified through the panel at the top; the line between the ellipses encodes the interaction between the two students, and the number on the line shows the specific number of interactions.

2. The visual analysis method for classroom group discussion according to claim 1 is characterized in that The following steps are also included: According to the dialogue reply relationship obtained in step S2, a topic segmentation algorithm based on a reply relationship tree is used to calculate the relationship between the speech and the topic, and a visual analysis is performed using an enhanced calendar diagram.

3. The visual analysis method for classroom group discussion according to claim 2 is characterized in that The following steps are also included: The student performance characteristic model is used to analyze and visualize the performance of students during the discussion, thereby assisting teachers in their analysis.

4. The visual analysis method for classroom group discussion according to claim 3 is characterized in that The step S1 of obtaining the group discussion dialogue data to be analyzed and preprocessing the data specifically includes the following steps: Speech data preprocessing, used to extract conversation text from speech data: The DeepSpeech speech recognition module is used for speech recognition; the speech of the participants in the speech data is recognized and converted into text data, and the start and end time of the sentence are marked; finally, the structured text data is saved; Speaker Recognition: First, obtain a set length of audio data of each participant in the group discussion; obtain the audio features of each participant based on the obtained audio data, and generate a voice signature belonging to each participant; finally, compare the voice in the voice data with the voice signature to identify the speaker corresponding to each voice segment; Video data preprocessing is used to extract key frame images and students' head postures from video data: Key frame image processing: OpenCV image processing module is used to extract key frame images from video data; the input time array T is set to T = [t1, t2, ..., t k ], t i is the time point of the i-th key frame image to be extracted in the video; the OpenCV image processing module is used to parse the target video and traverse the time array T in turn, so as to calculate each item t in the array i The corresponding frame index index is index=t i *fps, fps is the frame rate of the video; then the frame index is used to set the position of video decoding, and then the frame image extracted from the decoding position is output and saved; Head posture recognition algorithm: From the video data of the group discussion, extract the changes of head posture of each person in the group over time; use the downsampling algorithm to extract the key frame image from the video, and then use the OpenPose model to identify the coordinate values ​​of the target person's head key points from the key frame image. The head key points are the set left eye key points, eye key points, left ear key points and right ear key points; finally, use the following formula to calculate the head posture of the i-th frame: In the formula, up means the head posture is raised; is the Y-axis coordinate value of the right eye key point in the i-th frame; is the Y-axis coordinate value of the right ear key point in the i-th frame; is the Y-axis coordinate value of the left eye key point in the i-th frame; is the Y-axis coordinate value of the left ear key point in the i-th frame; T is the set threshold; down means the head posture is down; miss means leaving; NaN means the value is missing; Text data processing, used to clean and structure text data: Case conversion: convert all letters in English text to uppercase or lowercase; Remove special characters: Remove special characters from text and keep only letters and numbers; Remove stop words: Remove stop words from the text according to the stop word list; Lemmatization: The open source text processing Python library Gensim WordNetLemmatizer is used to restore the lemma of words.

5. The visual analysis method for classroom group discussion according to claim 4 is characterized in that The dialogue reply relationship in the group discussion dialogue data extracted and processed by the deep learning model in step S2 is specifically extracted and processed by the reply relationship recognition algorithm based on the MPC-BERT pre-trained deep learning model; specifically includes the following steps: A. Model pre-training: Use the conversation records of the Ubuntu online technical exchange forum to pre-train the deep learning model: After being encoded by MPC-BERT, each [CLS] token already contains the semantic context; after each [CLS] token is passed through a layer of nonlinear transformation and normalization, we get a U representing each individual sentence. i The vector v i ,and is a d-dimensional vector space; the following formula is used to calculate the statement U i The matching score m with all previous utterances ij : m ij =softmax(v i T ·A·v j ) Where softmax() is a normalized exponential function; A is a linear transformation; m ij For statement U i The matching score with all previous utterances is used to represent U j It was U i The matching degree of the responses; B. Model fine-tuning training: Perform small sample fine-tuning training on the pre-trained model obtained in step A on the classroom group discussion dataset to make the model adapt to the classroom group discussion scenario; During training, the cross entropy loss function L is calculated using the following formula: rur : Where N is the total number of sentences; y ij is an indicator variable, and if U j It was U i The reply statement, then y ij =1, otherwise y ij =0; 1≤j<i; C. Model inference: Use the trained model to make inferences and predictions on the processed group discussion dialogue data obtained in step S1, so as to mark the reply relationship between dialogues: The reasoning process of the model is as follows: input the i-th sentence and the k sentences before the i-th sentence, where k is a hyperparameter; the model output is a k-dimensional matching score vector S = [m i-k ,...,m j ,...,m i-1 ], indicating the matching score between the first k statements and the i-th statement, m j Sentence j is the matching score of the reply of sentence i; The sentence with the largest matching score is taken as the sentence replied by sentence i; If the maximum matching scores of k statements are all lower than the preset threshold m t , then it is determined that the i-th sentence has no reply to any statement and is regarded as opening a new session.

6. The visual analysis method for classroom group discussion according to claim 5 is characterized in that The dialogue reply relationship obtained in step S2 adopts a topic segmentation algorithm based on a reply relationship tree to calculate the relationship between the speech and the topic, and uses an enhanced calendar diagram to perform a visual analysis, which specifically includes the following steps: (1) Based on the obtained dialogue reply relationship, a topic segmentation algorithm based on reply relationship is used to identify the topics to which different sentences belong; specifically, the following steps are included: Compute the issue vector: Given topic A, use the word vector model word2vec to convert the N words contained in topic A into N M-dimensional word vectors, where M is a parameter set by yourself; add the N word vectors and calculate the average to get the topic vector V A for in is the word vector corresponding to the i-th word in topic A, and N is the total number of words contained in topic A; Calculate a session S i Related topics: All n statements generated in a group discussion are divided into h independent conversations [S1,...,S i ,...,S h ]; if statement i is a reply to statement j, then statements i and j belong to the same session; a statement that does not reply to any other statement is considered to have opened a new session; cosine similarity is used to calculate the session S i With the given topic A j SimilaritySim i,j for Where D t belongs to session S i The sentence vector corresponding to the tth sentence of S i The number of statements in the last selection and S i The topic with the largest similarity value is taken as the topic to which the conversation belongs; (2) Divide each group discussion into set time intervals and calculate the keywords and the amount of speech of each student in each time interval; specifically, the steps are as follows: Calculate keywords for a given time interval d: Suppose a certain time interval is I[t i ,t i+d ], count the words contained in all sentences whose speech start time falls within interval I, and sort them from high to low according to word frequency; retain the first k words with the largest word frequency; Count the number of member A's speeches during time interval I: If member A speaks a total of k sentences in interval I, then the number of speeches by A in time interval I is L A is the sum of the number of words contained in k sentences; (3) An enhanced calendar chart is used to visualize the progress of the topic, which is specifically used to assist teachers in understanding the discussion status of the group. An enhanced calendar chart is used to represent the progress of the group discussion topic and the distribution of hot keywords. The view divides a discussion into several time intervals from top to bottom and from left to right. Each interval encodes the distribution of topics, student participation and several keywords with a number of mentions greater than the set value. The time length of the interval can be freely adjusted through the top controller to support different analysis granularities. In order to display the complex information of each interval, a ring-based method is used to encode the information. The bar chart outside the ring encodes the number of speeches made by each student in the current time interval, and the color of the ring itself encodes the corresponding topic. The angle of the ring encodes the ratio of the discussion content to the topic. The larger the angle of the topic, the more discussions on the topic are held in the current interval. The hot keywords of the speeches in the time interval are displayed inside the ring, and the size of the keyword encodes the number of times it is mentioned.

7. The visual analysis method for classroom group discussion according to claim 6 is characterized in that The student performance characteristic model is used to analyze and visualize the performance of students during the discussion, thereby assisting teachers in the analysis, and specifically includes the following steps: 1) Based on the head posture data, structured text data and dialogue response relationship, a feature model is used to calculate five performance characteristics of students; the five performance characteristics include activity, contribution, leadership, collaboration and confusion; specifically, the following steps are included: Calculate contribution: It is defined that only when a speech receives a positive response from the majority of participants, it is considered that the speech has contributed to the topic under discussion; the following formula is used to calculate the contribution of student A in the discussion: Contribution = c / n In the formula, c is the number of sentences in student A's speech that are recognized by the majority, and n is the total number of student A's speeches. When counting the number, the affirmative vocabulary is used to identify the recognized sentences; Calculate activity: Activity is used to reflect the students' involvement and enthusiasm during the discussion. The activity of student A during the discussion is defined as the ratio of student A's speaking time to the total discussion time. The activity of student A is calculated using the following formula: In the formula, t i is the duration of student A’s speech in the i-th time, k is the total number of speeches made by student A, and t start The starting time of the first speech in the discussion; t end The end time of the last statement in the discussion; Calculate collaboration: Collaboration is used to reflect students' social communication skills. The collaboration of student A is defined as the ratio of the number of statements that student A actively responds to to the total number of speeches made by A. The collaboration of student A is calculated as Cooperation = col / n, where col is the number of statements recognized by student A and n is the total number of speeches made by student A. When calculating, the affidavit word list is used to judge the affidavit-type sentences. Calculate the perplexity: Define the perplexity of student A as the ratio of the number of questions asked to the total number of speeches; calculate the perplexity of student A as Perplexity = p / n, where p is the number of questions asked by student A and n is the total number of speeches made by student A; Computational Leadership: Define student A's leadership as the ratio of the number of new conversations started to the total number of speeches; Calculate student A's leadership as Leadership = l / n, where l is the number of new conversations opened by student A and n is the total number of speeches made by student A. 2) The five performance characteristics of each student and the data content related to the characteristics are displayed in real time using the student performance view; the student performance view specifically uses a radar chart to visualize the data of students participating in the group discussion; the radar chart is a radial layout chart composed of a set of coordinates and a number of concentric circles; the five axes of the radar chart respectively encode the five performance characteristics of the students; there is a circle on each numerical axis to highlight the specific value of the corresponding characteristic; the view is divided into two parts, the left part shows an overview of the characteristics of unselected students, and the right part is the characteristic information of the selected students and the real-time visualization display results of the specific characteristics.

Images