Classroom session mode automatic analysis method and device

Abstract

The invention relates to the technical field of classroom session mode analysis, and discloses a classroom session mode automatic analysis method and device.The method comprises the steps that classroom videos and audios are extracted, texts are transcribed, and session segments are divided to obtain sub-audios and sub-texts; fusing the sub-audio session main body and the sub-text session hierarchy and type to generate a macroscopic feature, converting a sub-text sentence into a semantic feature, and fusing the semantic feature with the macroscopic feature to obtain a fused feature vector; and finally, identifying the fusion feature vector through a preset Gaussian mixture model, and outputting a classroom session mode. According to the method, deep fusion of macroscopic behavior features and semantic content features is realized, the problem of insufficient multi-source heterogeneous feature fusion is solved, a deep structure and a potential mode of classroom interaction are accurately mined, a complex representative session mode is efficiently identified, powerful technical support is provided for improvement of teaching quality, and application limitation of the prior art is made up.

Description

Technical Field

[0001] This invention relates to the field of classroom conversation pattern analysis technology, and specifically to an automated method and apparatus for classroom conversation pattern analysis. Background Technology

[0002] With the continuous evolution of intelligent education technology, classroom interaction, as the core carrier of the teaching process, has become a crucial link in improving teaching quality and accurately evaluating teaching effectiveness through efficient and objective analysis. Classroom interaction encompasses multi-dimensional information such as language communication, behavioral feedback, and intellectual exchange between teachers and students. In-depth analysis of this interaction data can provide a scientific basis for optimizing teaching strategies, implementing personalized teaching, and quantifying teaching effectiveness, serving as an important support for promoting the digital transformation of education and achieving precision teaching.

[0003] However, traditional classroom interaction analysis mainly relies on manual observation and coding methods, which require a large investment of manpower and are time-consuming. At the same time, they are easily affected by the subjective experience and cognitive biases of the analysts, making it difficult to guarantee the objectivity and consistency of the analysis results, and thus failing to meet the needs of large-scale and dynamic modern classroom scenarios.

[0004] Therefore, current automated classroom analysis technologies typically employ several approaches. Some solutions rely on descriptive statistics based on behavioral perspectives, neglecting in-depth semantic analysis of conversations; others simply categorize the depth of textual dialogues, lacking a systematic exploration of interactive logic; still others depend on traditional turn-taking frameworks like IRF, only achieving basic turn-taking structure segmentation. Consequently, these methods fail to achieve deep integration of multi-source heterogeneous features, such as macro-behavioral characteristics and semantic content features, making it difficult to uncover the underlying structures and potential patterns of interaction. Consequently, they cannot accurately identify complex and representative classroom conversation patterns, thus failing to provide sufficient technical support for improving teaching quality. Summary of the Invention

[0005] This invention provides an automated analysis method and apparatus for classroom conversation patterns to address the problem that existing technologies fail to achieve deep integration of multi-source heterogeneous features such as macro-behavioral features and semantic content features in classroom conversations, making it difficult to uncover the deep structure and potential patterns behind interactions. Consequently, they are unable to accurately identify complex and representative classroom conversation patterns, resulting in insufficient technical support for improving teaching quality.

[0006] In a first aspect, the present invention provides an automated analysis method for classroom conversation patterns, the method comprising: Extract the audio files from the classroom video and transcribe the audio files into text data; The audio files and text data are divided according to the conversation segments to generate multiple sub-audio files and multiple sub-text data. By combining the conversation subject of each sub-audio file, the conversation hierarchy and conversation type of each sub-text data, macro-features are generated; Each sentence of each of the sub-text data is transformed into a semantic feature, and the semantic features and macro features of the same conversation segment are fused to generate multiple fused feature vectors, which are then input into a preset target Gaussian mixture model. The classroom conversation mode of the classroom video is obtained by identifying each of the fused feature vectors through the target Gaussian mixture model.

[0007] This invention first extracts classroom video audio and transcribes it into text, dividing the conversation into sub-audio and sub-text segments. Then, it fuses the conversation subject of the sub-audio, the conversation hierarchy and type of the sub-text to generate macro-features. The sub-text sentences are then transformed into semantic features and fused with the macro-features to obtain a fused feature vector. Finally, a pre-set Gaussian mixture model is used to identify the fused feature vector, outputting the classroom conversation pattern. This invention achieves deep fusion of macro-behavioral features and semantic content features, solving the problem of insufficient fusion of multi-source heterogeneous features. It accurately mines the deep structure and potential patterns of classroom interaction, efficiently identifies complex and representative conversation patterns, provides strong technical support for improving teaching quality, and overcomes the application limitations of existing technologies.

[0008] In one optional implementation, extracting the audio file from the classroom video and transcribing the audio file into text data includes: Get the class videos; The audio of the classroom video is separated using a multimedia video encoding tool to obtain the audio file of the classroom video; The audio file is transcribed into text data.

[0009] This invention uses professional multimedia video encoding tools to accurately separate classroom video audio, and combines this with efficient transcription technology to generate text data, thus realizing the transformation of classroom audio and video information into analyzable text. This ensures both the integrity and sound quality of the separated audio, avoiding information loss, and the accuracy of the transcribed text.

[0010] In one optional implementation, the step of generating macro-features by combining the session subject of each of the sub-audio files, the session hierarchy of each of the sub-text data, and the session type includes: Distinguish the conversation subject in each of the sub-audio files; Based on the conceptual and practical levels of the conversation content in each of the sub-text data, the conversation level of each of the sub-text data is determined; Based on the teaching behavior, questioning behavior, feedback behavior, instruction behavior, sharing behavior, and supplementary behavior in the conversation process of each sub-text data, determine the conversation type of each sub-text data; Macro-level features are generated using all the aforementioned session subjects, session levels, and session types.

[0011] This invention determines conversation types by accurately distinguishing conversation participants, defining conversation levels based on both conceptual and practical dimensions, and classifying and identifying six core conversation behaviors. It systematically extracts key macro-dimensional features of classroom interaction. The generated macro-features comprehensively cover conversation participants, content levels, and behavioral patterns, fully depicting the overall structure and interaction logic of classroom conversations.

[0012] In one optional implementation, the step of converting each sentence of each of the sub-text data into semantic features, fusing the semantic features and macro-features of the same conversation segment to generate multiple fused feature vectors, and inputting them into a preset target Gaussian mixture model includes: Each of the sub-text data is input into a preset sentence embedding model; The sentence embedding model is used to transform each sentence of each sub-text data into a high-dimensional vector to generate semantic features. Based on the multi-head attention mechanism, the semantic features and macro-features of the same conversation segment are fused to generate multiple fused feature vectors; Input all the fused feature vectors into the preset target Gaussian mixture model.

[0013] This invention transforms text into high-dimensional semantic features through a sentence embedding model, and combines this with a multi-head attention mechanism to achieve accurate fusion of semantic and macroscopic features. The fused feature vector is then input into a Gaussian mixture model. This not only solves the problem of insufficient fusion of multi-source heterogeneous features, but also strengthens the correlation between features through the attention mechanism, thereby improving the representational power of the feature vector.

[0014] In an optional implementation, prior to the step of inputting all the fused feature vectors into a preset target Gaussian mixture model, the method further includes: The model parameters of the preset initial Gaussian mixture model are initialized to generate an updated Gaussian mixture model; The updated Gaussian mixture model is solved iteratively using the expectation-maximization algorithm, and the optimal model parameters are determined based on the solution results. The updated Gaussian mixture model is updated using the optimal model parameters to obtain the target Gaussian mixture model.

[0015] This invention constructs a high-precision target model by reasonably initializing the parameters of the Gaussian mixture model and iteratively solving for the optimal parameters using the expectation-maximization algorithm.

[0016] In one optional implementation, the step of identifying each of the fused feature vectors through the target Gaussian mixture model to obtain the classroom conversation pattern of the classroom video includes: The posterior probability of each fused feature vector is calculated using the optimal model parameters of the target Gaussian mixture model. Each of the fused feature vectors is categorized into the corresponding session mode with the highest posterior probability. Clustering is performed on all the fused feature vectors to obtain the classroom conversation pattern of the classroom video.

[0017] This invention calculates the posterior probability of the fused feature vector based on optimal parameters, and classifies and clusters them according to the maximum probability criterion to achieve accurate recognition of classroom conversation patterns.

[0018] In an optional implementation, the method further includes: Calculate the cosine similarity between the conversation fragments corresponding to each of the fused feature vectors and the cluster centers; Based on the cosine similarity, the multiple conversation segments with the highest similarity are identified as representative segments, and the complete text data of the representative segments is output.

[0019] This invention calculates the cosine similarity between the fused feature vector and the cluster center to accurately select the most similar representative conversation segments and output the complete text. It leverages the strong representational power of feature vectors to ensure selection accuracy, and uses cosine similarity to quantify the correlation between segments and the core patterns, efficiently extracting typical samples from each conversation pattern.

[0020] Secondly, the present invention provides an automated analysis device for classroom conversation patterns, the device comprising: The extraction module is used to extract the audio files from the classroom video and transcribe the audio files into text data; The segmentation module is used to segment the audio file and the text data according to the conversation segments, and generate multiple sub-audio files and multiple sub-text data. The module is used to combine the session subject of each sub-audio file, the session hierarchy and session type of each sub-text data to generate macro-features; The input module is used to convert each sentence of each of the sub-text data into semantic features, and to fuse the semantic features and macro features of the same conversation segment to generate multiple fused feature vectors and input them into a preset target Gaussian mixture model; The recognition module is used to identify each of the fused feature vectors through the target Gaussian mixture model to obtain the classroom conversation mode of the classroom video.

[0021] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the classroom conversation pattern automated analysis method of the first aspect or any corresponding embodiment described above.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the classroom conversation pattern automated analysis method of the first aspect or any corresponding embodiment described above.

[0023] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the classroom conversation pattern automated analysis method described in the first aspect or any corresponding embodiment. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the first type of automated analysis method for classroom conversation patterns according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the second type of automated analysis method for classroom conversation patterns according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the third process of the automated analysis method for classroom conversation patterns according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the multi-head attention mechanism according to an embodiment of the present invention, which fuses semantic features and macroscopic features. Figure 5 This is a structural block diagram of an automated classroom conversation pattern analysis device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0028] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] This invention provides an automated analysis method for classroom conversation patterns. First, classroom video audio is extracted and transcribed into text, and conversation segments are divided into sub-audio and sub-text segments. Then, the subject of the sub-audio conversation, the conversation level and type of the sub-text conversation are fused to generate macro-features. Sub-text sentences are transformed into semantic features and fused with the macro-features to obtain a fused feature vector. Finally, a preset Gaussian mixture model is used to identify the fused feature vector, outputting the classroom conversation pattern. This achieves deep fusion of macro-behavioral features and semantic content features, solving the problem of insufficient fusion of multi-source heterogeneous features. It accurately mines the deep structure and potential patterns of classroom interaction, efficiently identifies complex and representative conversation patterns, provides strong technical support for improving teaching quality, and overcomes the limitations of existing technologies.

[0030] According to an embodiment of the present invention, an embodiment of an automated analysis method for classroom conversation patterns is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a method for automated analysis of classroom conversation patterns. Figure 1 This is a flowchart of an automated classroom conversation pattern analysis method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Extract the audio file from the classroom video and transcribe the audio file into text data.

[0032] It should be noted that classroom videos refer to dynamic video files that record the entire classroom teaching process (including teacher-student interaction, knowledge instruction, practical operation, and other scenarios).

[0033] Audio files refer to files extracted from classroom videos that contain all the sound information from the classroom (teacher-student dialogues, teaching instructions, interactive feedback, etc.).

[0034] Text data refers to the conversion of audio information from audio files into structured text using speech-to-text technology.

[0035] In this embodiment of the invention, when extracting the audio file of the classroom video, the FFmpeg multimedia processing tool is used to separate the audio and video streams of the classroom video, filter background noise and non-conversational audio interference, and obtain a clean conversational audio file; based on the speech-to-text model, combined with the optimized transcription algorithm using a corpus specific to the classroom teaching scenario, the clean conversational audio file is transcribed into text data sentence by sentence.

[0036] Step S102: Divide the audio files and text data according to the conversation segments to generate multiple sub-audio files and multiple sub-text data.

[0037] It should be noted that a conversation segment refers to a continuous conversation unit based on silent intervals and semantic pauses, with complete interactive logic at its core. It is the smallest analytical unit that carries local interactive content in the classroom.

[0038] A sub-audio file refers to a partial audio file generated after an audio file has been segmented. It contains only the speech information of a single conversation segment and has the same format as the original audio file.

[0039] Subtext data refers to local text data generated after synchronous partitioning of text data. It fully corresponds to the semantic content of a single sub-audio file and preserves the logical structure of the original conversation.

[0040] In this embodiment of the invention, when using a sliding window to divide audio files and text data according to conversation segments, a unified segment division threshold is set based on the silence detection results of the audio files and the semantic pause markers (such as punctuation marks and topic transition keywords) of the text data. The audio files and text data are synchronously split into consecutive conversation units through a time synchronization mechanism to generate multiple sub-audio files and multiple sub-text data that correspond one-to-one, ensuring that the speech content of each sub-audio file and the semantic information of the corresponding sub-text data are completely matched.

[0041] Specifically, the sliding window method is used to transform continuous classroom conversation records into discrete, analyzable samples. The process involves treating the classroom conversation record as a continuous time series and using a fixed-size window (typically 5-10) to slide along the time axis with specific steps. All the conversation content and related feature data within the window together constitute a conversation segment. This method effectively solves the problem of unbounded and difficult-to-process raw data, breaking down continuous and complex classroom interactions into a series of structured, independent analytical samples, thus allowing each window to serve as the smallest unit for subsequent feature extraction and pattern analysis.

[0042] Step S103: Combine the conversation subject of each sub-audio file, the conversation hierarchy of each sub-text data, and the conversation type to generate macro-features.

[0043] It should be noted that the subject of the conversation refers to the specific object participating in the interactive conversation of the classroom sub-audio file, mainly including the teacher, individual students or student groups, who are the initiators and participants of the conversation.

[0044] Conversational levels refer to the cognitive depth and level of abstraction of the content of a conversation, and are divided into conceptual levels and practical levels.

[0045] Conversation type refers to the specific behavioral functions in the teacher-student conversation process in the subtext data, including but not limited to core classroom interaction types such as lecturing, asking questions, providing feedback, giving instructions, sharing, and supplementing.

[0046] Macro features refer to a comprehensive feature vector that integrates information from dimensions such as the subject of the conversation, the level of the conversation, and the type of the conversation, and can reflect the overall attributes, interaction structure and core features of a conversation segment.

[0047] In this embodiment of the invention, the conversation subjects such as teachers and students in the sub-audio files are identified and labeled with identity tags. The conversation level is determined based on the knowledge logic and practical application dimensions of the sub-text data. The conversation type is classified and coded according to the interactive behavior types such as lecturing and questioning. Then, the identity tags, level categories, and type codes are integrated according to preset feature dimensions to generate a macro feature vector that can comprehensively represent the overall attributes of the conversation.

[0048] Step S104: Convert each sentence of each sub-text data into semantic features, and fuse the semantic features and macro features of the same conversation segment to generate multiple fused feature vectors and input them into the preset target Gaussian mixture model.

[0049] It should be noted that semantic features refer to high-dimensional vectors obtained by quantifying the semantics of a sentence.

[0050] A fused feature vector is a feature vector formed by weighted integration of semantic features and macroscopic features of the same conversation segment.

[0051] The target Gaussian mixture model refers to a probabilistic model used for conversation pattern recognition after parameter optimization. It achieves clustering and recognition of different conversation patterns by fitting the probability distribution of fused features through a weighted combination of multiple Gaussian distributions.

[0052] In this embodiment of the invention, each sentence of each sub-text data is transformed into a high-dimensional semantic feature vector with a fixed dimension. Then, the association weight between the semantic features and the macro-features of the same conversation segment is calculated based on the multi-head attention mechanism. The two types of features are weighted and fused according to the weight to generate multiple fused feature vectors that can simultaneously characterize the semantic connotation and overall attributes of the conversation. After standardizing all fused feature vectors, they are input into a preset target Gaussian mixture model.

[0053] Step S105: Identify each fused feature vector using the target Gaussian mixture model to obtain the classroom conversation mode of the classroom video.

[0054] It should be noted that classroom conversation patterns refer to typical patterns that reflect the structure and core logic of classroom interaction, obtained through model identification and clustering.

[0055] In this embodiment of the invention, the posterior probability of each fused feature vector belonging to various potential conversation modes is calculated using the optimized parameters of the target Gaussian mixture model. Each fused feature vector is then classified into the conversation mode category with the highest posterior probability. All fused feature vectors in the same category are then clustered and aggregated. The rationality of the clustering results is verified by combining the interactive logic of the classroom teaching scenario. Finally, a classroom conversation mode that can reflect the core features of classroom interaction is obtained.

[0056] This embodiment provides a method for automated analysis of classroom conversation patterns. Figure 2 This is a flowchart of an automated classroom conversation pattern analysis method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Extract the audio file from the classroom video and transcribe the audio file into text data.

[0057] Specifically, step S201 includes: Step S2011: Obtain the classroom video.

[0058] In this embodiment of the invention, the entire classroom teaching process is captured through multiple channels, such as the campus recording system interface, file upload to the teaching platform, or import from local storage devices. The captured video files are then format-verified and integrity-checked to remove invalid videos with damaged images or abnormal durations, while retaining valid classroom videos that contain complete teacher-student interaction scenes and synchronized audio and video.

[0059] Step S2012: Use a multimedia video encoding tool to separate the audio from the classroom video to obtain the audio file of the classroom video.

[0060] It should be noted that the multimedia video encoding tool refers to FFmpeg, a professional tool that supports audio and video stream parsing, separation, encoding and decoding, and format conversion.

[0061] Audio separation refers to the technical operation of extracting the audio track separately from the audio and video mixed stream of a classroom video.

[0062] like Figure 3 As shown, FFmpeg is used to separate the audio from the classroom video to generate audio files that conform to the format of a general speech recognition interface or a third-party API call. The audio files are used for audio transcription and speaker recognition. The classroom conversation records are stored in CSV file format, with the following record format: speaker, start time, end time, and transcribed text.

[0063] Step S2013: Transcribe the audio file into text data.

[0064] In a specific embodiment, multi-speaker recognition technology is used to distinguish and associate the roles of teachers and students in the conversation, and the audio content is transcribed sentence by sentence according to the audio time sequence. Homophone errors and technical terminology deviations in the transcription process are corrected simultaneously to generate structured text data that is semantically consistent with the original audio, time-aligned, and contains role identifiers.

[0065] Step S202: Divide the audio files and text data according to the conversation segments to generate multiple sub-audio files and multiple sub-text data.

[0066] Please see details Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0067] Step S203: Combine the conversation subject of each sub-audio file, the conversation hierarchy of each sub-text data, and the conversation type to generate macro-features.

[0068] In some optional implementations, step S203 above includes: Step S2031: Distinguish the conversation subject in each sub-audio file.

[0069] In this embodiment of the invention, the basic characteristics of classroom conversations are determined based on the constituent elements of Conversation Theory, including the conversation subjects, conversation levels, and conversation types. The conversation subjects refer to the speakers in the classroom conversation record, specifically distinguished as teachers and students.

[0070] Based on the speech features of sub-audio files (such as voiceprint, pitch, and speech rate), combined with a pre-trained speaker recognition model, voiceprint extraction and matching are performed on audio segments. At the same time, personal pronouns and interactive context in the transcribed text are associated to assist in the determination, accurately identifying conversation subjects such as teachers and students.

[0071] Step S2032: Determine the conversation level of each sub-text data based on the conceptual and practical levels of the conversation content in each sub-text data.

[0072] It should be noted that the conversation content refers to the classroom interaction information carried in the subtext data, including specific expressions such as knowledge explanation, question discussion, and practical guidance.

[0073] The conceptual level refers to a type of conversational level, which involves conversations centered around abstract or theoretical content.

[0074] The practice level refers to a type of conversation level, which involves conversations centered around actual behavior or specific cases.

[0075] In this embodiment of the invention, the basic characteristics of classroom conversations are determined based on the constituent elements of Conversation Theory, including the conversation subject, conversation level, and conversation type. The conversation level describes the cognitive depth and abstraction of the conversation content, and is divided into the conceptual level (conversations revolving around abstract or theoretical content) and the practical level (conversations revolving around actual behavior or specific cases). Specifically, by calculating the proportion and semantic relevance of the two types of keywords in the sub-text through keyword matching and semantic similarity calculation, and combining the logical connection characteristics between concept explanation and practical application in the classroom teaching scenario, the conversation level of the sub-text data is determined to be either the conceptual level or the practical level.

[0076] Step S2033: Determine the conversation type of each sub-text data based on the teaching behavior, questioning behavior, feedback behavior, instruction behavior, sharing behavior, and supplementary behavior in the conversation process of each sub-text data.

[0077] It should be noted that the conversation process refers to the teacher-student interaction process presented in the subtext data.

[0078] Teaching behavior refers to the conversational behavior in which teachers impart knowledge, explain concepts, and expound principles to students.

[0079] Questioning behavior refers to conversational behavior in which the participants (teachers or students) raise questions to obtain information and stimulate thinking.

[0080] Feedback behavior refers to conversational behavior that responds to others' lectures, questions, or other actions (such as affirmation, negation, or supplementary explanation).

[0081] Directive behavior refers to conversational behavior in which the subject of the conversation (often a teacher) issues instructions, guides actions, or makes explicit requests.

[0082] Sharing behavior refers to conversational behavior in which students or teachers proactively share their views, experiences, cases, and other content.

[0083] Supplementary behavior refers to conversational behavior that adds explanations or refines details to existing conversation content.

[0084] In this embodiment of the invention, the basic characteristics of classroom conversation are determined based on the constituent elements of Conversation Theory, including the conversation subject, conversation level, and conversation type. The conversation type refers to specific behavioral functions during teacher-student conversations, including but not limited to lecturing, questioning, feedback, instruction, sharing, and supplementation.

[0085] Specifically, the automatic identification of basic features mainly refers to the automatic classification of the transcribed text of classroom conversation records by conversation level and conversation type. The identification method can flexibly adopt the following technologies: one is to use a large language model (LLM) after data fine-tuning to achieve automatic classification and identification through API calls and prompt word engineering; the other is to use a supervised deep learning pre-trained model for automatic classification and identification.

[0086] The first method is automatic recognition using a large language model (LLM) after data fine-tuning. This involves collecting a small amount of classroom conversation data with clear conversation hierarchy and conversation type labels, using this data to fine-tune a pre-trained general LLM, allowing the model to adapt to specific classroom contexts and conversation theory definitions, thereby improving its classification accuracy on this task. In actual recognition, the fine-tuned LLM is called through an API interface, and the recognition task is encapsulated into carefully designed prompts and sent to the model.

[0087] The second type is supervised deep learning pre-trained model automatic recognition, which usually involves selecting a pre-trained language model (such as BERT, RoBERTa) and using a large amount of labeled classroom conversation data to perform end-to-end supervised training on the model. The training objective is to minimize the loss between the predicted labels and the real labels, so that the model can learn to directly classify the input transcribed text into the corresponding conversation level and conversation type.

[0088] It is worth mentioning that this invention employs a first type of fine-tuned Large Language Model (LLM) for automatic recognition, pre-labeling some classroom conversation data with conversation hierarchy and conversation type tags; subsequently, the generalized Large Language Model is fine-tuned by balancing the dataset, and the accuracy is verified on the test set to ensure the model's generalization performance. In the actual automatic labeling process, by calling the API of the fine-tuned Large Language Model and using cue word engineering (mainly setting tag classification criteria) as instructions, the conversation hierarchy and conversation type tags are efficiently extracted as basic features.

[0089] Step S2034: Generate macro-features using all session subjects, session levels, and session types.

[0090] In this embodiment of the invention, the macro-features of classroom conversations aim to quantify the overall composition of each conversation segment in terms of behavior and structure. Within each predefined sliding window (i.e., conversation segment), the frequency of occurrence of automatically identified basic features (including different conversation subjects, conversation levels, and conversation types) is first statistically analyzed. Then, these frequencies are divided by the total frequency or total duration within the window to calculate the proportion of each basic feature. These calculated proportions are standardized and formed into a unified vector representation, which constitutes the macro-feature vector representing the conversation segment. The specific formula is as follows: ; ; ; ; ; In the formula, F It is a set of basic features that includes all the different conversation subjects, conversation levels, and conversation types; M It is the total number of categories of basic features; w This is the current sliding window for analysis. It is a feature In the sliding window w Frequency of occurrence within; It is the total frequency of the sliding window; It is the original macroscopic feature vector composed of all proportional features; It is the final macro-vector of classroom conversation; and These are the mean and standard deviation of the original proportional characteristics obtained from the calculation.

[0091] Step S204: Convert each sentence of each sub-text data into semantic features, and fuse the semantic features and macro features of the same conversation segment to generate multiple fused feature vectors and input them into the preset target Gaussian mixture model.

[0092] Specifically, step S204 includes: Step S2041: Input each sub-text data into the preset sentence embedding model.

[0093] It should be noted that sentence embedding model refers to a semantic representation model based on deep learning, which can transform natural language sentences into low-dimensional or high-dimensional dense vectors to achieve quantitative expression of semantic information. In this invention, it specifically refers to the Sentence-BERT model.

[0094] In this embodiment of the invention, each sub-text data is divided into independent sentence units according to sentence boundaries. After removing redundant stop words and meaningless filler words, the data is input into a preset Sentence-BERT sentence embedding model.

[0095] Step S2042: Transform each sentence of each sub-text data into a high-dimensional vector using a sentence embedding model to generate semantic features.

[0096] It should be noted that a high-dimensional vector refers to a high-dimensional dense vector generated after a sentence is processed by an embedding model. It quantifies the semantic information of the sentence in numerical form and has computable semantic similarity.

[0097] In this embodiment of the invention, the semantic features of the classroom conversation are achieved through deep processing of each sub-text data. The pre-trained model paraphrase-multilingual-MiniLM-L12-v2 from the Sentence-BERT model is employed. This model is specifically designed to efficiently generate sentence-level embedding vectors, accurately capturing and representing the overall semantic and contextual information of each sentence while maintaining fast inference speed. In this way, each transcribed text is transformed into an accurate and high-quality semantic representation, providing a reliable information foundation for subsequent feature fusion. The specific formula is as follows: ; ; ; In the formula, It is the first in the classroom conversation transcript text i One sentence; MiniLM-L12 is an encoder; It is a context-dependent token vector; It is a sentence i The Middle j The context embedding vector of each token; It is the token length of the sentence; For sentences The semantic feature vector.

[0098] Step S2043: Based on the multi-head attention mechanism, the semantic features and macro-features of the same conversation segment are fused to generate multiple fused feature vectors.

[0099] It should be noted that the multi-head attention mechanism refers to a core component in the Transformer architecture, which performs multi-head attention on queries. Q ,key K Sum V Each head is linearly mapped to multiple subspaces (i.e., "heads"). Each head independently computes attention to capture feature correlations across different dimensions. Finally, the outputs of all heads are concatenated and linearly projected again, thereby enhancing the model's expressive power and its ability to capture complex dependencies.

[0100] In this embodiment of the invention, the multi-head attention mechanism uses semantic feature vectors as queries. Q s and keys K Using macroscopic eigenvectors as values V It learns how to associate each sentence of the transcribed text with macroscopic features. The MultiHead attention mechanism is developed by... h The results of calculations for each independent single-head attention head are concatenated. For the first... i One's attention, Q , K , V First, through their respective weight matrices , , Projected to a lower dimension and Then, attention weights are obtained through Softmax normalization. A Attention weight A Multiplied by the projected value V Get the first i The output of each attention head Concat(.) is used to concatenate all objects. h The results of each attention head are stitched together along the dimension. It is a linear projection matrix used to project the concatenated result (dimension 1). Projecting back to the expected output dimension of the model, such as Figure 4 The multi-head attention mechanism structure diagram is shown below, and the specific implementation formula is as follows: ; ; ; ; ; .

[0101] In the formula, , , They are Q , K , V The weight matrix; and These are lower dimensions; It is a linear projection matrix used to project the concatenated result (dimension 1). Project back to the expected output dimension of the model.

[0102] Step S2044: Input all fused feature vectors into the preset target Gaussian mixture model.

[0103] In this embodiment of the invention, the fused feature vector is the original semantic feature. e The features transposed by the MultiHead attention mechanism are averaged along the sequence (window) dimension to obtain a fixed-length feature vector, which is then concatenated and input into a fully connected layer to fuse the output features. x A fully connected layer contains two linear layers. , The ReLU activation function, with the specific formula as follows: ; ; .

[0104] In the formula, e These are the original semantic features; , These are two linear layers in a fully connected layer; ReLU is the activation function of a fully connected layer.

[0105] All fusion features x After standardization, input the preset target Gaussian mixture model.

[0106] In some optional implementations, prior to step S2044, the method further includes: Step a1: Initialize the model parameters of the preset initial Gaussian mixture model to generate an updated Gaussian mixture model.

[0107] It should be noted that the initial Gaussian mixture model refers to a probabilistic model that has undergone parameter optimization and only has the basic model structure. It is composed of a weighted combination of multiple Gaussian distributions and needs to be adapted to the classroom conversation pattern recognition requirements through parameter initialization and iterative optimization.

[0108] Model parameters refer to the core components of a Gaussian mixture model, including the mean, covariance matrix, and weights of each Gaussian distribution.

[0109] Initialization refers to the process of assigning appropriate initial values ​​to the parameters of the initial Gaussian mixture model.

[0110] Updating the Gaussian mixture model refers to the intermediate model obtained after parameter initialization, which has initial parameter configurations adapted to the classroom scenario.

[0111] In this embodiment of the invention, the number of Gaussian distribution components is determined based on the preset number of categories in the classroom conversation mode, and the standardized fusion features are processed using the K-means clustering algorithm. x Pre-clustering is performed, using the cluster centers as the initial means of each Gaussian distribution, combined with fusion features. x The variance statistics are used to set the initial covariance matrix, and the initial weights of each Gaussian distribution are assigned according to a uniform distribution. After the parameter initialization is completed, an updated Gaussian mixture model is generated.

[0112] Step a2: The expectation-maximization algorithm is used to iteratively solve the updated Gaussian mixture model, and the optimal model parameters are determined based on the solution results.

[0113] It should be noted that the Expectation-Maximization (EM) algorithm is an iterative optimization algorithm used for parameter estimation of probabilistic models with latent variables. By alternately executing the expectation step (E step) and the maximization step (M step), it gradually approaches the optimal parameters of the model and is suitable for solving the parameters of complex probabilistic models such as Gaussian mixture models.

[0114] Iterative solution refers to the process of continuously updating model parameters by repeatedly executing E-steps and M-steps to gradually optimize model performance until the convergence condition is met.

[0115] The optimal model parameters refer to the model parameters (mean, covariance matrix, weights) obtained after iterative convergence of the EM algorithm, at which point the model has the best fit to the classroom conversation feature data.

[0116] In this embodiment of the invention, the expectation-maximization algorithm is used to learn the parameters of the latent Gaussian distribution (i.e., classroom conversation patterns) in the data. The specific formula is as follows: ; ; ; .

[0117] In the formula, It is a set of parameters of a Gaussian distribution (i.e., classroom conversation patterns);k It is the number of Gaussian distributions (Gaussian components); It is the first k Weights / prior probabilities of Gaussian components; It is the mean vector of the k-th Gaussian component. It is the first k The covariance matrix of Gaussian components; It is the optimal parameter set, i.e., the set obtained by maximizing the log-likelihood function. The set of parameters found that best explains the data; N It represents the total number of samples (number of conversation segments). It is the first i The fused feature vector of each session yes In the k Gaussian components The probability density under, It is a fusion feature x In the k The probability density of Gaussian components.

[0118] Step a3: Update the new Gaussian mixture model using the optimal model parameters to obtain the target Gaussian mixture model.

[0119] In this embodiment of the invention, the optimal mean, covariance matrix and weight parameters obtained by the expectation-maximization algorithm iteration are used to replace the original initial parameters in the Gaussian mixture model. Simultaneously, it is verified whether the model's fit to the feature matrix meets the preset threshold. It is confirmed that the model after parameter replacement can accurately adapt to the probability distribution of classroom conversation features, and finally, a target Gaussian mixture model with optimal parameter configuration that can be directly used for conversation pattern recognition is obtained.

[0120] Step S205: Identify each fusion feature vector using the target Gaussian mixture model to obtain the classroom conversation mode of the classroom video.

[0121] In some optional implementations, step S205 above includes: Step S2051: Calculate the posterior probability of each fused feature vector using the optimal model parameters of the target Gaussian mixture model.

[0122] It should be noted that the posterior probability refers to the conditional probability that a fused feature vector belongs to a certain Gaussian distribution component, given the observed values ​​of the fused feature vector. It is used to quantify the degree of matching between a sample and a distribution component.

[0123] In this embodiment of the invention, the posterior probability of each session fragment belonging to each potential pattern is calculated, and it is assigned to the pattern with the highest probability to complete clustering and obtain the clustering label of the session fragment. The specific formula is as follows: ; In the formula, k It is the number of Gaussian distributions (Gaussian components); It is the first k Weights / prior probabilities of Gaussian components; It is a fusion feature x In the k The probability density of each Gaussian component; It is a fusion feature x Source: k The posterior probabilities of the Gaussian components; D is the fused feature. x The dimension; j Used to iterate through all K Gaussian components.

[0124] Step S2052: Classify each fused feature vector into the corresponding session mode with the highest posterior probability.

[0125] In this embodiment of the invention, each fusion feature is used as an example. x Based on the posterior probability of each conversation mode, the conversation mode with the highest probability value is selected as the category to which the feature vector belongs. At the same time, the consistency of the classification results is checked. If there is a conflict in the classification of multiple feature vectors under the same conversation segment, the classification results are corrected by combining the macro features of the conversation segment. Finally, all fused feature vectors are accurately mapped to the corresponding conversation mode categories.

[0126] Step S2053: Cluster all fused feature vectors to obtain the classroom conversation mode of the classroom video.

[0127] In this embodiment of the invention, based on the completed pattern classification results, the fusion features belonging to the same session pattern are... x The data is aggregated and summarized to extract the common semantic features and macroscopic attribute patterns of the feature vectors under this category. Combined with the interactive logic of classroom teaching scenarios, each cluster is given a concrete pattern name. At the same time, a small number of abnormal feature vectors that deviate from the cluster center are removed to optimize the clustering effect, and finally, the classroom conversation pattern is obtained.

[0128] Step S206: Calculate the cosine similarity between the conversation fragments corresponding to each fused feature vector and the cluster center.

[0129] It should be noted that the cluster center refers to the core representation vector of each cluster, which is obtained by taking the average of all fused feature vectors within the same cluster according to their dimensions, and reflects the typical characteristics of the cluster's conversation patterns.

[0130] Cosine similarity is a metric that measures the degree of similarity between two high-dimensional vectors.

[0131] In this embodiment of the invention, the cluster center vector of each cluster is extracted, and then each fused feature vector and the center vector of its respective cluster are substituted into the cosine similarity formula. The semantic and attribute similarity between the two vectors is quantified by calculating the cosine value of the angle between them. The closer the similarity value is to 1, the higher the fit between the conversation segment and the corresponding clustering pattern. The specific formula is as follows: ; In the formula, It is the first i The fused feature vector of each session; It is the first k Cluster centers (mean vectors of Gaussian mixture models) corresponding to each session pattern.

[0132] Step S207: Based on cosine similarity, identify the multiple conversation segments with the highest similarity as representative segments, and output the complete text data of the representative segments.

[0133] It should be noted that representative segments refer to conversation segments selected from the same cluster (corresponding to a certain conversation pattern) that rank high in cosine similarity and best reflect the core characteristics of the conversation pattern of that cluster. They are typical and representative, and are concrete samples of the conversation pattern.

[0134] Complete text data refers to structured text data that includes all conversational statements in representative segments, teacher and student role identifiers, chronological order, and contextual information, and can fully reconstruct the conversational scenario, interaction logic, and core semantics.

[0135] In this embodiment of the invention, by calculating the cosine similarity between each conversation fragment sample and the cluster center, multiple conversation fragments with high similarity are found, and the complete original text of the representative fragment is output, thereby providing interpretable and specific case support for the conversation pattern.

[0136] This embodiment also provides an automated classroom conversation pattern analysis device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0137] This embodiment provides an automated analysis device for classroom conversation patterns, such as... Figure 5 As shown, it includes: Extraction module 301 is used to extract the audio files from the classroom video and transcribe the audio files into text data; The segmentation module 302 is used to segment audio files and text data according to conversation segments, generating multiple sub-audio files and multiple sub-text data. Combined with module 303, it is used to generate macro features by combining the conversation subject of each sub-audio file, the conversation hierarchy and conversation type of each sub-text data; The input module 304 is used to convert each sentence of each sub-text data into semantic features, and to fuse the semantic features and macro features of the same conversation segment to generate multiple fused feature vectors and input them into the preset target Gaussian mixture model; The recognition module 305 is used to identify each fused feature vector through the target Gaussian mixture model to obtain the classroom conversation mode of the classroom video.

[0138] In some alternative implementations, the extraction module 301 includes: The acquisition unit is used to acquire classroom videos; The separation unit is used to separate the audio from the classroom video using multimedia video encoding tools to obtain the audio file of the classroom video; Transcription unit, used to transcribe audio files into text data.

[0139] In some alternative implementations, the incorporation module 303 includes: The differentiation unit is used to distinguish the conversation subject in each sub-audio file; The conversation hierarchy unit is used to determine the conversation hierarchy of each sub-text data based on the conceptual and practical levels of the conversation content in each sub-text data. The conversation type unit is used to determine the conversation type of each sub-text data based on the teaching behavior, questioning behavior, feedback behavior, instruction behavior, sharing behavior and supplementary behavior in the conversation process of each sub-text data; The macro feature unit is used to generate macro features by taking all session subjects, session levels, and session types.

[0140] In some alternative implementations, the input module 304 includes: The input unit is used to input the sub-text data into a preset sentence embedding model; The transformation unit is used to transform each sentence of each sub-text data into a high-dimensional vector through a sentence embedding model, thereby generating semantic features. The fusion unit is used to fuse the semantic features and macroscopic features of the same conversation segment based on the multi-head attention mechanism to generate multiple fused feature vectors; The input model unit is used to input all fused feature vectors into a preset target Gaussian mixture model.

[0141] In some alternative implementations, the device further includes, before inputting the model unit: The initialization unit is used to initialize the model parameters of the preset initial Gaussian mixture model and generate an updated Gaussian mixture model. The solution unit is used to iteratively solve the updated Gaussian mixture model using the expectation-maximization algorithm, and determine the optimal model parameters based on the solution results; The update unit is used to update the new Gaussian mixture model with the optimal model parameters to obtain the target Gaussian mixture model.

[0142] In some alternative implementations, the identification module 305 includes: The probability computation unit is used to calculate the posterior probability of each fused feature vector using the optimal model parameters of the target Gaussian mixture model. The classification unit is used to classify each fused feature vector into the corresponding session mode with the highest posterior probability. Clustering units are used to cluster all fused feature vectors to obtain the classroom conversation patterns of the classroom videos.

[0143] In some alternative embodiments, the device further includes: The computational unit is used to calculate the cosine similarity between the conversation fragments corresponding to each fused feature vector and the cluster centers; The output unit is used to identify the most similar conversation segments based on cosine similarity as representative segments and output the complete text data of the representative segments.

[0144] The automated classroom conversation pattern analysis device provided in this embodiment of the invention can execute the automated classroom conversation pattern analysis method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0145] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0146] The following is a detailed reference. Figure 6 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0147] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0148] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a memory 408, or installed from a ROM 402. When the computer program is executed by the processor 401, it performs the functions defined in the automated classroom conversation pattern analysis method of the embodiments of the present invention.

[0149] Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0150] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the automated classroom conversation pattern analysis method shown in the above embodiments is implemented.

[0151] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0152] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for automatically analyzing classroom conversation patterns, characterized in that, The method includes: Extract the audio files from the classroom video and transcribe the audio files into text data; The audio files and text data are divided according to the conversation segments to generate multiple sub-audio files and multiple sub-text data. By combining the conversation subject of each sub-audio file, the conversation hierarchy and conversation type of each sub-text data, macro-features are generated; Each sentence of each of the sub-text data is transformed into a semantic feature, and the semantic features and macro features of the same conversation segment are fused to generate multiple fused feature vectors, which are then input into a preset target Gaussian mixture model. The classroom conversation mode of the classroom video is obtained by identifying each of the fused feature vectors through the target Gaussian mixture model.

2. The method according to claim 1, characterized in that, The process of extracting the audio file from the classroom video and transcribing the audio file into text data includes: Get the class videos; The audio of the classroom video is separated using a multimedia video encoding tool to obtain the audio file of the classroom video; The audio file is transcribed into text data.

3. The method according to claim 1, characterized in that, The process of combining the session subject of each sub-audio file, the session hierarchy and session type of each sub-text data to generate macro-features includes: Distinguish the conversation subject in each of the sub-audio files; Based on the conceptual and practical levels of the conversation content in each of the sub-text data, the conversation level of each of the sub-text data is determined; Based on the teaching behavior, questioning behavior, feedback behavior, instruction behavior, sharing behavior, and supplementary behavior in the conversation process of each sub-text data, determine the conversation type of each sub-text data; Macro-level features are generated using all the aforementioned session subjects, session levels, and session types.

4. The method according to claim 1, characterized in that, The process of converting each sentence of each of the sub-text data into semantic features, fusing the semantic features and macro-features of the same conversation segment to generate multiple fused feature vectors, and inputting them into a preset target Gaussian mixture model includes: Each of the sub-text data is input into a preset sentence embedding model; The sentence embedding model is used to transform each sentence of each sub-text data into a high-dimensional vector to generate semantic features. Based on the multi-head attention mechanism, the semantic features and macro-features of the same conversation segment are fused to generate multiple fused feature vectors; Input all the fused feature vectors into the preset target Gaussian mixture model.

5. The method according to claim 4, characterized in that, Before the step of inputting all the fused feature vectors into a preset target Gaussian mixture model, the method further includes: The model parameters of the preset initial Gaussian mixture model are initialized to generate an updated Gaussian mixture model; The updated Gaussian mixture model is solved iteratively using the expectation-maximization algorithm, and the optimal model parameters are determined based on the solution results. The updated Gaussian mixture model is updated using the optimal model parameters to obtain the target Gaussian mixture model.

6. The method according to claim 1, characterized in that, The step of identifying the fused feature vectors through the target Gaussian mixture model to obtain the classroom conversation mode of the classroom video includes: The posterior probability of each fused feature vector is calculated using the optimal model parameters of the target Gaussian mixture model. Each of the fused feature vectors is categorized into the corresponding session mode with the highest posterior probability. Clustering is performed on all the fused feature vectors to obtain the classroom conversation pattern of the classroom video.

7. The method according to claim 6, characterized in that, The method further includes: Calculate the cosine similarity between the conversation fragments corresponding to each of the fused feature vectors and the cluster centers; Based on the cosine similarity, the multiple conversation segments with the highest similarity are identified as representative segments, and the complete text data of the representative segments is output.

8. An automated classroom conversation pattern analysis device, characterized in that, The device includes: The extraction module is used to extract the audio files from the classroom video and transcribe the audio files into text data; The segmentation module is used to segment the audio file and the text data according to the conversation segments, and generate multiple sub-audio files and multiple sub-text data. The module is used to combine the session subject of each sub-audio file, the session hierarchy and session type of each sub-text data to generate macro-features; The input module is used to convert each sentence of each of the sub-text data into semantic features, and to fuse the semantic features and macro features of the same conversation segment to generate multiple fused feature vectors and input them into a preset target Gaussian mixture model; The recognition module is used to identify each of the fused feature vectors through the target Gaussian mixture model to obtain the classroom conversation mode of the classroom video.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the classroom conversation pattern automated analysis method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the classroom conversation pattern automated analysis method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions for causing a computer to execute the classroom conversation pattern automated analysis method according to any one of claims 1 to 7.

Images