Classroom behavior classification method and device based on combined text classification and sequence labeling

Abstract

The embodiment of the present application discloses a classroom behavior classification method and device that combines text classification and sequence labeling; the method includes: obtaining teaching video data and performing speech transcription on the data to form an initial corpus; preprocessing the initial corpus to obtain chapter data, and inputting the chapter data into an unsupervised pre-training model for incremental pre-training; constructing a joint loss function of text classification and sequence labeling to train the chapter data, and using a supervised pre-training model to predict the chapter data to obtain a teaching behavior classification label matrix; analyzing the teaching behavior classification label matrix to obtain a classroom behavior classification that combines text classification and sequence labeling; the embodiment of the present application realizes rapid and accurate analysis and classification of computing resources and data volume, and the classroom teaching behavior classification effect is efficient and accurate.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of classroom behavior classification, and in particular to a classroom behavior classification method and device that combines text classification and sequence labeling. Background Art

[0002] Classroom teaching reform is entering a phase focused on promoting deep learning. This requires not only evaluating the effectiveness of external teaching methods, such as independent and collaborative inquiry, but also analyzing the effectiveness of the teaching language, which occupies the largest portion of classroom time. This analysis aims to assess the effectiveness of teaching language in promoting student self-directed participation and the development of higher-order thinking skills. American scholar Flanders noted that teachers and students primarily communicate through language in classroom teaching activities, with language accounting for 80% of all classroom teaching activities. However, traditional teaching language analysis is no longer sufficient to meet the needs of education in today's online classroom environment. Traditional teaching language analysis requires manual recording, followed by manual coding and annotation. However, AI-based speech recognition and natural language processing technologies can now replace this process. Empowered by more efficient and accurate information technology, classroom language information from online classes and other classroom recordings can be segmented, identified, classified, coded, and statistically analyzed. This allows for objective, detailed, and comprehensive teaching evaluations of both new and experienced teachers. However, existing classroom teaching analysis technologies lack the computing resources and data volume to rapidly and accurately analyze and classify, resulting in poor classification and prediction performance. Summary of the Invention

[0003] The embodiments of the present application provide a classroom behavior classification method and device that combines text classification and sequence labeling, which can achieve rapid and accurate analysis and classification of computing resources and data volume, and the classroom teaching behavior classification effect is efficient and accurate.

[0004] In a first aspect, an embodiment of the present application provides a classroom behavior classification method combining text classification and sequence labeling, the method comprising the following steps:

[0005] Obtain teaching video data and perform voice transcription to form the initial corpus;

[0006] Preprocessing the initial corpus to obtain passage data, and inputting the passage data into an unsupervised pre-training model for incremental pre-training;

[0007] Constructing a joint loss function of text classification and sequence labeling to train the passage data, using a supervised pre-training model to predict the passage data, and obtaining a teaching behavior classification label matrix;

[0008] The teaching behavior classification label matrix is ​​analyzed to obtain a classroom behavior classification based on joint text classification and sequence labeling.

[0009] Furthermore, before inputting the passage data into the unsupervised pre-training model for incremental pre-training, the method further includes:

[0010] Construct sentence-level contrastive learning samples, use the softmax classifier to predict the three types of loss functions for the contrastive learning samples and add them together to obtain the overall loss function of the NSP task:

[0011] L cNSP =αL NSP +βL PSP +σL RSP ;

[0012]

[0013] Among them, L cNSP is the overall loss function of the NSP task after the contrastive learning method is improved; L NSP 、L PSP and L RSP are the loss functions of the three subtasks under the cNSP task; α, β, and σ are the weight coefficients corresponding to the three subtasks in the total loss function; h is the output of the last hidden layer of the neural network under different subtasks; softmax() represents the use of the softmax function for normalization and the calculation of the cross entropy loss of the results;

[0014] Construct word-level contrastive learning samples. The loss function of the word-level task is:

[0015]

[0016] Among them, L cMLM represents the overall loss function of the MLM task after contrastive learning, θ represents the parameter set of the model, k represents the length of each disrupted subsequence, and pos represents the position embedding of the specified word position in the Bert model input.

[0017] Furthermore, the step of inputting the passage data into an unsupervised pre-training model for incremental pre-training includes:

[0018] The loss function of the unsupervised training phase is:

[0019] L=η1L cNSP +η2L cMLM ;

[0020] Among them, η1 and η2 are the weight coefficients corresponding to the subtasks;

[0021] The passage data is input into the loss function of the unsupervised training phase to complete incremental training.

[0022] Furthermore, the construction of a joint loss function for text classification and sequence labeling to train the passage data includes:

[0023] In the sequence labeling task, the Bi-LSTM+CRF model structure is used to complete this task. Therefore, the loss function in the unsupervised training phase is generated by CRF. The loss function of the sequence labeling task is:

[0024] y st =CRF(H st );

[0025] Among them, H st is the [CLS] sequence obtained after Bert encoding, CRF() represents the st Then a conditional random field is processed to obtain y st ;

[0026] Loss function for text classification task:

[0027] Among them, W lc is the weight matrix in the classification task, H lc is the sentence text representation for text classification tasks, b lc is the offset, so we get y lc

[0028] The joint loss function of training is obtained by the joint loss function of the two tasks of sequence labeling and text classification:

[0029] y all =αy lc +βy st ;

[0030] The passage data is input into the joint loss function for training.

[0031] Furthermore, the supervised pre-training model is used to predict the passage data to obtain a teaching behavior classification label matrix, including:

[0032] After the training is completed, the supervised pre-training model is used to predict the passage data to obtain a set of label data;

[0033] Arrange the chapter data according to the template of the teaching behavior classification label matrix to obtain the teaching behavior classification label matrix W teach ;

[0034] The teaching behavior classification label matrix is ​​an m×n matrix. For classes of different lengths, the size of the matrix (i.e., the size of m and n) is different. m is called a sentence batch and n is called a sentence batch. The entire matrix describes the arrangement order of teaching behavior labels in batches of n identical batches (m).

[0035] Furthermore, the teaching behavior classification label matrix is ​​analyzed to obtain classroom behavior classification by combining text classification and sequence labeling, including:

[0036] Calculate the percentage of teacher-student language in the total classroom language;

[0037] Count the duration of teacher-student interaction;

[0038] Calculate the percentage of specific teaching behaviors in the total teaching behaviors;

[0039] The classroom teaching model is analyzed based on the percentage of teacher-student language in the total classroom language, the duration of teacher-student interaction, and the percentage of specific teaching behaviors in the overall teaching behaviors.

[0040] Furthermore, the preprocessing of the initial corpus includes:

[0041] Performing a first segmentation operation on the initial corpus to obtain first segmentation data;

[0042] A second segmentation operation is performed on the first segmented data to obtain chapter data.

[0043] In a second aspect, a classroom behavior classification device combining text classification and sequence labeling includes:

[0044] The data acquisition module is used to obtain teaching video data and perform speech transcription to form the initial corpus;

[0045] A first processing module is configured to pre-process the initial corpus to obtain passage data, and input the passage data into an unsupervised pre-training model for incremental pre-training;

[0046] The second processing module is used to construct a joint loss function of text classification and sequence labeling to train the passage data, use a supervised pre-training model to predict the passage data, and obtain a teaching behavior classification label matrix;

[0047] The behavior classification module is used to analyze the teaching behavior classification label matrix to obtain classroom behavior classification by combining text classification and sequence labeling.

[0048] In a third aspect, an embodiment of the present application further provides a computer device, comprising: a memory and one or more processors;

[0049] The memory is used to store one or more programs;

[0050] When the one or more programs are executed by the one or more processors, the one or more processors implement the classroom behavior classification method that combines text classification and sequence labeling as described above.

[0051] In a fourth aspect, an embodiment of the present application further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute a classroom behavior classification method that combines text classification and sequence labeling as described above.

[0052] The embodiment of the present application forms an initial corpus by acquiring teaching video data and performing speech transcription on the data; pre-processing the initial corpus to obtain chapter data, and inputting the chapter data into an unsupervised pre-training model for incremental pre-training; constructing a joint loss function of text classification and sequence labeling to train the chapter data, and using a supervised pre-training model to predict the chapter data to obtain a teaching behavior classification label matrix; analyzing the teaching behavior classification label matrix to obtain a classroom behavior classification of combined text classification and sequence labeling; achieving rapid and accurate analysis and classification of computing resources and data volume, and achieving efficient and accurate classroom teaching behavior classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flowchart of a classroom behavior classification method combining text classification and sequence labeling provided in an embodiment of the present application;

[0054] Figure 2 This is a diagram of the algorithm model framework provided in the embodiments of the present application;

[0055] Figure 3 A diagram showing the effect of using the teaching behavior classification label matrix to perform teaching evaluation according to an embodiment of the present application;

[0056] Figure 4 This is a structural diagram of a classroom behavior classification device that combines text classification and sequence labeling provided by an embodiment of the present application;

[0057] Figure 5 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only parts related to the present application, not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0059] The embodiment of the present application establishes a classroom behavior classification method that combines text classification and sequence labeling to achieve rapid and accurate analysis and classification of computing resources and data volume, and the classroom teaching behavior classification effect is efficient and accurate.

[0060] The combined text classification and sequence labeling classroom behavior classification method provided in the embodiments can be performed by a combined text classification and sequence labeling classroom behavior classification device. The combined text classification and sequence labeling classroom behavior classification device can be implemented in software and / or hardware and integrated into a combined text classification and sequence labeling classroom behavior classification device. The combined text classification and sequence labeling classroom behavior classification device can be a computer or other device.

[0061] Figure 1 This is a flowchart of a classroom behavior classification method that combines text classification and sequence labeling provided in an embodiment of the present application. Figure 1 , the method comprises the following steps:

[0062] Step 110: Obtain teaching video data and perform voice transcription on it to form initial corpus.

[0063] For example, since the main object of the teaching evaluation in the embodiment of the present application is the course, when performing data preprocessing on the initial corpus, it is first necessary to segment the entire corpus into individual chapter-level classroom language texts. On this basis, the individual chapter-level classroom language texts are segmented into sentences, where the length of the sentences generally does not exceed 64 words. For sentences with more than 64 words, the embodiment of the present application adopts a segmentation method. Since simple segmentation may destroy the semantic coherence of the sentence, the embodiment of the present application completes this work through manual annotation.

[0064] Step 120: Preprocess the initial corpus to obtain passage data, and input the passage data into an unsupervised pre-training model for incremental pre-training.

[0065] Specifically, the preprocessing of the initial corpus includes: performing a first segmentation operation on the initial corpus to obtain first segmentation data;

[0066] A second segmentation operation is performed on the first segmented data to obtain chapter data.

[0067] Specifically, before inputting the passage data into the unsupervised pre-training model for incremental pre-training, the method further includes:

[0068] Build an unsupervised pre-training model, specifically:

[0069] Construct sentence-level contrastive learning samples, use the softmax classifier to predict the three types of loss functions for the contrastive learning samples and add them together to obtain the overall loss function of the NSP task:

[0070] L cNSP =αL NSP +βL PSP +σL RSP ;

[0071]

[0072] Among them, L cNSP is the overall loss function of the NSP task after the contrastive learning method is improved; L NSP 、L PSP and L RSP are the loss functions of the three subtasks under the cNSP task; α, β, and σ are the weight coefficients corresponding to the three subtasks in the total loss function; h is the output of the last hidden layer of the neural network under different subtasks; softmax() represents the use of the softmax function for normalization and the calculation of the cross entropy loss of the results;

[0073] Construct word-level contrastive learning samples. The loss function of the word-level task is:

[0074]

[0075] Among them, L cMLM represents the overall loss function of the MLM task after contrastive learning, θ represents the parameter set of the model, k represents the length of each disrupted subsequence, and pos represents the position embedding of the specified word position in the Bert model input.

[0076] Furthermore, the step of inputting the passage data into an unsupervised pre-training model for incremental pre-training includes:

[0077] The loss function of the unsupervised training phase is:

[0078] L=η1L cNSP +η2L cMLM ;

[0079] Among them, η1 and η2 are the weight coefficients corresponding to the subtasks;

[0080] The passage data is input into the loss function of the unsupervised training phase to complete incremental training.

[0081] Step 130: Construct a joint loss function of text classification and sequence labeling to train the passage data, use a supervised pre-training model to predict the passage data, and obtain a teaching behavior classification label matrix.

[0082] Since classroom language text data is both chapter-level text data that can be split into single sentences and classified sentence by sentence, and sentence-type sequence data with its own logic and sequence, it can simultaneously perform two natural language processing prediction tasks: text classification and sequence labeling.

[0083] For details, please refer to Figure 2 , the construction of a joint loss function for text classification and sequence labeling to train the passage data includes:

[0084] In the sequence labeling task, the Bi-LSTM+CRF model structure is used to complete this task. Therefore, the loss function in the unsupervised training phase is generated by CRF. The loss function of the sequence labeling task is:

[0085] y st =CRF(H st );

[0086] Among them, H st is the [CLS] sequence obtained after Bert encoding, CRF() represents the st Then a conditional random field is processed to obtain y st ;

[0087] Loss function for text classification task:

[0088] Among them, W lc is the weight matrix in the classification task, H lc is the sentence text representation for text classification tasks, b lc is the offset, so we get y lc

[0089] The joint loss function of training is obtained by the joint loss function of the two tasks of sequence labeling and text classification:

[0090] y all =αy lc +βy st ;

[0091] The passage data is input into the joint loss function for training.

[0092] Furthermore, the supervised pre-training model is used to predict the passage data to obtain a teaching behavior classification label matrix, including:

[0093] After the training is completed, the supervised pre-training model is used to predict the passage data to obtain a set of label data;

[0094] Arrange the chapter data according to the template of the teaching behavior classification label matrix to obtain the teaching behavior classification label matrix W teach ;

[0095] The teaching behavior classification label matrix is ​​an m×n matrix. For classes of different lengths, the size of the matrix (i.e., the size of m and n) is different. m is called a sentence batch and n is called a sentence batch. The entire matrix describes the arrangement order of teaching behavior labels in batches of n identical batches (m).

[0096] For example, the model's input is manually pre-labeled, sequential, chapter-level classroom language data. During the unsupervised training phase, no data labeling is required; the model simply feeds the text sequentially into the model to perform two pre-training tasks. For each MLM task, the contrastive learning constructs positive and negative samples as follows: one-third of the words maintain their normal order and are masked by the [mask] marker; one-third of the words have their current order swapped and are masked by the [mask] marker; and one-third of the words are replaced by random words and masked by the [mask] marker. For each NSP task, the contrastive learning constructs positive and negative samples as follows: one-third of the sentences retain their original task and predict the next sentence (NSP); one-third of the sentences predict the previous sentence (PSP); and one-third of the sentences predict a randomly replaced previous or next sentence (RSP), significantly improving the model's robustness.

[0097] It is understandable that because teaching language texts are continuous, logical, and cannot be arbitrarily reordered, the classification of a continuous classroom language text can be considered a sequence labeling task on top of the simple task of classifying teaching behavior texts. This embodiment of the application creatively proposes a model architecture that performs two tasks simultaneously, and improves the model's task effectiveness and performance for teaching behavior classification by combining a dual-task loss function.

[0098] It is understandable that since the embodiment of the present application selects a pre-trained BERT model, in the unsupervised pre-training stage, the embodiment of the present application can use a large amount of unlabeled educational corpus to incrementally pre-train the BERT model, which can effectively improve the performance of the model on subsequent downstream tasks. During the pre-training process, the embodiment of the present application uses contrastive learning to construct positive and negative samples to enhance the robustness and generalization ability of the model. Since the downstream task is a dual-task framework of text classification and sequence labeling, the two pre-training tasks of MLM and NSP just strengthen the vector representation of the model from the perspective of words and sentences, breaking the previous pre-training method that only focuses on MLM and tapping the potential value of the NSP task.

[0099] Step 140: Analyze the teaching behavior classification label matrix to obtain classroom behavior classification based on combined text classification and sequence labeling.

[0100] Specifically, the percentage of teacher-student language in the total classroom language is counted; the duration of teacher-student interaction is counted; and the percentage of specific teaching behaviors in the overall teaching behaviors is counted.

[0101] The classroom teaching model is analyzed based on the percentage of teacher-student language in the total classroom language, the duration of teacher-student interaction, and the percentage of specific teaching behaviors in the overall teaching behaviors.

[0102] For example, classroom language is first divided into two categories: teacher and student. Teacher language is divided into six modules: questioning, lecturing, directing, accepting student perspectives, responding to student questions, and answering one's own questions. Teacher questioning is further divided into five modules: questions focused on teaching content, questions that stimulate metacognition, questions that stimulate student participation, questions focused on evaluation, and other. Questions focused on teaching content are further divided into three modules: questions requiring students to answer yes or no, questions requiring students to simply name or state, and questions requiring students to describe or explain. Student language is divided into five modules: answering, questioning, lecturing, directing, and accepting. As these two examples demonstrate, categorizing and coding classroom language behavior according to certain standards facilitates automated classification using text classification technology. Once we have scientific and reasonable coding indicators for classroom language behavior, we can accurately, scientifically, and multi-dimensionally categorize and organize classroom language and derive meaningful educational information from it.

[0103] Figure 3 The model outputs a classification matrix of teaching behaviors, along with a method for evaluating teaching using this matrix. This matrix allows for statistical analysis of the order, timing, patterns, and frequency of various teaching behaviors within a class. This information can then be used to infer the teacher's classroom style and corresponding teaching effectiveness. The label matrix is ​​arranged according to the course of the class. This allows analysis not only of the proportions of various teacher and student behaviors, but also of key teacher behaviors during different time periods and their impact on students, ultimately contributing to a judgment of the overall teaching model.

[0104] For example, the percentage of language spoken by teachers and students in the total classroom language can be counted: assuming that the total language in a class consists of 5,000 sentences, then the labels of these 5,000 sentences are included. By counting the number of behavior labels from teachers and students, we can get the proportion of each teacher's and students' behaviors in the classroom, and use this to determine whether the class is more teacher-led or student-led.

[0105] For example, the duration of interaction between teachers and students is counted: when teachers and students interact, the behavior labels of teachers and students will change alternately in a sequence in the matrix. By counting the length of the changing sequence, the duration of interaction between teachers and students in a class can be obtained.

[0106] For example, the percentage of a certain teaching behavior in the total teaching behaviors can be counted: a key teaching behavior can be extracted from the matrix, such as the teacher's technical operation, and then the teacher's time in the whole class showing the technical operation to the students can be analyzed, which will pave the way for analyzing the value of a certain behavior of the teacher to the actual effectiveness of the class.

[0107] For example, overall analysis of classroom teaching mode: through the combination of different label types, we can know that the proportion of various teaching behaviors of lecture-oriented, independent inquiry-oriented, and question-and-answer interactive teachers in the classroom is different. By counting the proportion of different behavioral label type combinations, we can comprehensively analyze a teacher's classroom teaching mode in a class, and then provide help for teachers to improve their classrooms and for new and old teachers to learn from each other.

[0108] As mentioned above, the model proposed in the embodiment of the present application uses a pre-training model framework of upstream unsupervised learning + downstream supervised learning, adopts the contrastive learning method to construct positive and negative samples in the unsupervised learning part, and uses unlabeled data in the field of education to perform incremental pre-training on the model, which significantly improves the model's classification prediction performance.

[0109] The embodiment of the present application takes into account the particularity of classroom language texts and creatively combines the two tasks of sequence labeling and text classification to solve the problem of classroom behavior classification, making full use of the value of the NSP pre-training task in the unsupervised learning stage, which has previously received less attention from the relevant academic community.

[0110] The embodiment of the present application constructs a teaching behavior label matrix, combines artificial intelligence models and algorithms with classroom behavior analysis, and broadens the tools and approaches for teaching behavior analysis. Automated teaching evaluation using artificial intelligence is more efficient and objective than manual teaching evaluation, and helps to analyze a large number of courses at one time and filter out the required teaching evaluation information. Since the algorithm is based on the teaching evaluation results obtained by analyzing the teaching language, it also has certain reference value and reference significance for process teaching evaluation.

[0111] Based on the above embodiment, FIG4 is a structural diagram of a classroom behavior classification device for combining text classification and sequence labeling provided by an embodiment of the present application. Figure 4 The classroom behavior classification device for combined text classification and sequence labeling provided in this embodiment specifically includes: a data acquisition module 101, a first processing module 102, a second processing module 103 and a behavior classification module 104.

[0112] Among them, the data acquisition module is used to acquire teaching video data and perform speech transcription on it to form an initial corpus; the first processing module is used to pre-process the initial corpus to obtain chapter data, and input the chapter data into the unsupervised pre-training model for incremental pre-training; the second processing module is used to construct a joint loss function of text classification and sequence labeling to train the chapter data, and use the supervised pre-training model to predict the chapter data to obtain a teaching behavior classification label matrix; the behavior classification module is used to analyze the teaching behavior classification label matrix to obtain a classroom behavior classification of joint text classification and sequence labeling.

[0113] In the above, the teaching video data is obtained and voice-transcribed to form an initial corpus; the initial corpus is pre-processed to obtain chapter data, and the chapter data is input into the unsupervised pre-training model for incremental pre-training; a joint loss function of text classification and sequence labeling is constructed to train the chapter data, and the supervised pre-training model is used to predict the chapter data to obtain a teaching behavior classification label matrix; the teaching behavior classification label matrix is ​​analyzed to obtain a classroom behavior classification of joint text classification and sequence labeling; computing resources and data volume are quickly and accurately analyzed and classified, and the classroom teaching behavior classification effect is efficient and accurate.

[0114] The classroom behavior classification device for combined text classification and sequence labeling provided in the embodiment of the present application can be used to execute the classroom behavior classification method for combined text classification and sequence labeling provided in the above embodiment, and has corresponding functions and beneficial effects.

[0115] An embodiment of the present application also provides a computer device that can integrate the classroom behavior classification device for combined text classification and sequence labeling provided in an embodiment of the present application. Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 5 The computer device includes: an input device 43, an output device 44, a memory 42, and one or more processors 41; the memory 42 is used to store one or more programs; when the one or more programs are executed by the one or more processors 41, the one or more processors 41 implement the classroom behavior classification method of combining text classification and sequence labeling as provided in the above embodiment. The input device 43, the output device 44, the memory 42, and the processor 41 can be connected by a bus or other means. Figure 5 The bus connection is taken as an example.

[0116] The processor 41 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 42, that is, realizes the above-mentioned classroom behavior classification method of combined text classification and sequence labeling.

[0117] The computer device provided above can be used to execute the classroom behavior classification method of combined text classification and sequence labeling provided in the above embodiment, and has corresponding functions and beneficial effects.

[0118] An embodiment of the present application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute a classroom behavior classification method that combines text classification and sequence labeling. The classroom behavior classification method that combines text classification and sequence labeling includes: obtaining teaching video data and performing speech transcription on the data to form an initial corpus; preprocessing the initial corpus to obtain chapter data, and inputting the chapter data into an unsupervised pre-training model for incremental pre-training; constructing a joint loss function of text classification and sequence labeling to train the chapter data, and using a supervised pre-training model to predict the chapter data to obtain a teaching behavior classification label matrix; analyzing the teaching behavior classification label matrix to obtain a classroom behavior classification that combines text classification and sequence labeling.

[0119] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROMs, floppy disks, or tape devices; computer device memory or random access memory, such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer device where the program is executed, or it may be located in a different second computer device that is connected to the first computer device via a network (such as the Internet). The second computer device can provide program instructions to the first computer for execution. The term "storage medium" can include two or more storage media that can reside in different locations (e.g., in different computer devices connected via a network). The storage medium can store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

[0120] Of course, the storage medium containing computer-executable instructions provided in an embodiment of the present application, whose computer-executable instructions are not limited to the classroom behavior classification method of combined text classification and sequence labeling as described above, can also execute related operations in the classroom behavior classification method of combined text classification and sequence labeling provided in any embodiment of the present application.

[0121] The classroom behavior classification device, storage medium and computer equipment for combined text classification and sequence labeling provided in the above embodiments can execute the classroom behavior classification method for combined text classification and sequence labeling provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the classroom behavior classification method for combined text classification and sequence labeling provided in any embodiment of the present application.

[0122] The above are only preferred embodiments of the present application and the technical principles employed. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that are possible for those skilled in the art will not depart from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include more other equivalent embodiments without departing from the concept of the present application. The scope of the present application is determined by the scope of the claims.

Claims

1. A classroom behavior classification method combining text classification and sequence labeling, characterized by: The method comprises the following steps: Obtain teaching video data and perform voice transcription to form the initial corpus; Preprocessing the initial corpus to obtain passage data, and inputting the passage data into an unsupervised pre-training model for incremental pre-training; Constructing a joint loss function of text classification and sequence labeling to train the passage data, using a supervised pre-training model to predict the passage data, and obtaining a teaching behavior classification label matrix; Analyzing the teaching behavior classification label matrix to obtain classroom behavior classification based on combined text classification and sequence labeling; Before inputting the chapter data into the unsupervised pre-training model for incremental pre-training, the method further includes: Construct sentence-level contrastive learning samples, use the softmax classifier to predict the three types of loss functions for the contrastive learning samples and add them together to obtain the overall loss function of the NSP task: L cNSP =αL NSP +βL PSP +σL RSP ; Among them, L cNSP is the overall loss function of the NSP task after the contrastive learning method is improved; L NSP , L PSP and L RSP are the loss functions of the three subtasks under the cNSP task; α, β, and σ are the weight coefficients corresponding to the three subtasks in the total loss function; h is the output of the last hidden layer of the neural network under different subtasks; softmax() represents the use of the softmax function for normalization and the calculation of the cross entropy loss of the results; Construct word-level contrastive learning samples. The loss function of the word-level task is: Among them, L cMLM represents the overall loss function of the MLM task after contrastive learning, θ represents the parameter set of the model, k represents the length of each disrupted subsequence, and pos represents the position embedding of the specified word position in the Bert model input; The step of constructing a joint loss function for text classification and sequence labeling to train the passage data includes: In the sequence labeling task, the Bi-LSTM+CRF model structure is used to complete this task. Therefore, the loss function in the unsupervised training phase is generated by CRF. The loss function of the sequence labeling task is: the st =CRF(H st ); Among them, H st is the [CLS] sequence obtained after Bert encoding, CRF() represents the st Then a conditional random field is processed to obtain y st ; Loss function for text classification task: Among them, W lc is the weight matrix in the classification task, H lc is the sentence text representation for text classification tasks, b lc is the offset, so we get y lc The joint loss function of training is obtained by the joint loss function of the two tasks of sequence labeling and text classification: the all =αy lc +βy st 4 Inputting the passage data into the joint loss function for training; The supervised pre-training model is used to predict the passage data to obtain a teaching behavior classification label matrix, including: After the training is completed, the supervised pre-training model is used to predict the passage data to obtain a set of label data; Arrange the chapter data according to the template of the teaching behavior classification label matrix to obtain the teaching behavior classification label matrix W teach ; The teaching behavior classification label matrix is ​​an m×n matrix. For classes of different lengths, the size of the matrix is ​​different, that is, the size of m and n. m is called a sentence batch and n is called a sentence batch. The entire matrix describes the arrangement order of teaching behavior labels in n batches of the same batch size m.

2. The classroom behavior classification method based on combined text classification and sequence labeling according to claim 1 is characterized in that: Inputting the passage data into the unsupervised pre-training model for incremental pre-training includes: The loss function in the unsupervised training phase is: L=η1L cNSP +η2L cMLM ; Among them, η1 and η2 are the weight coefficients corresponding to the subtasks; The passage data is input into the loss function of the unsupervised training phase to complete incremental training.

3. The classroom behavior classification method combining text classification and sequence labeling according to claim 1 is characterized in that: The teaching behavior classification label matrix is ​​analyzed to obtain classroom behavior classification by combining text classification and sequence labeling, including: Calculate the percentage of teacher-student language in the total classroom language; Count the duration of teacher-student interaction; Calculate the percentage of specific teaching behaviors in the total teaching behaviors; The classroom teaching model is analyzed based on the percentage of teacher-student language in the total classroom language, the duration of teacher-student interaction, and the percentage of specific teaching behaviors in the overall teaching behaviors.

4. The classroom behavior classification method combining text classification and sequence labeling according to claim 1 is characterized in that: The preprocessing of the initial corpus includes: Performing a first segmentation operation on the initial corpus to obtain first segmentation data; A second segmentation operation is performed on the first segmented data to obtain chapter data.

5. A classroom behavior classification device combining text classification and sequence labeling, characterized in that: include: The data acquisition module is used to obtain teaching video data and perform speech transcription to form the initial corpus; A first processing module is configured to pre-process the initial corpus to obtain passage data, and input the passage data into an unsupervised pre-training model for incremental pre-training; The second processing module is used to construct a joint loss function of text classification and sequence labeling to train the passage data, use a supervised pre-training model to predict the passage data, and obtain a teaching behavior classification label matrix; A behavior classification module is used to analyze the teaching behavior classification label matrix to obtain classroom behavior classification based on combined text classification and sequence labeling; Before inputting the chapter data into the unsupervised pre-training model for incremental pre-training, the method further includes: Construct sentence-level contrastive learning samples, use the softmax classifier to predict the three types of loss functions for the contrastive learning samples and add them together to obtain the overall loss function of the NSP task: L cNSP =αL NSP +βL PSP +σL RSP ; Among them, L cNSP is the overall loss function of the NSP task after the contrastive learning method is improved; L NSP , L PSP and L RSP are the loss functions of the three subtasks under the cNSP task; α, β, and σ are the weight coefficients corresponding to the three subtasks in the total loss function; h is the output of the last hidden layer of the neural network under different subtasks; softmax() represents the use of the softmax function for normalization and the calculation of the cross entropy loss of the results; Construct word-level contrastive learning samples. The loss function of the word-level task is: Among them, L cMLM represents the overall loss function of the MLM task after contrastive learning, θ represents the parameter set of the model, k represents the length of each disrupted subsequence, and pos represents the position embedding of the specified word position in the Bert model input; The step of constructing a joint loss function for text classification and sequence labeling to train the passage data includes: In the sequence labeling task, the Bi-LSTM+CRF model structure is used to complete this task. Therefore, the loss function in the unsupervised training phase is generated by CRF. The loss function of the sequence labeling task is: the st =CRF(H st ); Among them, H st is the [CLS] sequence obtained after Bert encoding, CRF() represents the st Then a conditional random field is processed to obtain y st ; Loss function for text classification task: Among them, W lc is the weight matrix in the classification task, H lc is the sentence text representation for text classification tasks, b lc is the offset, so we get y lc The joint loss function of training is obtained by the joint loss function of the two tasks of sequence labeling and text classification: the all =αy lc +βy st 4 Inputting the passage data into the joint loss function for training; The supervised pre-training model is used to predict the passage data to obtain a teaching behavior classification label matrix, including: After the training is completed, the supervised pre-training model is used to predict the passage data to obtain a set of label data; Arrange the chapter data according to the template of the teaching behavior classification label matrix to obtain the teaching behavior classification label matrix W teach ; The teaching behavior classification label matrix is ​​an m×n matrix. For classes of different lengths, the size of the matrix is ​​different, that is, the size of m and n. m is called a sentence batch and n is called a sentence batch. The entire matrix describes the arrangement order of teaching behavior labels in n batches of the same batch size m.

6. A computer device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a classroom behavior classification method combining text classification and sequence labeling as described in any one of claims 1-4.

7. A storage medium containing computer-executable instructions, characterized in that: When executed by a computer processor, the computer executable instructions are used to execute a classroom behavior classification method that combines text classification and sequence labeling as described in any one of claims 1-4.

Images