Teaching humor language detection model based on transfer learning

Abstract

The invention discloses a teaching humor language detection model based on transfer learning, and belongs to the technical field of natural language processing. Comprising a transfer learning module, a TCCA fusion network and a long text semantic understanding module, the transfer learning module extracts context humor semantic features by using a text encoder of a ChineseClip model, and the TCCA fusion network module performs convolution, matrix operation, attention calculation and parameter aggregation on word vector features through an improved TCCA fusion network; and the long text semantic understanding module performs bidirectional modeling on the feature sequence output by the TCCA, captures a long-distance semantic relationship and semantic turning, and outputs a classification result. According to the method, a T-humor data set is further constructed, experiments are carried out on CCL2018, CCL2019 and T-humor data sets, and it is proved that the model can effectively extract humor semantic features and recognize semantic turning, is excellent in performance on indexes such as F1 scores and has good detection performance and generalization ability, and an effective solution is provided for automatic detection of teaching humor languages.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing (NLP), and in particular to a teaching humor language detection model based on transfer learning. Background Art

[0002] In classroom teaching, humor is an important manifestation of a teacher's linguistic artistry, effectively stimulating students' interest in learning and fostering emotional connection between teachers and students. The structure of a humorous sentence is typically divided into two parts: the "setup" and the "punch." Different words have different effects on the expression of humorous meanings. Humorous meanings are often expressed through polysemy, semantic shifts, and ambiguous puns. This is the "anchor point" theory of humorous sentences, which aims to achieve humorous effects through key "anchor" words.

[0003] Automatic recognition of humorous sentences is about giving computers the ability to process and understand the semantics of humorous texts, and to accurately identify humorous texts. Because humor often relies on diverse contexts, it sometimes requires external knowledge to fully understand it. Early humor recognition classifiers were mostly based on feature engineering and statistical machine learning methods, with manual feature design being a widely adopted strategy. For example, in 2009, Mihalcea and Strapparova defined three humor features: alliteration, antonyms, and adult slang; in 2015, Yang et al. constructed phonetic features, ambiguous features, inconsistency features, and sentiment features, and applied these features to humor recognition on social media. In 2021, Fan Xiaochao, building on the integration of the above features, further introduced syntactic features and fuzzy characteristics, and designed multiple sub-features for different categories, achieving significant results using the SVM model. In addition to utilizing the multi-dimensional intrinsic features of humorous sentences, Lin Hongfei also calculated semantic distances through word vectors, calculated path distances using WordNet retrieval, and performed tendency analysis with the help of sentiment dictionaries, integrating internal and external features to achieve humor recognition. Tunico Tursun et al. used dynamic common sense and multidimensional semantic features to infer the speaker's implicit emotional expression to achieve humor recognition.

[0004] Most of the above works use statistical or matching methods to extract shallow humor features in text, but are unable to represent the deep latent semantics of humor, thus limiting the performance of humor recognition.

[0005] With the rapid increase in computing resources and power, the application of deep learning models in humor recognition has gradually emerged. Donahue et al. extracted latent semantic and speech features from data from different dimensions and used a variety of deep learning fusion methods to achieve optimal performance in the SemEval 2016 humor evaluation. Weller and Seppi used the Transformer architecture to identify humor. Hasan et al. used recurrent neural networks for multimodal humor recognition. Fan et al. used a Bi-GRU network to fuse speech features and ambiguous features for humor detection. Zhang Tongyue et al. used a deep learning model based on convolutional neural networks and combined it with label transfer relationships to propose a multi-task learning model for humor recognition.

[0006] Humor recognition methods based on deep neural networks typically use pre-trained language models (such as BERT) to represent text, then construct neural networks with different structural characteristics to extract deep humor features. For example, Mirai and Aono used BERT to generate text tokens and sentence embeddings, which were sent as input to a neural network to detect humor in short texts. Annamoradnejad improved the BERT model and verified its effectiveness on a self-created humor dataset, ColBERT. However, these models still have the following problems when processing humorous language in teaching scenarios:

[0007] 1. Inadequate processing of long-distance semantic dependencies: Traditional recurrent neural networks (such as RNNs) are prone to vanishing or exploding gradients when processing long sequences of text, making it difficult to capture long-distance semantic associations in the "foreshadowing-punch" structure.

[0008] 2. Lack of domain-specific adaptation: The general humor detection model is not optimized for classroom scenarios and ignores the characteristics of the combination of subject terminology and humor in the teaching language.

[0009] 3. Data scarcity: Public humor datasets (such as CCL2018 and CCL2019) lack targeted classroom teaching scenario data, resulting in insufficient generalization ability of the model in teaching humor detection.

[0010] Existing technologies, such as the TextCNN model, can capture local relevance within text but cannot handle long-range dependencies. While the Transformer model addresses long-range dependencies, it suffers from high computational complexity and lacks optimization for the "anchor" characteristics of humorous content. Therefore, a specialized model that integrates transfer learning, improved attention mechanisms, and sequence modeling is urgently needed to address the key technical challenges of humorous content detection. Summary of the Invention

[0011] The technical problem to be solved by the present invention is to provide a teaching humor language detection model (T-Humor-CCTB, ChineseClip+TCCA+BiGRU) based on transfer learning, which can accurately capture the "anchor" words and their contextual semantic relationships in classroom humor language, strengthen general semantic understanding through transfer learning, combine with the improved cross-attention mechanism to focus on local key features, and use bidirectional recurrent networks to model long-distance dependencies, thereby improving the detection accuracy and robustness of humorous language in teaching scenarios.

[0012] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0013] A transfer learning-based teaching humor language detection model (T-Humor-CCTB) includes three core modules:

[0014] Transfer learning module: The text encoder of the ChineseClip model is used to extract word-level context features, and the fine-grained loss value and the loss value of the original label are weighted summed;

[0015] TCCA Fusion Network: The Criss-Cross-Attention (CCA) mechanism used for images is improved and migrated to text processing tasks. The (Text-Criss-Cross-Attention) TCCA fusion network TextCCNet is established. It performs convolution, matrix operations, attention calculation, and parameter aggregation on word vector features to enhance the weight allocation of words with high semantic relevance.

[0016] Long text semantic understanding module: Uses BiGRU units to perform bidirectional modeling on the feature sequence output by TCCA, controls information flow through update gates and reset gates, captures long-distance semantic relationships and semantic transitions, and outputs a binary classification result of humor or non-humor through the classification layer.

[0017] As a further description of the above technical features, in the transfer learning module, the hidden state of the last hidden layer of the ChineseClip model is a three-dimensional tensor with the shape:

[0018] [batch_size,sequrnce_length,hidden_size];

[0019] Where batch_size is the batch size of the input sequence, denoted by C; sequence_length is the length of the input sequence, denoted by H; hidden_size is the hidden layer size of the BERT model, denoted by W.

[0020] As a further description of the above technical features, in the transfer learning module, the weighted summation formula is:

[0021]

[0022] Among them, total_loss represents the total loss value after weighting, f BCE is the binary cross entropy loss function, y pred and f true denote the predicted label and the true label respectively, is the fine-grained label of the “anchor” word position, w i is a dynamic weight parameter.

[0023] As a further description of the above technical features, the processing process of the TCCA fusion network TextCCNet is as follows:

[0024] The input word vector features are passed through the deep convolutional neural network Conv1d, which compresses the hidden layer of the output word vector to 1 / 8 of the input word vector through one-dimensional dilated convolution, generating 7 compressed word vectors: query matrices q_H, q_W, key matrices k_H, k_W, and value matrices v_H, v_W, v;

[0025] The compressed word vectors q_H, k_H and q_W, k_W are collected in the horizontal and vertical directions through matrix multiplication to calculate the horizontal attention matrix e_H and the vertical attention matrix e_W. The calculation equation is:

[0026] q_H,q_W,k_H,k_W,v_H,v_W,v=convld(v1,v2,v3,v4,v5)

[0027] e_H=q_H×k_H

[0028] e_W=q_W×k_W

[0029] Among them, {q_H,k_H,v_H,v}∈R C,H,W / 8 ,{q_W,k_H,v_W}∈R C,W / 8,H , e_H∈R C,H,H , e_W∈R C ,W / 8,W / 8 ;

[0030] Eliminate the diagonal elements of e_H and e_W, flatten and concatenate them, and then normalize them to obtain the attention scores att_H and att_W. The formula for the normalization calculation is:

[0031] att_H,att_W=re(sm(flat(e_H-diag(e_H)),flat(e_W-diag(e_W))))

[0032] Among them, att_H∈RC,W / 8,W / 8 , att_W∈R C,H,H , re means reshape operation, sm means normalization operation, flat means flattening operation, and diag means setting the diagonal to 0;

[0033] The weighted aggregation value matrix and the attention score are combined to generate the fusion feature H. The equation for weighted aggregation is:

[0034] out_H=v_H×att_H

[0035] out_W=v_W×att_W

[0036] H=Addptpool(β(att_H+att_W)+v)

[0037] Where H∈R C,W / 8 , β is the adaptive attention weight, and Adaptpool is the adaptive pooling layer.

[0038] As a further description of the above technical features, the model also includes a classroom humor dataset T-Humor, and the dataset T-Humor contains annotated humorous and non-humorous sentences.

[0039] As a further description of the above technical features, the dataset T-Humor obtains raw data through web crawlers, generative models or speech recognition, and the raw data is annotated and preprocessed by professional reviewers. After being unified and organized through comprehensive analysis and majority voting, it is divided into training set, validation set and test set in a ratio of 8:1:1.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. Utilize the text-image alignment features of the ChineseCLIP pre-trained model to effectively extract the contextual semantics of the "anchor" words, reducing training parameters by 50% compared to the traditional BERT model.

[0042] 2. The TCCA module improves sensitivity to features such as polysemy and puns through horizontal / vertical cross-attention and self-attention removal mechanisms, enhances attention to key semantics in the text (such as "anchor" words), improves the model's ability to capture semantic associations and transitions, and reduces irrelevant noise interference compared to traditional self-attention mechanisms (F1 value increased by 1.25% in ablation experiments).

[0043] 3. BiGRU's bidirectional gating mechanism effectively addresses the vanishing gradient problem and, combined with bidirectional information flow, accurately identifies semantic transitions, achieving a recall rate of 99.25% on the T-Humor dataset.

[0044] 4. The T-Humor dataset, built for classroom scenarios, enables the model to adapt to the combination of teaching terminology and humor. It achieved an F1 score of 0.7188 on the CCL2018 dataset and 0.8514 on the CCL2019 dataset, outperforming most participating models and demonstrating both generalization and professionalism. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is the overall architecture of the T-Humor-CCTB model.

[0046] Figure 2 This is a comparison of the loss curves of all models in the ablation experiment.

[0047] Figure 3 is the confusion matrix of the T-Humor-CCTB model on the T-Humor dataset. DETAILED DESCRIPTION

[0048] The specific technical contents of the present invention are further described in detail below through examples, but this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples. All technologies implemented based on the above contents of the present invention belong to the scope of the present invention.

[0049] Example 1

[0050] A teaching humor language detection model based on transfer learning (T-Humor-CCTB) has the following overall architecture: Figure 1 As shown, it includes three core modules:

[0051] Transfer learning module: The text encoder of the ChineseClip model is used to extract word-level context features, and the fine-grained loss value and the loss value of the original label are weighted summed;

[0052] TCCA Fusion Network: The Criss-Cross-Attention (CCA) mechanism used for images is improved and migrated to text processing tasks. The (Text-Criss-Cross-Attention) TCCA fusion network TextCCNet is established. It performs convolution, matrix operations, attention calculation, and parameter aggregation on word vector features to enhance the weight allocation of words with high semantic relevance.

[0053] Long text semantic understanding module: Uses BiGRU units to perform bidirectional modeling on the feature sequence output by TCCA, controls information flow through update gates and reset gates, captures long-distance semantic relationships and semantic transitions, and outputs a binary classification result of humor or non-humor through the classification layer.

[0054] The transfer learning module adopts the ChineseCLIP transfer learning model, uses its text encoder to extract the common features of the training data, and performs a weighted summation of the fine-grained loss value and the loss value of the original label. The calculation formula of the weighted summation is:

[0055]

[0056] Among them, total_loss represents the total loss value after weighting, f BCE is the binary cross entropy loss function, y represents the true label and the predicted label, w i is a dynamic weight parameter.

[0057] The hidden state of the last hidden layer of the ChineseCLIP transfer learning model is a three-dimensional tensor, storing the hidden state corresponding to each word in the input sequence. Its shape is [batch_size, sequence_length, hidden_size]. After pre-training on approximately 200 million text images, the ChineseCLIP model has rich common feature information parameters for text and images. Where batch_size is the batch size of the input sequence, denoted by C; sequence_length is the length of the input sequence, denoted by H; hidden_size is the hidden layer size of the BERT model (length 768), denoted by W.

[0058] Compared to filtering and learning redundant information from the entire text features, the ChineseCLIP model strengthens its focus on "anchor" words through word feature training. The specific algorithm equation for this process is as follows:

[0059] (v1,v2,v3,v4,v5)=ChineseCLIP(token(You are humorous!))

[0060] Here, (v1,v2,v3,v4,v5)∈R C,H,W , which represents the word features of the text after passing through the ChineseCLIP encoder.

[0061] For serialized text information, based on scanning global information, inputs at different time nodes are weighted according to the "anchor point" theory, further enhancing the weight distribution of words with high semantic relevance. At the same time, focusing on important information also means forgetting information with low semantic relevance, thereby prolonging the model's memory of text sequences and improving model performance in both time and space. To address this issue, the Criss-Cross-Attention (CCA) mechanism used for images was improved and migrated to text task processing, forming the Text-Criss-Cross-Attention (TCCA) fusion network TextCCNet. Due to the special convolutional network in TCCA, the model can specifically utilize the word vector features of the text for contextual semantic understanding, strengthening the focus on "anchor" words in the teacher's humorous language text.

[0062] The special thing about TCCA is that it works on word vectors instead of sentence vectors. Figure 1 As shown in the figure, the input word vector features are passed through a deep convolutional neural network (Conv1d). One-dimensional dilated convolution is used in the convolution layer to compress the hidden layer of the output word vector to 1 / 8 of the input word vector, obtaining a total of 7 compressed word vectors: query matrix q_H, q_W, key matrix k_H, k_W and value matrix v_H, v_W, v; q_H, k_H and q_W, k_W are multiplied horizontally and vertically to collect context information, and the horizontal attention matrix e_H and the vertical attention matrix e_W are calculated to enhance the word representation ability. The specific calculation equation is:

[0063] q_H,q_W,k_H,k_W,v_H,v_W,v=convld(v1,v2,v3,v4,v5)

[0064] e_H=q_H×k_H

[0065] e_W=q_W×k_W

[0066] Among them, {q_H,k_H,v_H,v}∈R C,H,W / 8 ,{q_W,k_H,v_W}∈R C,W / 8,H , e_H∈R C,H,H , e_W∈R C ,W / 8,W / 8 .

[0067] After obtaining the above data, extract the off-diagonal elements of e_H and e_W respectively and flatten them into two-dimensional elements to avoid the self-attention effect and enhance the contrast of local features. Then, concatenate the two two-dimensional elements to obtain D, and apply normalization to D to calculate the attention scores att_H, att_W. The calculation formula is:

[0068] att_H,att_W=re(sm(flat(e_H-diag(e_H)),flat(e_W-diag(e_W))))

[0069] Among them, att_H∈R C,W / 8,W / 8 , att_W∈R C,H,H , re represents the reshaping operation, sm represents the normalization operation, flat represents the flattening operation, and diag represents setting the diagonal to 0, thus obtaining all the parameters required for the overall word vector H that integrates the attention score.

[0070] Next, the parameters (including attention scores att_H, att_W and value matrices v_H, v_W, and v) are aggregated by weighted addition to generate the fusion feature H, which improves the flexibility and adaptability of the model. The equation for weighted aggregation is:

[0071] out_H=v_H×att_H

[0072] out_W=v_W×att_W

[0073] H=Addptpool(β(att_H+att_W)+v)

[0074] Where H∈R C,W / 8 , β is the adaptive attention weight, and Adaptpool is the adaptive pooling layer.

[0075] When processing long sequences of text, RNNs can experience vanishing and exploding gradients, potentially preventing them from effectively capturing long-range dependencies, which in turn affects the accuracy of semantic understanding. The Gated Recurrent Unit (GRU) is an improved recurrent neural network model that better controls the flow of information by setting update and reset gates. The update gate determines the extent to which the previous state should be incorporated into the current state, while the reset gate determines whether the information from the previous state should be forgotten. This mechanism helps capture long-range dependencies in text and enables a better understanding of the semantics of long texts. Furthermore, the GRU uses cell states to preserve long-term memory and can be combined with short-term memory, allowing the model to not only remember previous information but also effectively process new information, thereby improving its ability to understand the semantics of long texts.

[0076] By simplifying its gate structure and optimizing its transmission strategy, GRU is able to capture long-distance semantic information while also streamlining the number of parameters, significantly reducing the amount of computation and convergence time. Since the unit information in the unidirectional GRU model is transmitted from front to back in chronological order, it can only consider information forward to the current time node and cannot analyze the text from a global perspective. To address this issue, a bidirectional BiGRU is added after TCCA. The flow of information is controlled by setting update gates and reset gates. The update gate determines the extent to which the state of the previous moment is carried over to the current state, while the reset gate determines whether the information of the previous state is forgotten, so as to better capture the association of contextual information. Finally, the classification layer is used to obtain the final classification result.

[0077] To address the lack of data on classroom teacher humor, we constructed a classroom humor dataset, T-Humor. This dataset was obtained using web crawling, generative modeling, and speech recognition. The web crawler data primarily came from Baidu Wenku and Ruiwen.com, while the generative model primarily used databases like Tongyi Qianwen and Wenxin Yiyan. The remaining data was extracted through speech recognition from online video lectures. The raw data was then annotated and preprocessed by five professional reviewers. The data was then consolidated through comprehensive analysis and majority voting. The resulting dataset contains 2,999 entries. Like the public dataset, it is divided into training, validation, and test sets in an 8:1:1 ratio. The composition is shown in Table 1.

[0078] Table 1 Composition structure of the T-humor dataset

[0079] Data structure classification Number of humor tags Number of non-humorous tags Total amount of data training set 1214 1230 2444 Validation set 146 137 283 Test set 135 137 272 Total amount of data 1495 1504 2999

[0080] Example 2 Model Performance Verification

[0081] To evaluate the performance of the T-Humor-CCTB model, we used the publicly available datasets for binary classification of Chinese humor from CCL2018 and CCL2019. The CCL2018 dataset, released at the 17th China Conference on Computational Linguistics, contains 8,243 pieces of content, including 8,046 training pieces and 197 test pieces. The CCL2019 dataset, released at the 18th China Conference on Computational Linguistics, contains 16,420 pieces of content. Since the full validation and test sets for these datasets are not officially available, we randomly split the dataset into training, validation, and test sets in an 8:1:1 ratio.

[0082] With reference to the evaluation indicators of the official CCL2018 and CCL2019 competition results, the experimental comparison uses F1 score as the primary evaluation indicator, and precision, recall rate and accuracy as three secondary evaluation indicators.

[0083] T-Humor-CCTB and all baselines were trained using PyTorch on an NVIDIA RTX 3090 GPU with 24GB of dedicated memory, using the CUDA-11.8 and cuDNN-8.9.5 packages. The training hyperparameters included the text encoder version: chinese-clip-vit-base-patch16, the optimizer: AdamW, and the learning rate: 1×10 -4 .

[0084] To verify the performance of the T-HumorCCTB model in detecting humorous language used by classroom teachers, we conducted ablation experiments and generalization verification. The ablation experiments were conducted on the T-humor dataset, comparing models including ChineseClip+BiLSTM, ChineseClip+BiGRU, and ChineseClip+SA+BiGRU, where SA represents the self-attention mechanism. The generalization experiments were conducted on the CCL2018 and CCL2019 Chinese Humor datasets, and compared with the competing models.

[0085] 1 Ablation Experiment

[0086] Table 2 compares the results of all models in the ablation experiment. It shows that T-Humor-CCTB achieved a score of 0.978 on the primary evaluation metric, F1, making it the best performer compared to the other three models. It also achieved excellent results on the other three secondary evaluation metrics, with recall scores of 0.9925, precision scores of 0.9640, and accuracy scores of 0.9779. This demonstrates the effectiveness of the T-Humor-CCTB model in detecting humorous language used by classroom teachers.

[0087] Table 2 Comparison of ablation experiment results

[0088] Model F1 Recall Accuracy Accuracy ChineseClip+BiLSTM 0.9576 0.9704 0.9428 0.9569 ChineseClip+BiGRU 0.9655 0.9618 0.9692 0.9669 ChineseClip+SA+BiGRU 0.9498 0.9448 0.9549 0.9522 ChineseClip+TCCA+BiGRU(T-Humor-CCTB) 0.9780 0.9925 0.9640 0.9779

[0089] Figure 2The following is a comparison of the loss curves for all models in the ablation experiment. As can be seen in the figure, the loss value represented by the green curve (ChineseClip+SA+BiGRU model) is significantly higher than that of the other models. This may be because the self-attention mechanism focuses on all elements of the entire input sequence or feature map, introducing a lot of irrelevant or noisy information, which in turn worsens the performance. The red curve (ChineseClip+BiLSTM model) and the yellow curve (ChineseClip+BiGRU model) converge significantly better than the green curve due to their long-range semantic understanding achieved through the bidirectional recurrent network. However, these two models still cannot escape the influence of weak noise from irrelevant features and fail to capture important local feature information. The T-Humor-CCTB model utilizes the TCCA module to convolve word features and eliminate diagonals, effectively capturing local "anchor" words in humorous language, thereby improving the recognition performance of humorous speech.

[0090] Figure 3 is the confusion matrix of the T-Humor-CCTB model on the T-Humor dataset, Figure 3 It can be seen that the T-Humor-CCTB model successfully detected 266 out of 272 test samples, showing a high level of effectiveness in detecting true and false samples.

[0091] 2 Generalization Experiment

[0092] To verify the T-Humor-CCTB model's versatility in understanding humorous language, it was tested on the 2018CCL and 2019CCL Chinese Humor datasets. Table 3 shows a comparison of the T-Humor-CCTB model's experimental results with the top nine teams in the 2018CCL competition. The T-Humor-CCTB model achieved the best F1 score of 0.7188. For the 2019CCL dataset, the T-Humor-CCTB model was compared with the teams ranked in the first phase. The results, shown in Table 4, show that the T-Humor-CCTB model achieved an F1 score of 0.8514. While not surpassing the first-place team, the Kingsoft AI Lab-MRC team, this demonstrates the T-Humor-CCTB model's robust ability to understand humorous language.

[0093] Table 3 Results of the 2018 CCL Chinese humor ratings

[0094] Ranking Team Name F1 1 water 0.6272 2 ECNU 0.6252 3 Aurora Particles 0.6248 4 YNU-HPCC 0.6223 5 ytkj 0.6153 6 ZMU 0.6113 7 27 0.6088 8 TRSunnyJY 0.6088 9 hqu 0.5850 T-Humor-CCTB 0.7188

[0095] Table 4 2019CCL Chinese humor classification results

[0096] Ranking Team Name F1 1 Kingsoft AILab-MRC 0.8531 2 SXU-NLP 0.8406 3 Qingbo AI 0.8439 4 TenaciousBirds 0.8463 T-Humor-CCTB 0.8514

[0097] The above embodiments describe the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the scope of the principles of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of protection of the present invention.

Claims

1. A teaching humor language detection model based on transfer learning, characterized by: include: Transfer learning module: The text encoder of the ChineseClip model is used to extract word-level context features, and the fine-grained loss value and the loss value of the original label are weighted summed; TCCA fusion network: The CCA mechanism used in image processing is improved and migrated to text processing tasks. A TCCA fusion network is constructed to calculate attention in the horizontal and vertical directions, increasing the weight of words with high semantic relevance. Long text semantic understanding module: Uses BiGRU units to perform bidirectional modeling on the feature sequence output by TCCA, controls information flow through update gates and reset gates, captures long-range semantic relationships and semantic transitions, and outputs humorous / non-humorous binary classification results through the classification layer.

2. The teaching humor language detection model according to claim 1 is characterized in that: In the transfer learning module, the hidden state of the last hidden layer of the ChineseClip model is a three-dimensional tensor with the shape: [batch_size,sequrnce_length,hidden_size]; Where batch_size is the batch size of the input sequence, denoted by C; sequence_length is the length of the input sequence, denoted by H; hidden_size is the hidden layer size of the BERT model, denoted by W.

3. The teaching humor language detection model according to claim 1 is characterized in that: In the transfer learning module, the weighted summation formula is: Among them, total_loss represents the total loss value after weighting, f BCE is the binary cross entropy loss function, y pred and f true denote the predicted label and the true label respectively, is the fine-grained label of the "anchor" word position, w i is a dynamic weight parameter.

4. The teaching humor language detection model according to claim 1 is characterized in that: The processing process of the TCCA fusion network is as follows: Compress the hidden layer of the output word vector to 1 / 8 of the input word vector through one-dimensional dilated convolution, generating query matrices q_H, q_W, key matrices k_H, k_W, and value matrices v_H, v_W, v; Calculate the horizontal attention matrix e_H = q_H × k_H and the vertical attention matrix e_W = q_W × k_W through matrix multiplication; Eliminate the diagonal elements of e_H and e_W, flatten, concatenate and normalize them to obtain the attention scores att_H and att_W; The weighted aggregation value matrix and the attention score are combined to generate the fusion feature H.

5. The teaching humor language detection model according to claim 4 is characterized in that: The formula for the normalization calculation is: att_H,att_W=re(sm(flat(e_H-diag(e_H)),flat(e_W-diag(e_W)))) Among them, re represents the reshape operation, sm represents the normalization operation, flat represents the flattening operation, and diag represents setting the diagonal to 0.

6. The teaching humor language detection model according to claim 4 is characterized in that: The equation for the weighted aggregation is: H=Addptpool(β(att_H+att_W)+v) Among them, β is the adaptive attention weight and Adaptpool is the adaptive pooling layer.

7. The teaching humor language detection model according to claim 1 is characterized in that: Also included is a classroom humor dataset T-Humor, which contains labeled humorous and non-humorous sentences.

8. The teaching humor language detection model according to claim 7 is characterized in that: The dataset T-Humor obtains raw data through web crawlers, generative models, and speech recognition, annotates and preprocesses it by professional reviewers, and then organizes it uniformly through comprehensive analysis and majority voting, and divides it into training set, validation set, and test set in a ratio of 8:1:1.

Images