Construction method of forum text cognitive state perception model

The forum text cognitive state perception model addresses the challenge of capturing temporal semantic features in forum discussions by using BERT and BiLSTM layers to classify cognitive levels, improving the assessment of learners' cognitive states.

CN119066508APending Publication Date: 2024-12-03HUAZHONG NORMAL UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410933940.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Traditional text analysis methods are inadequate for assessing learners' cognitive states in forum discussions, failing to capture the evolution of semantic structures and deep cognitive connotations over time, leading to poor performance in cognitive state perception tasks.

Method used

A method for constructing a forum text cognitive state perception model comprising data acquisition, preprocessing, semantic extraction, temporal feature fusion, and classification, utilizing BERT and BiLSTM layers to integrate temporal semantic features and classify cognitive levels based on Bloom's Taxonomy.

Benefits of technology

The model effectively classifies cognitive levels in forum discussions, outperforming baseline models by integrating temporal semantic features, enhancing the understanding of complex cognitive states and improving learning strategies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066508A_ABST
    Figure CN119066508A_ABST
Patent Text Reader

Abstract

The invention discloses a method for constructing a forum text cognitive state perception model. The forum text cognitive state perception model comprises a data acquisition module, a preprocessing module, a semantic extraction module, a time sequence feature fusion module and a classification module. Wherein the data acquisition module is used for collecting forum text data; the preprocessing module is used for classifying forum texts and converting the texts into a vector form which can be accepted by a machine; the semantic extraction module is used for obtaining context semantic features in the independent text; the time sequence feature fusion module is used for fully mining and fusing time sequence cognitive features between independent discussion texts to obtain global features of the discussion texts; and the classification module is used for classifying the cognitive level of the text representation. Experiments show that the model provided by the invention is superior to other baseline models in a cognitive state perception task based on forum discussion data, and meanwhile, ablation experiment verification shows that the time sequence semantic features between statements before and after fusion of the discussion text have positive influence on the classification effect of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of model construction, and in particular to a method for constructing a forum text cognitive state perception model. Background Art

[0002] Forum discussions in a collaborative learning environment are the externalized expression of learners' thinking processes and knowledge and skills, and are an important way to identify learners' cognitive levels and autonomous inquiry capabilities. Using forum texts to assess learners' cognitive levels not only helps learners to promptly identify problems in the online learning process and improve their learning strategies, but also helps teachers provide personalized learning guidance, improve teaching quality and learning outcomes. Traditional text analysis methods are more suitable for text topic classification or sentiment recognition in general fields, and ignore the evolution of students' cognitive states over time when they participate in discussions. Therefore, it is difficult to fully explore and understand the complex semantic structures and deep cognitive connotations in the discussion texts, resulting in poor performance in cognitive state perception tasks. Based on this, it is crucial for the present invention to propose a forum text cognitive state perception model that integrates the temporal semantic features between sentences before and after the text. Summary of the invention

[0003] In view of this, the present invention provides a method for constructing a forum text cognitive state perception model.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A method for constructing a forum text cognitive state perception model, characterized by comprising a data acquisition module, a preprocessing module, a semantic extraction module, a temporal feature fusion module and a classification module; wherein,

[0006] The data collection module is used to collect forum text data;

[0007] The preprocessing module is used to classify forum texts and convert them into vector forms acceptable to machines;

[0008] The semantic extraction module is used to obtain the contextual semantic features within the discussion text;

[0009] The temporal feature fusion module is used to fully mine and fuse the temporal cognitive features, i.e., global features, between independent discussion texts;

[0010] The classification module is used to classify the cognitive level of text representation.

[0011] Preferably, the data collection method is:

[0012] First, the forum texts were labeled and classified. The collected forum text data were manually labeled into two categories: 0 and 1 according to the revised version of Bloom's Taxonomy of Educational Objectives proposed by Krathwohl; 0 represents the three low-order cognitive levels of memory, comprehension, and application, and 1 represents the three high-order cognitive levels of analysis, evaluation, and creation.

[0013] Preferably, the pretreatment method is:

[0014] First, the length of all text segments is normalized, zero padding is performed on texts whose length is less than the specified maximum length, and truncation is performed on texts whose length is greater than the specified maximum length, so that the length of each text is unified to N. Secondly, special tags [CLS] and [SEP] are added at the beginning and end of the sequence respectively, and then the cleaned text is divided into tokens and the encoded token embeddings corresponding to each token. At the same time, the corresponding attention mask is generated to distinguish the original token from the padded token. The input_ids and attention_mask tensors together constitute the word embedding vector, where N represents the maximum length of the text.

[0015] Preferably, the semantic extraction module consists of a BERT layer and a fully connected layer, wherein BERT is a deep learning model based on the Transformer architecture, which learns rich language representations by pre-training on large-scale unlabeled text data.

[0016] Preferably, the semantic extraction method is:

[0017] Select BERT to extract the local semantic features of the text, and use the word embedding vector X obtained in the preprocessing stage as input, that is:

[0018] E=BERT(X)#(1)

[0019] Thus, we can obtain the local semantic feature vector E=(e0,e1,…,e N-1 ), N represents the maximum sequence length; the vector is then input into a fully connected layer to capture the key semantic features in the text while retaining the original features of the text as much as possible, that is:

[0020] E′=EW+b#(2)

[0021] in, d1,d2 represent the hidden layer size of BERT respectively;

[0022] Thus, the compressed local semantic feature vector E′=(e0,e1,…,e N-1), N represents the maximum sequence length.

[0023] Preferably, the temporal feature fusion module consists of an attention layer, a BILSTM layer and a dropout layer.

[0024] Preferably, the time series feature fusion method is:

[0025] For each local semantic feature vector E′ obtained by the semantic extraction module, the first n and last n text vectors of the current text are obtained to form a context window, and the 2n+1 text vector sequence S=(S0,…,S n ,...S 2n ) is used as the input of the attention layer, in which the interdependencies between the text sequences are calculated and the feature representation with attention is output; the calculation formula of the attention score can be expressed as:

[0026] Q=SW Q ,K=SW K ,V=SW V #(3)

[0027]

[0028] Among them, the weight matrix d2 is the hidden layer size of BILSTM, d k is the dimension of the key matrix K, and the output is the weighted sum of the value (V) matrix, with the weight determined by the attention score; the attention layer obtains an enhanced vector representation A = (A0,…,A n ,...,A 2n ), where A i =(a1,a2,…,a N ) indicates that the output of each text segment contains not only its own information, but also relevant information of other segments. The text segments that contribute more to the cognitive level assessment are given higher weights and used as the input of the BILSTM layer.

[0029] Preferably, the time series feature fusion method also includes a BILSTM model that fully mines the long-term features of the sequence data through forward and backward LSTM, A = (A0, ..., A n ,...,A 2n ) is the input part of the model, which represents 2n+1 independent text vector representations containing other text sequence features. The output calculation method at time t is as follows:

[0030]

[0031] in, and They represent the forward and backward output results of LSTM at time t, respectively. t represents the output result of BILSTM at time t, w t and v t Denote the forward and backward weight matrices of the forward output, respectively, and b t represents the bias at time t;

[0032] Thus, the fusion temporal cognitive features of the text sequence are obtained Represents the vector representation after integrating temporal cognitive features;

[0033] The feature vector h of the last time step 2n It reflects the cognitive characteristics of the entire sequence and integrates the context information of the forward and backward LSTMs; therefore, the output h of the last time step of the sequence is taken 2n Sending it to the Dropout layer randomly discards some features, which helps reduce the overfitting of the model on the training data; the calculation formula is as follows:

[0034]

[0035] in, Each element in m is an independent and identically distributed Bernoulli random variable, and p represents h 2n The probability of each element being retained in ; thus the global feature vector of time series feature fusion is obtained

[0036] Preferably, the classification module includes a fully connected layer.

[0037] Preferably, the classification method is:

[0038] The global feature vector after dropout processing is input into the fully connected layer, which maps the feature vector to the prediction score of the final category, which can be expressed as:

[0039]

[0040] in, b∈R 1×2 ;

[0041] The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true probability distribution, which is used as the basis for optimizing the model parameters. The output of the model is converted into category probability, which is then compared with the probability distribution of the true category label.

[0042] The mathematical formula is as follows:

[0043]

[0044] Among them, y is the probability of the true label, is the probability predicted by the model.

[0045] Compared with the prior art, the present invention has achieved the following technical effects:

[0046] Experiments have shown that the model proposed in the present invention outperforms other baseline models in the cognitive state perception task based on forum discussion data. At the same time, ablation experiments have verified that the proposed model has a positive impact on the classification effect of the model by integrating the temporal semantic features between sentences before and after the discussion text. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A model architecture diagram of a method for constructing a forum text cognitive state perception model according to the present invention; DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] A method for constructing a forum text cognitive state perception model includes a data acquisition module, a preprocessing module, a semantic extraction module, a temporal feature fusion module and a classification module; wherein,

[0050] The data collection module is used to collect forum text data;

[0051] The preprocessing module is used to classify forum texts and convert them into vector forms acceptable to machines;

[0052] The semantic extraction module is used to obtain the contextual semantic features within the discussion text;

[0053] The temporal feature fusion module is used to fully mine and fuse the temporal cognitive features, i.e., global features, between independent discussion texts;

[0054] The classification module is used to classify the cognitive level of text representation.

[0055] The data collection method is:

[0056] First, the forum texts were annotated and classified. The collected forum text data were manually annotated into two categories: 0 and 1 according to the revised version of Bloom's Taxonomy of Educational Objectives proposed by Krathwohl; 0 represents the three low-level cognitive levels of memory, understanding, and application, and 1 represents the three high-level cognitive levels of analysis, evaluation, and creation. The specific annotations and data examples are shown in Table 1:

[0057] Table 1: Specific annotations and data examples

[0058]

[0059]

[0060]

[0061] The preprocessing method is:

[0062] In order to classify forum texts, it is necessary to convert the texts into a vector form acceptable to the machine. First, the lengths of all text segments are normalized, zero padding is performed on texts whose lengths are less than the specified maximum length, and truncation is performed on texts whose lengths are greater than the specified maximum length, so that the length of each text is unified to N. Secondly, special tags [CLS] and [SEP] are added at the beginning and end of the sequence respectively, and then the cleaned text is divided into tokens and the encoded token embeddings corresponding to each token. At the same time, the corresponding attention mask is generated to distinguish the original tokens from the filled tokens. The input_ids and attention_mask tensors together constitute the word embedding vector, where N represents the maximum length of the text.

[0063] The semantic extraction module consists of a BERT layer and a fully connected layer. BERT is a deep learning model based on the Transformer architecture. It learns rich language representations through pre-training on large-scale unlabeled text data. Studies have shown that BERT performs well in many NLP tasks such as text classification and semantic similarity assessment.

[0064] The method of semantic extraction is:

[0065] Select BERT to extract the local semantic features of the text, and use the word embedding vector X obtained in the preprocessing stage as input, that is:

[0066] E=BERT(X)#(1)

[0067] Thus, we can obtain the local semantic feature vector E=(e0,e1,…,e N-1 ), N represents the maximum sequence length; the vector is then input into a fully connected layer to capture the key semantic features in the text while retaining the original features of the text as much as possible, that is:

[0068] E′=EW+b#(2)

[0069] in, d1,d2 represent the hidden layer size of BERT respectively;

[0070] Thus, the compressed local semantic feature vector E′=(e0,e1,…,e N-1 ), N represents the maximum sequence length.

[0071] The temporal feature fusion module consists of an attention layer, a BILSTM layer, and a dropout layer.

[0072] Starting from a global perspective, the present invention considers the temporal relationship between independent discussion texts and the implicit cognitive connections therein, and designs a temporal feature fusion module based on the attention mechanism to fully mine and fuse the temporal cognitive features between independent discussion texts.

[0073] The time series feature fusion method is:

[0074] For each local semantic feature vector E′ obtained by the semantic extraction module, the first n and last n text vectors of the current text are obtained to form a context window, and the 2n+1 text vector sequence S=(S0,…,S n ,...S 2n ) is used as the input of the attention layer, in which the interdependencies between the text sequences are calculated and the feature representation with attention is output; the calculation formula of the attention score can be expressed as:

[0075] Q=SW Q ,K=SW K ,V=SW V #(3)

[0076]

[0077] Among them, the weight matrix d2 is the hidden layer size of BILSTM, d k is the dimension of the key matrix K, and the output is the weighted sum of the value (V) matrix, with the weight determined by the attention score; the attention layer obtains an enhanced vector representation A = (A0,…,A n ,...,A 2n ), where A i =(a1,a2,…,a N ) indicates that the output of each text segment contains not only its own information, but also relevant information of other segments. The text segments that contribute more to the cognitive level assessment are given higher weights and used as the input of the BILSTM layer.

[0078] The time series feature fusion method also includes the BILSTM model, which fully mines the long-term features of sequence data through forward and backward LSTM.n ,...,A 2n ) is the input part of the model, which represents 2n+1 independent text vector representations containing other text sequence features. The output calculation method at time t is as follows:

[0079]

[0080] in, and They represent the forward and backward output results of LSTM at time t, respectively. t represents the output result of BILSTM at time t, w t and v t Denote the forward and backward weight matrices of the forward output, respectively, and b t represents the bias at time t;

[0081] Thus, the fusion temporal cognitive features of the text sequence are obtained Represents the vector representation after integrating temporal cognitive features;

[0082] The feature vector h of the last time step 2n It reflects the cognitive characteristics of the entire sequence and integrates the context information of the forward and backward LSTMs; therefore, the output h of the last time step of the sequence is taken 2n Sending it to the Dropout layer randomly discards some features, which helps reduce the overfitting of the model on the training data; the calculation formula is as follows:

[0083]

[0084] in, Each element in m is an independent and identically distributed Bernoulli random variable, and p represents h 2n The probability of each element being retained in ; thus the global feature vector of time series feature fusion is obtained

[0085] The classification module consists of a fully connected layer.

[0086] The classification method is:

[0087] The global feature vector after dropout processing is input into the fully connected layer, which maps the feature vector to the prediction score of the final category, which can be expressed as:

[0088]

[0089] in, b∈R 1×2 ;

[0090] The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true probability distribution, which is used as the basis for optimizing the model parameters. The output of the model is converted into category probability, which is then compared with the probability distribution of the true category label.

[0091] The mathematical formula is as follows:

[0092]

[0093] Among them, y is the probability of the true label, is the probability predicted by the model.

[0094] The construction principle of a forum text cognitive state perception model is:

[0095] First, extract the local semantic features of the text based on the BERT pre-training model;

[0096] Then, the cognitive attention coefficients of the previous and next sentences are obtained through the attention mechanism, and the global feature vector is obtained by combining BiLSTM and the attention coefficient;

[0097] Finally, a classifier is used to classify the cognitive level of text representation.

[0098] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for constructing a forum text cognitive state perception model, characterized in that: It includes data acquisition module, preprocessing module, semantic extraction module, temporal feature fusion module and classification module; among them, The data collection module is used to collect forum text data; The preprocessing module is used to classify forum texts and convert them into vector forms acceptable to machines; The semantic extraction module is used to obtain the contextual semantic features within the discussion text; The temporal feature fusion module is used to fully mine and fuse the temporal cognitive features, i.e., global features, between independent discussion texts; The classification module is used to classify the cognitive level of text representation.

2. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The method of data collection is: First, the forum texts were labeled and classified. The collected forum text data were manually labeled into two categories: 0 and 1 according to the revised version of Bloom's Taxonomy of Educational Objectives proposed by Krathwohl; 0 represents the three low-order cognitive levels of memory, comprehension, and application, and 1 represents the three high-order cognitive levels of analysis, evaluation, and creation.

3. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The pretreatment method is: First, the length of all text segments is normalized, zeropadding is performed on texts whose length is less than the specified maximum length, and truncation is performed on texts whose length is greater than the specified maximum length, so that the length of each text is unified to N. Secondly, special tags [CLS] and [SEP] are added at the beginning and end of the sequence respectively, and then the cleaned text is divided into tokens and the encoded token embeddings corresponding to each token. At the same time, the corresponding attention mask is generated to distinguish the original token from the padded token. The input_ids and attention_mask tensors together constitute the word embedding vector, where N represents the maximum length of the text.

4. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The semantic extraction module consists of a BERT layer and a fully connected layer, wherein BERT is a deep learning model based on the Transformer architecture, which learns rich language representations by pre-training on large-scale unlabeled text data.

5. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The method of semantic extraction is: Select BERT to extract the local semantic features of the text, and use the word embedding vector X obtained in the preprocessing stage as input, that is: E=BERT(X)#(1) Thus, we can obtain the local semantic feature vector E=(e0,e1,…,e N-1 ), N represents the maximum sequence length; the vector is then input into a fully connected layer to capture the key semantic features in the text while retaining the original features of the text as much as possible, that is: E′=EW+b#(2) Among them, E′∈R d1 ,W∈R d1×d2 , d1,d2 represent the hidden layer size of BERT respectively; Thus, the compressed local semantic feature vector E′=(e0,e1,…,e N-1 ), N represents the maximum sequence length.

6. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The temporal feature fusion module consists of an attention layer, a BILSTM layer and a dropout layer.

7. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The temporal feature fusion method is: For each local semantic feature vector E′ obtained by the semantic extraction module, the first n and last n text vectors of the current text are obtained to form a context window, and the 2n+1 text vector sequence S=(S0,…,S n ,...S 2n ) is used as the input of the attention layer, in which the interdependencies between the text sequences are calculated and the feature representation with attention is output; The score calculation formula can be expressed as: Q=SW Q ,K=SW K ,V=SW V #(3) Among them, the weight matrix d2 is the hidden layer size of BILSTM, d k is the dimension of the key matrix K, and the output is the weighted sum of the value (V) matrix, with the weight determined by the attention score; the attention layer obtains an enhanced vector representation A = (A0,…,A n ,...,A 2n ), where A i =(a1,a2,…,a N ) indicates that the output of each text segment contains not only its own information, but also relevant information of other segments. The text segments that contribute more to the cognitive level assessment are given higher weights and used as the input of the BILSTM layer.

8. The method for constructing a forum text cognitive state perception model according to claim 7, characterized in that: The time series feature fusion method also includes a BILSTM model that fully mines the long-term features of sequence data through forward and backward LSTM, A=(A0,…,A n ,...,A 2n ) is the input part of the model, which represents 2n+1 independent text vector representations containing other text sequence features. The output calculation method at time t is as follows: in, and They represent the forward and backward output results of LSTM at time t, respectively. t represents the output result of BILSTM at time t, w t and v t Denote the forward and backward weight matrices of the forward output, respectively, and b t represents the bias at time t; Thus, the fusion temporal cognitive features of the text sequence are obtained Represents the vector representation after integrating temporal cognitive features; The feature vector h of the last time step 2n It reflects the cognitive characteristics of the entire sequence and integrates the context information of the forward and backward LSTMs; therefore, the output h of the last time step of the sequence is taken 2n Sending it to the Dropout layer randomly discards some features, which helps reduce the overfitting of the model on the training data; the calculation formula is as follows: in, Each element in m is an independent and identically distributed Bernoulli random variable, and p represents h 2n The probability of each element being retained in ; thus the global feature vector of time series feature fusion is obtained 9. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The classification module includes a fully connected layer.

10. The method for constructing a forum text cognitive state perception model according to claim 1, characterized in that: The classification method is: The global feature vector after dropout processing is input into the fully connected layer, which maps the feature vector to the prediction score of the final category, which can be expressed as: in, The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true probability distribution, which is used as the basis for optimizing the model parameters. The output of the model is converted into category probability, which is then compared with the probability distribution of the true category label. The mathematical formula is as follows: Among them, y is the probability of the true label, is the probability predicted by the model.

Citation Information

Patent Citations

  • Forum pair identification method based on BERT and Att-BiLSTM

    CN112347269A

  • Multi-label classification method for processing microblog text cognitive distortion

    CN112347766A

  • BERT-based multi-feature fusion fuzzy text classification model

    CN112732916A

  • Chinese character statement classification method based on semantic information enhancement

    CN115203422A

  • Cognitive service-oriented user intention recognition method and system

    WO2022033073A1