Online education group question and answer matching method based on user style and time perception

Abstract

This invention discloses an online education group question-and-answer matching method based on user style and time awareness, relating to the field of question-and-answer matching using deep learning natural language processing technology. The method involves constructing a BigData dataset; dividing the BigData dataset into training, validation, and test sets; building user style-aware and time-aware question-and-answer matching models; training the models using the training set and obtaining performance metrics using the validation set to find the optimal hyperparameters; and inputting the test set into the final user style-aware and time-aware question-and-answer matching model to obtain the matching results. This invention enhances question extraction by recognizing user style through user style awareness, reducing the impact of noise caused by severe imbalances between the number of questions and other types of dialogue. It also reduces the noise caused by a large number of potential answers to a single question through time awareness. Compared with other traditional question-and-answer matching models, this method improves the model's question-and-answer matching performance and reduces the impact of data noise.

Description

Technical Field

[0001] This invention relates to the field of question-answer matching in deep learning natural language processing technology, and more particularly to a method for matching online education group questions and answers based on user style and time awareness. Background Technology

[0002] Group Question-and-Answer Matching (TGQAM) in online education systems is an innovative educational technology that matches student conversations with appropriate answers on multi-person group chat platforms such as QQ groups and WeChat groups. This intelligent matching system aims to improve teacher productivity and provide personalized learning support for each student.

[0003] In TGQAM, the system learns students' individual learning styles by analyzing their questioning patterns and dialogue content. This allows educators to better understand students' needs and provide more targeted answers and learning resources. This process helps assess teaching quality, providing educators with valuable feedback to improve teaching methods and resources. TGQAM also helps promote collaboration and interaction among students. By matching questions and answers, students can share knowledge and experiences in group chats, forming a more engaged and collaborative learning community. This not only improves the learning atmosphere but also helps build friendly relationships among students and fosters teamwork. Educators can use the TGQAM system to more precisely adjust teaching strategies to meet the needs of different students. By monitoring the matching process, they can better understand students' comprehension of the material and the difficulties they encounter, allowing for targeted adjustments to teaching. This personalized teaching approach helps improve teaching effectiveness, enabling students to participate more actively in the learning process and enhancing the overall learning experience.

[0004] In assessing teaching effectiveness, TGQAM can measure system performance not only by matching accuracy but also by monitoring student engagement and interaction frequency to evaluate student acceptance of online education. This provides educators with more comprehensive data, helping them to evaluate and improve teaching methods more thoroughly and scientifically, thereby better meeting students' learning needs.

[0005] In summary, TGQAM, as a key technology in online education systems, not only improves students' learning efficiency and comprehension but also promotes interaction and collaboration among students. Through personalized matching and real-time teaching feedback, it provides educators with powerful tools to better adapt to students' needs, improve teaching quality, and drive the continuous innovation and development of online education.

[0006] like Figure 1As shown, analyzing the frequency of student questions and answers provides a deeper understanding of students' learning behaviors and participation patterns. Precise matching here can further clarify individual learning preferences and facilitate more targeted teaching methods.

[0007] Students who frequently answer questions may be classified as respondent social learners, who tend to absorb knowledge by actively participating in answering questions. They may be interested in interacting with others, sharing perspectives, and resolving doubts; this learning style helps build learning communities and promote information sharing. Conversely, those who ask questions more frequently or show a preference for limited online interaction can be categorized as problem-oriented social learners or solitary learners, based on their learning styles. Problem-oriented social learners may be more inclined to think proactively and solve problems independently, while solitary learners may prefer to explore knowledge through self-directed learning.

[0008] The frequency with which students ask or answer questions can also indicate their level of active participation in the online learning environment. High-frequency interaction may suggest deeper reflection and proactive engagement with the learning content, while low frequency may indicate a need for more motivation or personalized support. Beyond measuring student engagement, tracking the number of questions and answers and ensuring their accurate matching can reveal the quality of instruction and the educator's teaching ability. Accurate matching of questions and answers not only reflects whether educators are meeting students' learning needs but also assesses the clarity and effectiveness of the teaching materials.

[0009] In Natural Language Processing (NLP), TGQAM-related work primarily focuses on question answering. Traditional question answering methods emphasize identifying the textual relationships between questions and answers, including answer selection methods, word-level and sentence-level matching techniques, and semantic matching research. These techniques help improve matching accuracy, ensuring the system can effectively respond to student questions. Community question answering involves the utilization of external data, with research focusing on multimodal information, user social networks, and user roles. This provides TGQAM systems with a broader range of information sources, enabling a more comprehensive understanding of students' learning environments and needs, thereby better supporting personalized instruction. Research findings in these areas provide a strong technological foundation for TGQAM in online education systems, driving continuous innovation in educational technology.

[0010] While traditional question answering and community question answering provide the foundation for TGQAM, they do not fully address the unique challenges it faces. One major challenge is the lack of publicly available dialogue datasets that encapsulate the unique characteristics of educational community groups and the extreme noise present in the data.

[0011] The TGQAM task differs from traditional question-and-answer matching tasks because it incorporates extreme data noise. In educational community settings, the fusion of various dialogues and distracting information leads to an excessive number of candidate answers for each question, resulting in a significantly lower probability of positive samples (i.e., the second sentence being the answer to the first) compared to negative samples. These issues pose a significant obstacle to effective question-and-answer matching in educational community settings.

[0012] In the field of Natural Language Processing (NLP), existing question-answering models are typically designed to match appropriate questions to a relatively small pool of candidate answers. However, these models often struggle when faced with data noise present in educational communities. The specific requirements of TGQAM necessitate a rethinking and innovation of existing question-answering models to address the large-scale student dialogues and complex interactive contexts involved.

[0013] TGQAM involves managing extended student dialogues, presenting data noise challenges that existing models fail to cover. To fill this gap, our work focuses on designing and building a robust model that better accommodates the unique needs of TGQAM. Summary of the Invention

[0014] To address the shortcomings of existing technologies, this invention provides an online education group question-and-answer matching method based on user style and time awareness. It aims to promote research and development in the TGQAM field by creating a dataset suitable for TGQAM tasks and designing a model specifically designed to solve the inherent and significant data noise problem in TGQAM tasks. By deeply addressing the data noise problem, it provides more intelligent and efficient question-and-answer matching services for online education systems, thereby improving students' learning experience and teaching quality.

[0015] The technical solution of this invention is as follows:

[0016] The online education group question-and-answer matching method based on user style and time awareness includes the following steps:

[0017] Step 1: Construct several BigData datasets; each BigData dataset contains five key attributes: sentence, user ID, tag, match, and timestamp.

[0018] Step 2: Divide each BigData dataset into a training set, a validation set, and a test set according to a set ratio;

[0019] Step 3: Construct user style-aware and time-aware question-answering matching models;

[0020] The user style-aware and time-aware question-answering matching model includes a user style-aware attention module and a time-aware attention module.

[0021] The user style-aware attention module receives N dialogue sequences as input. It uses a word embedding layer to embed the words that make up the dialogue sequence, obtaining the embedding results for each word. Then, a Bi-LSTM layer is used to obtain the latent state information of each word's context in each dialogue. An average pooling layer is then used to aggregate the latent state information of each word in each dialogue, resulting in the encoded representation of each dialogue. A user embedding layer is used to embed the user ID in the dialogue sequence, obtaining the embedding result for the user ID. The confidence score of the dialogue is calculated based on the word embedding results and the user ID embedding results, thus obtaining a conversation representation that considers textual information and user style. Finally, a self-attention mechanism is used to understand the contextual information of the dialogue to obtain the latent vectors of each dialogue, and these latent vectors are input into a linear layer to predict the dialogue type.

[0022] The time-aware attention module is used to concatenate the latent vectors of two dialogues and input them into the dialogue relationship matching layer to obtain a matching degree vector of the two dialogue contents; concatenate the embedding results of the user IDs of the two dialogues and input them into the user relationship matching layer to obtain a matching degree vector of the two speaking users; concatenate the matching degree vector of the dialogue content and the matching degree vector of the speaking users to obtain a matching degree vector that considers text information and user style; then use an exponential decay kernel function to obtain the personalized relationship weight between the two dialogues; and input the matching degree vector considering text information and user style, the personalized relationship weight, and the dialogue type into the linear layer to obtain the probability of whether the two dialogues match.

[0023] Step 4: Train the user style-aware and time-aware question-answering matching models using the training set, and use the validation set to obtain performance metrics and find the optimal hyperparameters to obtain the final user style-aware and time-aware question-answering matching models.

[0024] Step 4.1: Input the N dialogue sequences from the training set into the user style-aware attention module to obtain the predicted dialogue type;

[0025] The N dialogue sequences are represented as [X1, ..., X...]. N ] = [(u1,s1,t1,l1,m1), ... (u N ,s N ,t N ,l N ,m NEach dialogue in the sequence consists of multiple features, where X represents a single dialogue in the dialogue sequence, N represents the number of dialogues, u represents the user ID, s represents the natural language word sequence, t represents the timestamp of the dialogue, l represents the type of dialogue, including Q represents a question, A represents an answer, NQA represents a dialogue that is neither a question nor an answer, and m represents which dialogue in the dialogue sequence this dialogue matches.

[0026] Step 4.1.1: Use the word embedding layer to embed the words that make up the dialogue in the dialogue sequence to obtain the embedding results of each word. Then use the Bi-LSTM layer to obtain the hidden state information of each word in each dialogue. Then use the average pooling layer to aggregate the hidden state information of each word in each dialogue to obtain the encoded representation of each dialogue.

[0027] e w,i,j =Embedding(w i,j )

[0028]

[0029]

[0030] Among them, w i,j It is a natural language word sequence s i The j-th word in It is the word w i,j The word embedding results obtained after embedding through the Embedding layer, where i is the number of the natural language word sequence, j is the word number, and Dw is the dimension of the word embedding result; The hidden state information of each word context in each dialogue obtained through the Bi-LSTM layer. This represents the encoded representation of each dialogue obtained through average pooling; Embedding represents the Embedding layer, Bi-LSTM represents the Bi-LSTM layer, and AvgPool represents the average pooling layer.

[0031] Step 4.1.2: Use the user embedding layer to embed the user ID in the dialogue sequence to obtain the embedding result of the user ID;

[0032] e u,i =Embedding(u i )

[0033] Among them, u i For natural language word sequences s i The corresponding user ID; User ID u iThe embedding result of the user ID obtained after embedding through the Embedding layer; Du is the dimension of the embedding result of the user ID;

[0034] Step 4.1.3: Calculate the confidence of the dialogue based on the word embedding results and the user ID embedding results, and then obtain the conversation representation that takes into account text information and user style;

[0035]

[0036]

[0037] Among them, w t,i Represents a sequence of words in a natural language. i The confidence level, where softmax represents the softmax function. To consider both textual information and user style in the session representation, Ds represents the dimension of the session representation that considers both textual information and user style, and W represents the dimension of the session representation that considers both textual information and user style. o W k and W v These are trainable parameters;

[0038] Step 4.1.4: Use the self-attention mechanism to understand the contextual information of the dialogue to obtain the latent vectors of each dialogue, and input the latent vectors into the linear layer to obtain the predicted dialogue type;

[0039]

[0040]

[0041] in, The latent vectors of the dialogue; W represents the probability that the dialogue type is a question. q and b q These are trainable parameters;

[0042] Step 4.2: Use the time-aware attention module to obtain the matching probability;

[0043] Step 4.2.1: Concatenate the latent vectors of the two dialogues and input them into the dialogue relationship matching layer to obtain the matching degree vector of the two dialogue contents;

[0044] h s,i,i' =ReLu(W s [h s,i ,h s,i' ]+b s )

[0045] in, Let W be the vector representing the degree of matching of the dialogue content, and ReLU be the activation function; s and b sis the trainable parameter; Dh is the dimension of the matching degree vector of the dialogue content;

[0046] Step 4.2.2: Concatenate the embedding results of the user IDs of the two dialogues and input them into the user relationship matching layer to obtain the matching degree vector of the two dialogue speakers;

[0047] h r,i,i' =ReLu(W r [e u,i ,e u,i' ]+b r )

[0048] in, W is the matching degree vector of the speaking users; r and b r These are trainable parameters;

[0049] Step 4.2.3: Concatenate the matching vector of the dialogue content and the matching vector of the speaking user to obtain a matching vector that takes into account text information and user style. Then, use the exponential decay kernel function to obtain the personalized relationship weight between the two dialogues.

[0050]

[0051]

[0052]

[0053] in, To consider the matching degree vector of text information and user style, To calculate the time decay rate of the answer dialogue, Δ i,i' w is the time difference between speaking in a question-and-answer dialogue and in an answer-and-answer dialogue. t,i,i' For personalized relationship weights, W u and b u These are trainable parameters;

[0054] Step 4.2.4: Input the matching vector considering text information and user style, personalized relationship weights and dialogue type into the linear layer to obtain the probability of whether two dialogues match;

[0055]

[0056]

[0057] in, For the purpose of dialogue, W represents the probability of a match between two dialogues. m and b m These are trainable parameters;

[0058] Step 4.3: Repeat the training until the set number of training rounds is reached to obtain the trained user style-aware and time-aware question-answering matching model;

[0059] The loss function used is the cross-entropy loss function, as shown in the following formula:

[0060]

[0061] Where LOSS is the loss function; y m,i,i' The actual label;

[0062] Step 4.4: Use the validation set to obtain performance metrics and then find the optimal hyperparameters to obtain the final user style-aware and time-aware question-answering matching model;

[0063] Step 5: Input the test set into the final user style-aware and time-aware question-answering matching model to obtain the matching results.

[0064] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0065] This invention addresses the data noise problem in question-and-answer matching tasks within online education systems by designing a user style and time-aware method. User style awareness enhances question extraction by identifying user styles, reducing the noise impact of severe imbalances between the number of questions and other types of dialogue. Time awareness reduces the noise impact of a large number of potential answers to a single question. Compared to other traditional question-and-answer matching models, this invention improves the model's question-and-answer matching performance and reduces the impact of data noise. Attached Figure Description

[0066] Figure 1 This is an example model for matching educational group questions and answers in existing online education systems.

[0067] Figure 2 This is a comparison chart of the distribution between the virtual dataset and the BigData22 dataset in a specific embodiment of the present invention;

[0068] Among them, (a) is a comparison chart of the probability distribution of dialogues on different dates; (b) is a comparison chart of the probability distribution of dialogues within a day; (c) is a comparison chart of the probability distribution of the time interval between questions and answers; (d) is a comparison chart of the probability distribution of users asking questions; (e) is a comparison chart of the probability distribution of users providing answers; and (f) is a comparison chart of the probability distribution of users providing unrelated information.

[0069] Figure 3 This is a graph showing the data analysis results in a specific embodiment of the present invention;

[0070] Among them, (a) is a learning style map of question type; (b) is a learning style map of answer type; and (c) is a learning style map of question-answer time interval.

[0071] Figure 4 This is a framework diagram of the User Style and Time-Aware Question-Answer Matching Model (StyleTimeQAM) in a specific embodiment of the present invention;

[0072] Figure 5 This is a diagram illustrating the feature extraction process of StyleTimeQAM in a specific embodiment of the present invention.

[0073] Figure 6 This is an ablation experiment diagram of StyleTimeQAM on the BigData22 dataset according to a specific embodiment of the present invention;

[0074] Figure 7 This is an experimental diagram of noise-free ablation of StyleTimeQAM on the BigData22 dataset in a specific embodiment of the present invention. Detailed Implementation

[0075] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0076] The online education group question-and-answer matching method based on user style and time awareness includes the following steps:

[0077] Step 1: Construct several BigData datasets; each BigData dataset contains five key attributes: sentence, user ID, tag, match, and timestamp.

[0078] In this implementation, to support in-depth research and analysis of the dialogue data, two unique datasets were created, named BigData22 and BigData23 respectively. These two datasets were collected from the "Big Data Technology Principles and Applications" course at a university in China, with time spans from March to May 2022 and from February to April 2023 respectively. The format of the BigData dataset is shown in Table 1 and is stored in CSV format.

[0079] Table 1. Statistical Information of the Dataset

[0080] Dataset BigData22 BigData23 synthesis #Students 61 77 61 #Problems 295 316 2118 #Answers 473 310 1040 #Conversations 1784 1616 22478

[0081] Each attribute in the BigData dataset has a specific and well-defined data type, providing a clear structure for in-depth analysis and understanding of student dialogues. The following is a detailed description of each attribute:

[0082] Sentence (String Type): This attribute stores the text content of the user's conversation, including the questions asked and the answers given. This string type attribute allows tracking students' specific statements to gain deeper insights into their learning needs and engagement levels.

[0083] User ID (integer type): This attribute is an integer that assigns a unique identifier to each user. This ID allows for personalized analysis of each student's conversational behavior, revealing differences and commonalities among students.

[0084] Label (Integer): The label attribute is an integer representing the nature of each user's chat log. Labels 0, 1, and 2 correspond to noise, question, and answer, respectively. This attribute provides key information for categorizing the conversation content, helping to accurately distinguish and understand conversations of different natures.

[0085] Match (string type): The Match attribute represents the relevance of the current record to the next 100 records. If the current sentence is a question and there is a suitable answer in the next 100 sentences, it is marked as 1; otherwise, it is marked as 0. This attribute provides information about the correlation between questions and answers, providing key clues for question-and-answer matching in the TGQAM task.

[0086] Timestamp (Integer): The timestamp, as an integer attribute, specifies the time when the user's statement was generated. To simplify calculations, the initial timestamp is set to 10000, representing the time when the first sentence began. This attribute allows analysis of the temporal distribution of student dialogue activities, enabling research into students' learning behaviors and interaction patterns across different time periods.

[0087] These detailed and comprehensive attributes make the BigData dataset a powerful tool for in-depth research into the structure, patterns, and trends of student conversations in online education environments. This will provide valuable insights for research, help understand students' learning needs in courses, and provide real-world data support for model development in TGQAM tasks. Table 2 shows an example of the dataset.

[0088] Table 2 Dataset Examples

[0089]

[0090] In addition to the BigData22 and BigData23 datasets, a synthetic dataset was constructed to evaluate the performance of StyleTimeQAM on larger-scale datasets. The synthetic dataset was generated to broaden the research scope, particularly when facing larger-scale and more diverse dialogue data, to more comprehensively validate the model's robustness and generalization ability. However, existing QA datasets, such as the Wiki QA dataset, primarily contain questions and answers, lacking detailed information on dialogue time or user type. Therefore, the distribution of dialogue time and user type in the dataset was analyzed, and similar features were generated in the synthetic dataset based on these distributions.

[0091] When generating the synthetic dataset, two key features were considered: conversation time and user type. First, a detailed analysis was performed using the distributions of conversation time and user type from the BigData22 dataset. Then, these distributions were applied to the generation process of the synthetic dataset, ensuring that the generated dataset remained consistent with the original dataset in both aspects.

[0092] For textual information, questions and answers from the Wiki QA dataset are directly merged and described as user group statements in the synthetic dataset. This approach preserves the semantic information of the original questions and answers while providing quality and diversity to the synthetic dataset.

[0093] Overall, the synthetic dataset is built upon the BigData22 dataset, with each dialogue possessing the same characteristics as those in both the BigData22 and BigData23 datasets. This design allows the synthetic dataset to better simulate the dialogue characteristics of real-world educational community settings, thus providing an in-depth evaluation of StyleTimeQAM's performance in more complex and large-scale environments. By ensuring that the features of the synthetic dataset reflect the distribution of the real dataset, the effectiveness and robustness of the model can be inferred with greater confidence in a wider range of application scenarios.

[0094] During the generation of the synthetic dataset, a series of clearly defined principles were adhered to to ensure that the synthetic dataset matched the original BigData22 dataset in several key aspects, thereby maintaining the authenticity and plausibility of the generated data. The following is a detailed explanation of each principle followed:

[0095] Principle 1: Matching the probability distribution of dialogues across different dates: Ensure that the probability distribution of dialogues across different dates in the synthetic dataset matches the probability distribution in the BigData22 dataset. This means reasonably simulating the date distribution of dialogues in the synthetic dataset to reflect the temporal distribution patterns of dialogues in real educational community groups.

[0096] Principle 2: Match the probability distribution of dialogues throughout the day: Ensure that the probability distribution of dialogues in the synthetic dataset throughout the day matches the probability distribution in the BigData22 dataset. This helps simulate the changing trends of dialogue activity throughout the day, making the synthetic data closer to real-world scenarios.

[0097] Principle 3: Match the probability distribution of the time interval between questions and answers: Maintain the probability distribution of the time interval between questions and corresponding answers in the synthetic dataset to match the probability distribution in the BigData22 dataset. This helps to accurately simulate students' reaction times when answering questions and the time structure in conversations.

[0098] Principle 4: Match the probability of users asking questions: Ensure that the probability of users asking questions in the synthetic dataset matches the probability of asking questions in the BigData22 dataset. This principle helps maintain the authenticity and diversity of student questioning behavior.

[0099] Principle 5: Match the probability of users providing answers: Maintain that the probability of users providing answers in the synthetic dataset matches the probability of providing answers in the BigData22 dataset. This helps to simulate patterns of students engaging in conversations and sharing knowledge.

[0100] Principle 6: Match the probability of users providing irrelevant information: Ensure that the probability of users providing irrelevant information in the synthetic dataset matches the probability of providing irrelevant information in the BigData22 dataset. This helps to simulate noise and irrelevant information in dialogues, making the generated data closer to real-world dialogue scenarios.

[0101] By adhering to these principles, the goal is to create a highly realistic and comparable dialogue environment within a synthetic dataset, enabling accurate and comprehensive evaluation of StyleTimeQAM's performance on larger datasets. This set of principles helps ensure that the generated dataset maintains key features and probability distributions consistent with real-world data.

[0102] Figure 2 (a) shows a comparison of the probability distribution of conversations on different dates between the synthetic dataset and the BigData22 dataset. The results show a matching probability distribution, thus confirming that the synthetic dataset conforms to principle 1.

[0103] Figure 2 (b) The probability distributions of conversations within a day were compared between the synthetic dataset and the BigData22 dataset. The results showed a consistent probability distribution, confirming that the synthetic dataset conforms to principle 2.

[0104] like Figure 2As shown in (c), the probability distribution of dialogues within a day was compared between the synthetic dataset and the BigData22 dataset. The comparison verified that the synthetic dataset meets the requirements of principle 3.

[0105] Figure 2 (d) Figure 2 (e) and Figure 2 (f) shows the proportions of questions, answers, and irrelevant information in the dialogues of the 61 users identified in the synthetic dataset and the BigData22 dataset. These results confirm that the synthetic dataset successfully follows principles 4, 5, and 6.

[0106] Step 2: Divide each BigData dataset into a training set, a validation set, and a test set according to a set ratio;

[0107] Step 3: Construct user style-aware and time-aware question-answering matching models;

[0108] To address the significant data noise issue in Education Group Question-Answer Matching (TGQAM), we have meticulously designed a user-style-aware and time-aware question-answer matching model called StyleTimeQAM (hereinafter referred to as StyleTimeQAM). StyleTimeQAM comprises two key modules, each dedicated to solving one of the two main problems that generate substantial data noise in educational group communities.

[0109] First, we identified a problem: questions were being conflated with other forms of conversation. To address this, we introduced an innovative user-aware attention mechanism. Through... Figure 3 As seen in the straight lines and dashed segments in (a) and 3(b), a group of students contributed over 15% and 20% of the questions and answers in the dialogue. This user group with high question-and-answer rates holds a significant position in the dialogue, and our model utilizes a user-style-aware attention mechanism to extract these questions more effectively. Figure 3 (a) and Figure 3 (b) The dashed line further demonstrates that these users have a relatively high question-answering rate. By identifying and leveraging the learning styles of these users, our model is able to improve its ability to distinguish questions from other dialogue types, thereby significantly reducing data noise caused by the mixing of questions and answers, ultimately making the execution of TGQAM more accurate and effective. To fully utilize this insight, we cleverly integrate a user style-aware attention mechanism into our model to extract questions more accurately from all dialogues.

[0110] Secondly, we discovered another problem related to an overabundance of candidate answers. To address this issue, we introduce a time-aware answer-matching attention mechanism. Figure 3As observed in (c), the number of question-answer pairs decreases exponentially with increasing time intervals. To address this issue, we designed a time-aware attention mechanism that intelligently matches questions and answers by considering time intervals, user style consistency, and the relevance of textual information between questions and answers. This module employs an exponentially decaying kernel function to model the personalized relationship weights between questions and answers for different users, ensuring more flexible and accurate matching of questions and answers across different users and time periods.

[0111] Through the synergistic effect of these two key modules, the StyleTimeQAM model excels in addressing data noise issues in TGQAM. This innovative design not only improves the accuracy of question extraction but also effectively addresses the problem of excess candidate answers, providing a more robust solution for question-and-answer matching tasks in educational community groups.

[0112] The StyleTimeQAM framework is as follows: Figure 4 As shown. Our work aims to properly align questions and answers in multi-party, multi-round dialogues.

[0113] A significant source of noise in TGQAM is the mixing of questions with other types of dialogue. To mitigate this issue, we propose a user-style-aware attention module to identify questions, thereby reducing the impact of this extreme noise.

[0114] However, one challenge we encountered was the significant imbalance between the number of questions and other types of conversations. This imbalance made the task of question extraction extremely complex. However, we observed that the majority of questions across all conversations were asked by similar users. Interestingly, the sparsity of these users within the user group was much smaller compared to the overall dispersion of questions throughout the conversation. Figure 3 As shown in (a). Therefore, the user style-aware attention module aims to enhance question extraction by recognizing user styles.

[0115] like Figure 4 As shown, the user style-aware and time-aware question-answering matching model includes a user style-aware attention module and a time-aware attention module;

[0116] like Figure 4 and Figure 5As shown, the user style-aware attention module receives N dialogue sequences as input. It uses a word embedding layer to embed the words that make up the dialogue sequence, obtaining the embedding results for each word. Then, a Bi-LSTM layer is used to obtain the hidden state information of each word's context in each dialogue. An average pooling layer is then used to aggregate the hidden state information of each word in each dialogue, resulting in the encoded representation of each dialogue. A user embedding layer is used to embed the user ID in the dialogue sequence, obtaining the embedding result for the user ID. The confidence score of the dialogue is calculated based on the word embedding results and the user ID embedding results, thus obtaining a conversation representation that considers textual information and user style. Finally, a self-attention mechanism is used to understand the contextual information of the dialogue to obtain the hidden vectors of each dialogue, and these hidden vectors are input into a linear layer to predict the dialogue type.

[0117] The large number of potential answers to a question constitutes a significant source of noise. To mitigate this, we introduce a time-aware question-answering matching attention mechanism.

[0118] The design of the time-aware attention mechanism is based on four principles. First, the probability of a user answering a question decreases exponentially over time. Second, different users have different probabilities of answering the same question within the same time interval. Third, the degree of matching between user styles is considered. Finally, the degree of matching of textual information in the dialogue is discussed. These principles are consistent with our intuitive understanding and common sense.

[0119] The time-aware attention module is used to concatenate the latent vectors of two dialogues and input them into the dialogue relationship matching layer to obtain a matching degree vector of the two dialogue contents; concatenate the embedding results of the user IDs of the two dialogues and input them into the user relationship matching layer to obtain a matching degree vector of the two speaking users; concatenate the matching degree vector of the dialogue content and the matching degree vector of the speaking users to obtain a matching degree vector considering text information and user style; then, an exponential decay kernel function is used to obtain the personalized relationship weight between the two dialogues; and the matching degree vector considering text information and user style, the personalized relationship weight, and the dialogue type are input into the linear layer to obtain the probability of whether the two dialogues match.

[0120] Step 4: Train the user style-aware and time-aware question-answering matching models using the training set, and use the validation set to obtain performance metrics and find the optimal hyperparameters to obtain the final user style-aware and time-aware question-answering matching models.

[0121] Step 4.1: Input the N dialogue sequences from the training set into the user style-aware attention module to obtain the predicted dialogue type;

[0122] The N dialogue sequences are represented as [X1, ..., X...]. N] = [(u1,s1,t1,l1,m1), ... (u N ,s N ,t N ,l N ,m N Each dialogue in the sequence consists of multiple features, where X represents a single dialogue in the dialogue sequence, N represents the number of dialogues, u represents the user ID, s represents the natural language word sequence, t represents the timestamp of the dialogue, l represents the type of dialogue, including Q for question, A for answer, NQA for non-question, non-answer dialogue, and m for which dialogue in the dialogue sequence this dialogue matches. If m = 0, it means that it does not match any dialogue in the dialogue sequence, and if m = 1, it means that it matches the first dialogue in the dialogue sequence.

[0123] Step 4.1.1: Use the word embedding layer to embed the words that make up the dialogue in the dialogue sequence to obtain the embedding results of each word. Then use the Bi-LSTM layer to obtain the hidden state information of each word in each dialogue. Then use the average pooling layer to aggregate the hidden state information of each word in each dialogue to obtain the encoded representation of each dialogue.

[0124] e w,i,j =Embedding(w i,j )

[0125]

[0126]

[0127] Among them, w i,j It is a natural language word sequence s i The j-th word in It is the word w i,j The word embedding results obtained after embedding through the Embedding layer, where i is the number of the natural language word sequence, j is the word number, and Dw is the dimension of the word embedding result; The hidden state information of each word context in each dialogue obtained through the Bi-LSTM layer. This represents the encoded representation of each dialogue obtained through average pooling; Embedding represents the Embedding layer, Bi-LSTM represents the Bi-LSTM layer, and AvgPool represents the average pooling layer.

[0128] Step 4.1.2: Use the user embedding layer to embed the user ID in the dialogue sequence to obtain the embedding result of the user ID;

[0129] e u,i=Embedding(u i )

[0130] Among them, u i For natural language word sequences s i The corresponding user ID; User ID u i The embedding result of the user ID obtained after embedding through the Embedding layer; Du is the dimension of the embedding result of the user ID;

[0131] Step 4.1.3: Calculate the confidence of the dialogue based on the word embedding results and the user ID embedding results, and then obtain the conversation representation that takes into account text information and user style;

[0132]

[0133]

[0134] Among them, w t,i Represents a sequence of words in a natural language. i confidence level, w t,i The higher the value, the stronger the natural language word sequence s i With user number u i The better the match with the user's speaking style; softmax represents the softmax function, To consider the conversational representation of textual information and user style, D s To consider the dimensions of conversational representation, including textual information and user style, W o W k and W v These are trainable parameters;

[0135] Step 4.1.4: Use the self-attention mechanism to understand the contextual information of the dialogue to obtain the latent vectors of each dialogue, and input the latent vectors into the linear layer to obtain the predicted dialogue type;

[0136] To capture user style, we use the embedding of the user ID as the query and the conversation representation, which considers both textual information and user style, as the key and value. This approach allows us to learn the latent vectors of the conversation, while simultaneously optimizing these latent vectors by leveraging user style.

[0137]

[0138]

[0139] in, The latent vectors of the dialogue; W represents the probability that the dialogue type is a question. q and bq These are trainable parameters;

[0140] Step 4.2: Use the time-aware attention module to obtain the matching probability;

[0141] Step 4.2.1: Concatenate the latent vectors of the two dialogues and input them into the dialogue relationship matching layer to obtain the matching degree vector of the two dialogue contents;

[0142] h s,i,i' =ReLu(W s [h s,i ,h s,i' ]+b s )

[0143] in, Let W be the vector representing the degree of matching of the dialogue content, and ReLU be the activation function; s and b s For trainable parameters; D h The dimension of the vector representing the degree of matching of the dialogue content;

[0144] Step 4.2.2: Concatenate the embedding results of the user IDs of the two dialogues and input them into the user relationship matching layer to obtain the matching degree vector of the two dialogue speakers;

[0145] h r,i,i' =ReLu(W r [e u,i ,e u,i' ]+b r )

[0146] in, W is the matching degree vector of the speaking users; r and b r These are trainable parameters;

[0147] Step 4.2.3: Concatenate the matching vector of the dialogue content and the matching vector of the speaking user to obtain a matching vector that takes into account text information and user style. Then, use the exponential decay kernel function to obtain the personalized relationship weight between the two dialogues.

[0148]

[0149]

[0150]

[0151] in, To consider the matching degree vector of text information and user style, To calculate the time decay rate of the answer dialogue, Δ i,i' w is the time difference between speaking in a question-and-answer dialogue and in an answer-and-answer dialogue.t,i,i' For personalized relationship weights, W u and b u These are trainable parameters;

[0152] Step 4.2.4: Input the matching vector considering text information and user style, personalized relationship weights and dialogue type into the linear layer to obtain the probability of whether two dialogues match;

[0153]

[0154]

[0155] in, For the purpose of dialogue, W represents the probability of a match between two dialogues. m and b m These are trainable parameters;

[0156] Step 4.3: Repeat the training until the set number of training rounds is reached to obtain the trained user style-aware and time-aware question-answering matching model;

[0157] The goal is to predict the matching matrix of the dialogue. We achieve this by learning a function f, as follows:

[0158]

[0159] At the same time y m,i,i' The tag representing a session match is defined as:

[0160]

[0161] It's important to note that a single question may have multiple answers. However, the accuracy of all these answers cannot be guaranteed, highlighting the complexity of achieving proper question-answer alignment.

[0162] The loss function used is the cross-entropy loss function, as shown in the following formula:

[0163]

[0164] Where LOSS is the loss function; y m,i,i' For the actual label, when the actual label y m,i,i' When the value is 1, the loss is mainly affected by The impact of the term, which means the model's predicted value. The closer to 1, the smaller the loss. When the actual label y m,i,i' When the value is 0, the loss is mainly affected by The impact of the term, which means the model's predicted value. The closer to 0, the smaller the loss;

[0165] Step 5: Input the test set into the final user style-aware and time-aware question-answering matching model to obtain the matching results;

[0166] To verify the effectiveness of the StyleTimeQAM model, we conducted comparative experiments with three QA models with pre-trained initialization and four QA models without pre-trained initialization. For the pre-trained QA models, we chose CNN-LSTM-CRF, ESIM, and ABCNN. For the QA models without pre-trained initialization, we chose CNN, AP-CNN, BiLSTM-attention, and AP-LSTM.

[0167] Table 3. AUC results of StyleTimeQAM and baseline on the three datasets.

[0168] Model BigData22 BigData23 synthesis AP-CNN 0.8187±0.0051 0.8041±0.0135 0.9294±0.0040 BiLSTM-Attention 0.8236±0.0078 0.8363±0.0051 0.9440±0.0075 AP-LSTM 0.7886±0.0189 0.8155±0.0073 0.9565±0.0066 CNN 0.8235±0.0070 0.7914±0.0096 0.9628±0.0028 CNN-LSTM-CRF 0.8310±0.0060 0.8156±0.0080 0.9531±0.0070 ABCNN 0.7870±0.0093 0.7953±0.0117 0.9276±0.0069 eSIM 0.8047±0.0107 0.8081±0.0094 0.9403±0.0109 StyleTimeQAM 0.8985±0.0154 0.9353±0.0135 0.8581±0.0048

[0169] Table 4. AUC results of StyleTimeQAM and baseline on three problem-free noisy datasets.

[0170]

[0171]

[0172] For each dataset, we allocated the first 60% of dialogues to the training set, the next 20% to the test set, and the remaining 20% ​​to the validation set. For all baseline models, we performed hyperparameter searches on weight decay and learning rate within the ranges {1e-3, 5e-4, 1e-4, 5e-5, 1e-5, 5e-6} and {5e-3, 1e-3, 5e-4, 1e-4, 5e-5, 1e-5, 5e-6, 1e-6}, respectively. Furthermore, while keeping other hyperparameters constant, we searched for the optimal batch size in the range of 32 to 128. Five seeds were randomly selected for each trial, and the average results were used for analysis.

[0173] Table 3 shows the performance of StyleTimeQAM and the baseline model across three datasets. Table 4 shows the model performance without question noise, meaning that we explicitly identified which dialogues were questions during testing and only matched answers to those specific questions. Therefore, the test set corresponding to Table 4 is a subset of the test set associated with Table 3.

[0174] As shown in Tables 3 and 4, StyleTimeQAM outperforms the baseline models on both the BigData22 and BigData23 datasets, under both disturbed and undisturbed conditions. This highlights the effectiveness of StyleTimeQAM in educational group settings. StyleTimeQAM's time-aware strategy also demonstrates its effectiveness, as it outperforms the baseline models on all three datasets when problem noise is removed.

[0175] However, it is worth noting that on synthetic datasets, under perturbation conditions, StyleTimeQAM significantly outperforms the baseline. This can be attributed to the significant difference between text features in large datasets and those in traditional question-answering datasets. The baseline model, designed specifically for traditional QA datasets, performs better than StyleTimeQAM even without explicit question labels. This stems from the fact that identifying questions on traditional datasets is relatively simple compared to the datasets used in the TGQAM task, while the task becomes more challenging on the TGQAM dataset.

[0176] This difference underscores the importance of the datasets we collect, and by highlighting the unique challenges within this context, we can stimulate further innovation and improvement in question-answering systems.

[0177] Figure 6 and Figure 7 The performance of StyleTimeModule was demonstrated after multiple elimination studies, including removing question extraction, removing time information, not providing user ID information, and removing the user relationship matching module.

[0178] Experimental results show that removing question extraction, time awareness, and user learning style significantly impacts model performance. Conversely, the presence or removal of the user relationship matching module has little effect on model effectiveness. This effect can be attributed to the fact that user learning style information is already embedded in the distributed representation of the dialogue within the user style-aware attention module. Therefore, the dialogue relationship matching process takes into account the user's learning style, while user relationship matching is merely a supplement to this process.

Claims

1. A method for matching online education group questions and answers based on user style and time awareness, characterized in that, Includes the following steps: Step 1: Construct several BigData datasets; each BigData dataset contains five key attributes: sentence, user ID, tag, match, and timestamp. Step 2: Divide each BigData dataset into a training set, a validation set, and a test set according to a set ratio; Step 3: Construct user style-aware and time-aware question-answering matching models; The user style-aware and time-aware question-answering matching model includes a user style-aware attention module and a time-aware attention module. The user style-aware attention module is used to receive N dialogue sequences as input. It uses a word embedding layer to embed the words that make up the dialogue in the dialogue sequence to obtain the embedding results of each word. Then, it uses a Bi-LSTM layer to obtain the hidden state information of each word's context in each dialogue. Finally, it uses an average pooling layer to aggregate the hidden state information of each word in each dialogue to obtain the encoded representation of each dialogue. The user ID is embedded in the dialogue sequence using the user embedding layer to obtain the embedding result of the user ID; The confidence of the dialogue is calculated based on the word embeddings and user ID embeddings, and then the conversation representation considering text information and user style is obtained. Finally, the self-attention mechanism is used to understand the context information of the dialogue to obtain the latent vectors of each dialogue, and the latent vectors are input into the linear layer to predict the dialogue type. The time-aware attention module is used to concatenate the latent vectors of two dialogues and input them into the dialogue relationship matching layer to obtain a matching degree vector of the two dialogue contents; concatenate the embedding results of the user IDs of the two dialogues and input them into the user relationship matching layer to obtain a matching degree vector of the two speaking users; concatenate the matching degree vector of the dialogue content and the matching degree vector of the speaking users to obtain a matching degree vector that considers text information and user style; then use an exponential decay kernel function to obtain the personalized relationship weight between the two dialogues; input the matching degree vector considering text information and user style, the personalized relationship weight, and the dialogue type into the linear layer to obtain the probability of whether the two dialogues match; Step 4: Train the user style-aware and time-aware question-answering matching models using the training set, and use the validation set to obtain performance metrics and find the optimal hyperparameters to obtain the final user style-aware and time-aware question-answering matching models. Step 4 specifically includes: Step 4.1: Input the N dialogue sequences from the training set into the user style-aware attention module to obtain the predicted dialogue type; Step 4.2: Use the time-aware attention module to obtain the matching probability; Step 4.3: Repeat the training until the set number of training rounds is reached to obtain the trained user style-aware and time-aware question-answering matching model; Step 4.4: Use the validation set to obtain performance metrics and then find the optimal hyperparameters to obtain the final user style-aware and time-aware question-answering matching model; Step 5: Input the test set into the final user style-aware and time-aware question-answering matching model to obtain the matching results.

2. The online education group question-and-answer matching method based on user style and time awareness according to claim 1, characterized in that, The N dialogue sequences are represented as [X1, ..., X...]. N ] = [(u1, s1, t1, l1, m1),…(u N , s N , t N ,l N , m N Each dialogue in the sequence consists of multiple features, where X represents a single dialogue in the dialogue sequence, N represents the number of dialogues, u represents the user ID, s represents the natural language word sequence, t represents the timestamp of the dialogue, l represents the type of dialogue, including Q for question, A for answer, NQA for non-question, non-answer dialogue, and m for which dialogue in the dialogue sequence this dialogue matches.

3. The online education group question-and-answer matching method based on user style and time awareness according to claim 1, characterized in that, Step 4.1 specifically includes: Step 4.1.1: Use the word embedding layer to embed the words that make up the dialogue in the dialogue sequence to obtain the embedding results of each word. Then use the Bi-LSTM layer to obtain the hidden state information of each word in each dialogue. Then use the average pooling layer to aggregate the hidden state information of each word in each dialogue to obtain the encoded representation of each dialogue. ； ； ； in, It is a sequence of words in natural language The j-th word in It is a word The word embedding results obtained after embedding through the Embedding layer is the number of the word sequence in the natural language, where j is the word number; The dimension of the word embedding results; The hidden state information of each word context in each dialogue obtained through the Bi-LSTM layer. This is the encoded representation of each dialogue obtained through average pooling; Indicates the Embedding layer. Indicates a Bi-LSTM layer. Indicates the average pooling layer; Step 4.1.2: Use the user embedding layer to embed the user ID in the dialogue sequence to obtain the embedding result of the user ID; ； in, Natural language word sequence The corresponding user ID; User ID The embedding result of the user ID obtained after embedding through the Embedding layer; The dimension of the embedding results for user IDs; Step 4.1.3: Calculate the confidence of the dialogue based on the word embedding results and the user ID embedding results, and then obtain the conversation representation that takes into account text information and user style; ； ； in, Represents a sequence of words in natural language Confidence level, This represents the softmax function. To consider the textual information and user style in the session representation, To consider the dimensions of conversation representation, including textual information and user style, , and These are trainable parameters; Step 4.1.4: Use the self-attention mechanism to understand the contextual information of the dialogue to obtain the latent vectors of each dialogue, and input the latent vectors into the linear layer to obtain the predicted dialogue type; ； ； in, The latent vectors of the dialogue; This indicates the probability that the dialogue type is a question. and These are trainable parameters.

4. The online education group question-and-answer matching method based on user style and time awareness according to claim 1, characterized in that, Step 4.2 specifically includes: Step 4.2.1: Concatenate the latent vectors of the two dialogues and input them into the dialogue relationship matching layer to obtain the matching degree vector of the two dialogue contents; ； in, This is a vector representing the degree of matching between the dialogue content. For activation functions; and These are trainable parameters; The dimension of the vector representing the degree of matching of the dialogue content; Step 4.2.2: Concatenate the embedding results of the user IDs of the two dialogues and input them into the user relationship matching layer to obtain the matching degree vector of the two dialogue speakers; ； in, The matching degree vector for the speaking users; and These are trainable parameters; Step 4.2.3: Concatenate the matching vector of the dialogue content and the matching vector of the speaking user to obtain a matching vector that takes into account text information and user style. Then, use the exponential decay kernel function to obtain the personalized relationship weight between the two dialogues. ； ； ； in, To consider the matching degree vector of text information and user style, To calculate the time decay rate of the answer dialogue, The time difference between speaking in the question-and-answer dialogue and the answer-and-answer dialogue. For personalized relationship weights, and These are trainable parameters; Step 4.2.4: Input the matching vector considering text information and user style, personalized relationship weights and dialogue type into the linear layer to obtain the probability of whether two dialogues match; ； ； in, For the purpose of dialogue, The probability of a match between two dialogues; and These are trainable parameters.

5. The online education group question-and-answer matching method based on user style and time awareness according to claim 1, characterized in that, In step 4.3, the cross-entropy loss function is used during the training process, and the formula is as follows: ； in, The loss function; This is the actual label.

Images