A big data analysis method for online course quality evaluation based on knowledge graph
Through knowledge graph embedding and game theory calculation, combined with crawling technology and emotional word segmentation processing library, the theme mining and emotional label integration of user evaluations of online course platforms is solved, and the problem of evaluation objectivity and personalized evaluation under the scale of big data is achieved. More accurate teaching quality assessment and platform optimization.
Patent Information
- Application Number
- CN202411166073.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-23
AI Technical Summary
The existing online course platforms are difficult to effectively analyze the objectivity and credibility of user evaluations under the scale of big data. There are problems such as semantic deviation and lack of personalized evaluation, which leads to insufficient depth and accuracy in teaching quality evaluation, and the course evaluation standards of different platforms There are significant differences between them and quality assessments, making it difficult to achieve personalized assessments.
Using the online course quality evaluation big data analysis method based on knowledge graph, the course evaluation data is preprocessed and emotional label integration through topic modeling, emotional word segmentation processing library, knowledge graph embedding and game theory calculation, combined with crawler technology and classifier sequence. and global importance assessment, and targeted improvement strategies are proposed.
It has achieved a more fair, effective and accurate teaching quality assessment, improved prediction performance, provided in-depth teaching quality insights and scientific basis, and helped the platform continuously improve and optimize teaching quality.
Smart Images

Figure CN118673161B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing, and specifically relates to a big data analysis method for online course quality evaluation based on knowledge graph. Background Art
[0002] Online education course platform is a popular education method nowadays, and it is more prominent in the field of computer education. Different course platforms dynamically generate new user data, which needs to be managed in an effective way to explore how to better use massive data and its diversity to create value and usable insights. With the development of deep learning, in the face of a large amount of platform user course evaluation data, the emergence of sentiment analysis technology provides favorable help for massive analysis, effectively solving the information distortion caused by poor text quality. The correct use of technology for big data analysis, combined with the use of teaching-oriented knowledge, has an important impact on enhancing the teaching quality of platform courses. Nowadays, people pursue high-quality teaching. Online course education platforms need good teaching quality measurement standards, analyze course text evaluations, identify user needs and preferences, and infer the importance and performance of the overall course platform teaching dimensions from individual users, so as to propose more targeted teaching improvement strategies.
[0003] With the rapid development of Internet technology, the construction of online course platforms has been continuously promoted in the field of education. The increase in online courses has also brought about the defect of uneven teaching quality. In order to effectively control the course teaching quality of online course platforms, it is necessary to accurately analyze the course evaluation data of users, continuously promote and adjust the teaching process, and provide powerful improvement methods for the platform. However, due to the huge amount of course evaluation data collected in the big data scale, user feedback information is often prone to semantic deviation, grammatical deficiency and vocabulary restriction, and there may also be false evaluations, information distortion or unbalanced evaluation category learning experience feedback that affects teaching improvement. How to effectively analyze the objectivity and credibility of user evaluation has also become a challenge. The existing methods need to be improved in terms of investment in big data analysis and evaluation technology. It is difficult to support the depth and accuracy of teaching quality evaluation under massive platform data. There are also significant differences in course evaluation standards and quality evaluation in different platforms, making it difficult to achieve personalized evaluation of individual learning needs of different students on different platforms. Summary of the invention
[0004] In order to solve the problems of semantic bias and lack of personalized evaluation under the scale of big data and realize more fair, effective and accurate teaching quality evaluation, the purpose of the present invention is to propose a big data analysis method for online course quality evaluation based on knowledge graph, which extracts themes from course evaluation data and obtains course evaluation sentiment scores in thematic dimensions. The sentiment labels of course evaluation are integrated by knowledge graph embedding, and the course is classified and predicted by classifier sequence. The global importance of the course is obtained by combining the calculation method of game theory, and targeted improvement strategies are proposed according to the key elements affecting teaching quality.
[0005] The present invention is achieved through the following technical solutions.
[0006] A big data analysis method for online course quality evaluation based on knowledge graph, the steps are as follows:
[0007] Step 1: Use crawler technology to crawl user course evaluation data from the online course platform, and perform data cleaning and preprocessing to obtain a course evaluation data set;
[0008] Step 2: Use the topic modeling model to conduct topic mining on the course evaluation dataset, calculate the confidence to determine the optimal number of topics and model configuration, and obtain the topic list of different online course platforms;
[0009] Step 3: Combine the sentiment segmentation processing library (SnowNLP) to calculate the sentiment score of each user course evaluation under each topic in the topic list, and obtain the sentiment score matrix as the evaluation feature data of the input data set of the classifier sequence;
[0010] Step 4: Use knowledge graph embedding to modify the original sentiment labels of the course evaluation dataset, map the evaluation text information into the vector embedding of entity triples, and combine the text embedding model to enhance the semantic information to obtain the final embedding of the user course evaluation;
[0011] Step 5: The final embedding of the user course evaluation is passed into the sentiment classifier, and the secondary sentiment label is obtained through the sentiment classifier prediction. Finally, the original sentiment label and the secondary sentiment label are integrated using the support vector machine to obtain the final sentiment label of each user course evaluation as the label data of the input data set of the classifier sequence;
[0012] Step 6: Establish a classifier sequence, pass in the obtained input data set for K-fold cross-validation training, compare the performance differences of each classifier in the classifier sequence, calculate the contribution of each evaluation feature in each classifier as performance, and combine the calculation algorithm based on game theory to obtain the global importance value of each evaluation feature in different topic dimensions;
[0013] Step 7: Combine the global importance value and performance of each evaluation feature under different subject dimensions, and use the importance-performance technique to analyze the impact of each subject on different online course platforms.
[0014] Further preferably, in step 2, the latent Dirichlet distribution model is used to perform topic mining on the course evaluation dataset, the user course evaluation is embedded and processed to obtain a noun matrix, a sentence matrix and a word matrix, and a corpus is constructed using the noun matrix by setting a hyperparameter combination of the latent Dirichlet distribution model, the number of occurrences of each noun is recorded, and a consistency measure is obtained. The optimal number of topics of the course evaluation dataset is determined according to the consistency measure, and finally a topic list of the course evaluation dataset of different online course platforms is obtained.
[0015] Further preferably, in step three, based on the acquired topic list, for each sentence vector in the sentence matrix, the word vectors in the word matrix are traversed for comparison, and the sentence vectors that meet the topic list are calculated using the sentiment segmentation processing library to obtain the sentiment score of each topic in each user's course evaluation.
[0016] Further preferably, step 4 includes the following process:
[0017] Use syntactic dependency analysis to extract entity triples from the course evaluation dataset;
[0018] Use the TransE algorithm to map entity triples into a dense low-dimensional space;
[0019] Initialize low-dimensional vector representations of entities and relationships by traversing entity triples, obtain low-dimensional head entity set, low-dimensional entity relationship vector set and low-dimensional tail entity set, construct the low-dimensional head entity set, low-dimensional entity relationship vector set and low-dimensional tail entity set as training sets, define a minimization loss function to learn the embedding vectors of entities and relationships, obtain the embedding vectors of entity triples through repeated iterative training, and obtain the entity triple embedding after averaging;
[0020] The BERT model is combined with the FastText model to perform semantic enhancement processing on the course evaluation dataset at the word and part-of-speech level, and the word matrix and part-of-speech matrix of user course evaluation are constructed. The word vector embedding and part-of-speech vector embedding are obtained by pre-training the BERT model, and then averaged with the entity triple embedding to obtain the final embedding of the user course evaluation.
[0021] Further preferably, in step five, the final embedding of the user course evaluation obtained in step four is passed into the BERT model and combined with the bidirectional long short-term memory network to obtain the secondary sentiment label of each user course evaluation.
[0022] In the process of label integration of the original emotion label and the secondary emotion label using support vector machine, the original emotion label L is assumed to be a positive sample, and the secondary emotion label U is assumed to be a negative sample. 1 ,lr 2 ,…,lr 2i} represents the set of all sample points, lr 1 ,lr 2 ,…,lr 2i The decision function used to determine the label assigned to the data point LR is represented by the 1st, 2nd, .., 2ith user course evaluation. The specific process is shown in the following formula:
[0023] ;
[0024] in, represents the weight vector, b represents the bias term, when f(LR)>0, the LR data point is assigned to class L; when f(LR)<0, the LR data point is assigned to class U, and finally the final sentiment label T={t f1 ,t f2 ,…,t fi}, t f1 ,t f2 ,…,t fi They represent the final sentiment labels of the 1st, 2nd,..,i-th user course evaluations respectively.
[0025] Further preferably, in step six, the acquired sentiment score matrix is combined with the final sentiment label to establish an input data set for the classifier sequence, the input data set is randomly classified into K training sets and test sets of equal size, K-1 sub-samples are set as training sets, and the remaining sub-samples are used as test sets to verify the classifier, and K classifier sequences are constructed based on the K training sets.
[0026] Further optimization is performed by selecting the best classifier in the classifier sequence as the performance input for the global importance evaluation of teaching, and selecting the performance difference predicted by the best classifier on the course evaluation dataset to calculate the contribution of each evaluation feature in each classifier as the performance result, and using KL divergence to estimate the difference:
[0027] ;
[0028] in represents the value of KL divergence, Indicates evaluation characteristics The prediction result on the kth classifier is: To evaluate the characteristics The real result of The contribution ZD of each evaluation feature based on the classifier is obtained as the performance result.
[0029] Further preferably, a computational feature method based on game theory is used to obtain a global importance estimate of the entire input data set, estimate K importance values in K classifiers, weighted average the marginal contribution of each evaluation feature in all possible feature orders, and obtain the importance value of each evaluation feature. K classifiers are combined with the importance value of each evaluation feature to obtain a global importance value of each evaluation feature based on the classifier performance, and normalized. According to the normalized global importance value of each evaluation feature, it is iterated with the topic list to obtain the global importance value of each evaluation feature corresponding to each topic.
[0030] The present invention adopts a research method that combines a topic modeling model to mine massive data, uses knowledge graph embedding to modify the emotional labels of course evaluation data, and combines the calculation method of game theory to evaluate the teaching quality of the overall course information for targeted improvement. Through thematic mining of massive course data, the organizational results and key areas of course content are understood, and the user's emotional attitude is more accurately evaluated on the basis of label integration of the original course evaluation data, which provides a reference basis for course improvement and performs a global importance assessment of the overall course information. It not only achieves significant improvements in predictive performance, but also provides in-depth insights and scientific basis for evaluating the teaching quality of the platform, and realizes a more fair, effective and accurate teaching quality evaluation. Help the platform to achieve continuous improvement and optimization of teaching quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a flow chart of the big data analysis method for online course quality evaluation based on knowledge graph provided by the present invention.
[0032] Figure 2 This is a schematic diagram of the overall structure of the big data analysis method for online course quality evaluation based on knowledge graph.
[0033] Figure 3 It is a structural diagram of the sentiment label correction strategy.
[0034] Figure 4 This is a diagram of the label distribution of the course evaluation datasets on four platforms. DETAILED DESCRIPTION
[0035] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.
[0036] Reference Figure 1 and Figure 2 , a big data analysis method for online course quality evaluation based on knowledge graph, the steps are as follows:
[0037] Step 1: Use crawler technology to crawl the user course evaluation data of the online course platform, and perform data cleaning and preprocessing to obtain the course evaluation data set. Crawl the evaluation text and evaluation star rating of the course evaluation details page of each online course platform respectively. It is known that the crawled user course evaluation data includes five-star original sentiment label data. Remove duplicate content, blanks, punctuation marks, numbers, special characters, emoticons and other invalid content from the text information. The course evaluation data sets of the four online course platforms finally obtained are shown in Table 1:
[0038] Table 1
[0039]
[0040] The distribution of original sentiment labels of the course evaluation datasets on the four platforms is as follows: Figure 4 As shown, it can be observed that the MOOC course evaluation dataset and the Coursera course evaluation dataset have a more even distribution of categories, while the NetEase Cloud course evaluation dataset and the Udemy course evaluation dataset have a more unbalanced distribution.
[0041] Step 2: Use the topic modeling model to mine the course evaluation dataset, calculate the confidence to determine the optimal number of topics and model configuration, and obtain the topic list of different online course platforms. 1 ,e 2 ,…,e i}, where e 1 ,e 2 ,…,e i Represent the 1st, 2nd, …, i-th user course evaluations respectively. The latent Dirichlet allocation model (LDA) is used to perform topic mining on the course evaluation dataset, and the user course evaluations are embedded to obtain the noun matrix N={n 1 ,n 2 ,…,n i}, where n 1 ,n 2 ,…,n i Respectively represent the noun vectors of the 1st, 2nd, …, i-th user course evaluation, and the sentence matrix C = {c 1 ,c 2 ,…,c i} and the word matrix W = {w 1 ,w 2 ,…,w i}, where c 1 ,c 2 ,…,c i Respectively represent the sentence vectors of the 1st, 2nd, …ith user course evaluation, w 1 ,w 2 ,…,wi Represent the word vectors of the 1st, 2nd, …, i-th user course evaluation respectively. By setting the hyperparameter combination of the latent Dirichlet distribution model, the noun matrix N={n 1 ,n 2 ,…,n i}Build the corpus Corpus to record the number of times each noun appears. ,in Respectively represent the total number of nouns in the noun vectors of the 1st, 2nd, …, i-th user course evaluations, ,in Represents the noun vector n of the i-th user course evaluation i The number of times the 1st, 2nd, ..., uth nouns appear in . Get the consistency metric C v , the calculation formula is shown in formula (1):
[0042] (1);
[0043] Where I represents the number of user course evaluations in the corpus, C i represents the number of nouns in the i-th user course evaluation, The noun vector n represents the i-th user course evaluation i The total number of nouns, represents the number of unique words that appear only once in all user course reviews. represents the smoothing parameter. The optimal number of topics for the course evaluation dataset is determined based on the consistency metric. The specific process is shown in formula (2):
[0044] (2);
[0045] Among them J * represents the optimal number of topics, and J represents the number of different topics obtained by the consistency metric under the hyperparameter combination. Finally, the topic list of the course evaluation dataset of different online course platforms is obtained. 1 ,the 2 ,…,the j}, where the 1 ,the 2 ,…,the j They represent the 1st, 2nd,…,jth topics respectively.
[0046] Step 3: Combine the sentiment segmentation processing library (SnowNLP) to calculate the sentiment score of each user course evaluation under each topic in the topic list, and obtain the sentiment score matrix as the evaluation feature data of the input data set of the classifier sequence. 1 ,the2 ,…,the j}, for each sentence vector in the sentence matrix C, traverse the word vectors in the word matrix W for comparison, use the sentiment segmentation processing library (SnowNLP) to calculate the sentence vectors that meet the topic list, and get the sentiment score of the jth topic in the i-th user course evaluation. score ∈ [ 0 , 1 ] In order to construct the data set required for the classifier sequence, the obtained sentiment scores need to be converted into five labels, namely 0, 1, 2, 3, and 4. The sentiment threshold is limited to the range of [0, 1]. The closer the value is to 1, the more positive the sentiment is, and the closer the value is to 0, the more negative the sentiment is. The intensity range of sentiment polarity is expanded to a triple classification of positive emotion, neutral emotion, and negative emotion. The sentiment scores of 3 and 4 are classified as positive emotions; 2 is classified as neutral emotions; and 0 and 1 are classified as negative emotions. The specific conversion process is shown in formula (3):
[0047] (3);
[0048] Sentiment score matrix corresponding to the topic list As the input variable of the global importance assessment model, S 1 ,S 2 ,…,S i denote the sentiment scores of the 1st, 2nd, …, i-th user course evaluations, respectively, and , where S i1 ,S i2 ,…,S ij They represent the sentiment scores of the i-th user course evaluation on the 1st, 2nd, …, jth topics respectively.
[0049] Step 4: Use knowledge graph embedding to modify the original sentiment labels of the course evaluation dataset, map the evaluation text information into the vector embedding of entity triples, and combine the text embedding model to enhance the semantic information to obtain the final embedding of the user course evaluation. The specific process is as follows Figure 3As shown in the figure, syntactic dependency analysis is used to extract entity triples from the course evaluation dataset, and a dependency system is constructed according to the dependency semantic normal form DSNF structure, including describing the relationship between verbs and other components in the sentence through the predicate-argument structure, assigning syntactic and semantic relationships using roles and reference grammar, and combining the mapping relationship between syntactic structure and semantic structure with the meaning-text theory pattern matching to obtain a list of candidate entities, and identify entities such as names of people, institution names, place names, other nouns, abbreviations, etc. in user course evaluations. For each user course evaluation, syntactic dependency analysis is used to identify the entity information in the user course evaluation, and a triple relationship of (head entity, entity relationship, tail entity) is constructed. The BIE entity annotation rule is used for sequence annotation, where the B label is used to mark the beginning of an entity, the I label is used to mark the middle of an entity, and the E label is used to mark the end of an entity. The head entity set h={h 1 ,h 2 ,…,h i}、Tail entity set t={t 1 ,t 2 ,…,t i}, where h 1 ,h 2 ,…,h i Respectively represent the head entity vectors of the 1st, 2nd, .., i-th user course evaluation, t 1 ,t 2 ,…,t i Represent the tail entity vectors of the 1st, 2nd, .., i-th user course evaluation respectively; then, the relationship between the head and tail entities is judged according to the DSNF discrimination rule, and the entity relationship vector set r={r 1 ,r 2 ,…,r i}, where r 1 ,r 2 ,…,r i Represent the entity relationship vectors of the 1st, 2nd, .., i-th user course evaluation respectively, and combine them to obtain the entity triple M={(h 1 ,t 1 ,r 1 ), (h 2 ,t 2 ,r 2 ),…, (h i ,t i ,r i )}.
[0050] The TransE algorithm is used to map entity triples into a dense low-dimensional space to achieve semantic information representation of entities and relationships, in preparation for subsequent sentiment analysis. The low-dimensional vector representation of entities and relationships is initialized by traversing entity triples to obtain a low-dimensional head entity set. , low-dimensional entity relationship vector set and low-dimensional tail entity sets ,in , , , Respectively represent the low-dimensional head entity vectors of the 1st, 2nd, .., i-th user course evaluation, Respectively represent the low-dimensional tail entity vectors of the 1st, 2nd, .., i-th user course evaluations, Represent the low-dimensional entity relationship vectors of the 1st, 2nd, .., i-th user course evaluation respectively, and construct the low-dimensional head entity set, low-dimensional entity relationship vector set and low-dimensional tail entity set as the training set , define the minimization loss function to learn the embedding vectors of entities and relations, divide the entity triples into positive triples and negative triples, and use the difference between the two to minimize the Euclidean distance of the positive samples as much as possible, so that the vector representation of the positive triples is closer in the low-dimensional vector space, and the vector representation of the negative triples is more dispersed. The minimization loss function and energy formula are defined as shown in formula (4) and formula (5):
[0051] min ∑ ( h i ' , r i ' , t i ' ) ∈ A ∑ ( h i ' ' , r i ' ' , t i ' ' ) ∈ A ' ( h i ' ' , r i ' ' , t i ' ' ) ∈ softmax γ + d ( h i ' + r i ' , t i ' ) − d ( h i ' ' + r i ' ' , t i ' ' )] (4);
[0052] d ( h i ' + r i ' , t i ' ) = [( h i ' + r i ' ) − t i ' ] 2 (5);
[0053] in Indicates the control of the gap between positive and negative samples. and represents the entity that does not belong to the i-th triple, represents the entity relationship that does not belong to the i-th triple, represents the training set consisting of entity triplets that do not belong to the i-th triplet. The embedding vector of the entity triplet is obtained through repeated iterative training and learning. Let the embedding of the entity set be Ent={ent 1 ,ent 2 ,…,ent i}, the embedding of the relation set is Ral={ral 1 ,ral 2 ,…,ral i}, ent 1 ,ent 2 ,…,ent iRespectively represent the entity embedding of the 1st, 2nd,..,ith user course evaluation, ral 1 ,ral 2 ,…,ral i They represent the relation embeddings of the 1st, 2nd, .., i-th user course evaluations respectively, and after averaging, the entity triple embedding V is obtained. The specific process is shown in formula (6):
[0054] (6);
[0055] In order to make up for the problem of insufficient semantic information and inaccurate sentiment analysis caused by the incompleteness of some entities in user course evaluations, the BERT model (Bidirectional Encoder Representations from Transformers) is combined with the FastText model to analyze the course evaluation dataset E={e 1 ,e 2 ,…,e i} to perform semantic enhancement processing at the word and part-of-speech level and construct the word matrix W={w 1 ,w 2 ,…,w i} and the part-of-speech matrix P = {p 1 ,p 2 ,…,p i}, where w 1 ,w 2 ,…,w i Respectively represent the word vectors of the 1st, 2nd, …, i-th user course evaluation, p 1 ,p 2 ,…,p i Represent the part-of-speech vectors of the 1st, 2nd, …, i-th user course evaluation respectively, and obtain the word vector embedding B={b 1 ,b 2 ,…,b i} and part-of-speech vector embedding F={f 1 ,f 2 ,…,f i}, b 1 ,b 2 ,…,b i Respectively represent the word vector embeddings of the 1st, 2nd,..,ith user course evaluation, f 1 ,f 2 ,…,f i Respectively represent the part-of-speech vector embeddings of the 1st, 2nd, .., i-th user course evaluation, combined with the entity triple embedding V={v 1 ,v 2 ,…,v i} for averaging, where v1 ,v 2 ,…,v i represents the entity triple embedding of the 1st, 2nd, …, i-th user course evaluation, and obtains the final embedding G={g 1 ,g 2 ,…,g i}, g 1 ,g 2 ,…,g i They represent the final embeddings of the 1st, 2nd,..,i-th user course evaluations respectively.
[0056] Step 5: The final embedding of the user course evaluation is passed into the sentiment classifier, and the secondary sentiment label is obtained through the sentiment classifier prediction. Finally, the original sentiment label and the secondary sentiment label are integrated using the support vector machine to obtain the final sentiment label of each user course evaluation as the label data of the input data set of the classifier sequence. The final embedding G={g 1 ,g 2 ,…,g i} is passed into the BERT model combined with the bidirectional long short-term memory network BiLSTM (Bidirectional Long Short-Term Memory) to obtain the secondary sentiment label U={u 1 ,u 2 ,…,u i},u 1 ,u 2 ,…,u i They represent the secondary sentiment labels of the 1st, 2nd, .., i-th user course evaluations, including positive and negative sentiment polarities. The recognition results of the four user course evaluation datasets in the present invention are shown in Table 2.
[0057] Table 2
[0058]
[0059] For the original sentiment label L={l 1 ,l 2 ,…,l i} and the secondary sentiment label U={u 1 ,u 2 ,…,u i}, use the SVM (Support Vector Machine) supervised learning algorithm to perform label integration. SVM is a supervised learning framework that separates positive and negative data points by finding an optimal hyperplane. Let the original sentiment label L be the positive sample and the secondary sentiment label U be the negative sample. LR={lr 1,lr 2 ,…,lr 2i} represents the set of all sample points, lr 1 ,lr 2 ,…,lr 2i The decision function used by the 1st, 2nd, .., 2ith user course evaluations respectively determines the label assigned to the data point LR. The specific process is shown in formula (7):
[0060] (7);
[0061] in, represents the weight vector, defining the direction of the hyperplane, b represents the bias term, determining the position of the hyperplane, when f(LR)>0, the LR data point is assigned to class L; when f(LR)<0, the LR data point is assigned to class U, and finally the final sentiment label T={t f1 ,t f2 ,…,t fi}, t f1 ,t f2 ,…,t fi They represent the final sentiment labels of the 1st, 2nd,..,i-th user course evaluations respectively.
[0062] Step 6: Establish a classifier sequence, pass in the obtained input data set for K-fold cross-validation training, compare the performance differences of each classifier in the classifier sequence, calculate the contribution of each evaluation feature in each classifier as performance, and combine the calculation algorithm based on game theory to obtain the global importance value of each evaluation feature in different topic dimensions. With the final sentiment label T = {t f1 ,t f2 ,…,t fi}, and establish the input data set D of the classifier sequence. The specific process is shown in formulas (8) and (9):
[0063] (8);
[0064] (9);
[0065] Where X represents the evaluation feature data of the input data set, Y represents the label data of the input data set, and F 1 and F 2 represents the data preprocessing function, represents the i-th evaluation feature of the input data set, Represents the i-th label of the input data set. The input data set is randomly classified into K training sets and test sets of equal size, K-1 subsamples are set as training sets, and the remaining subsamples are used as test sets to verify the classifier. K classifier sequences are constructed based on the K training sets. In order to solve the problem of sample imbalance in the input data set, the present invention mainly uses a neural network classifier with a hidden layer or with gradient boosting processing, and sets MLP multi-layer perceptron classifier (Multi-Layer Perceptron Classifier), LightGBM lightweight gradient booster (Light Gradient Boosting Machine), GBM (Gradient Boosting Machine) gradient boosting machine, CatBoost (Categorical boosting) category enhancement classifier 4 classifiers to iterate the classifier sequence, compare the recognition effect of the classifier, and select the classifier with the best effect in the classifier sequence as the performance input for the global importance evaluation of teaching. The prediction results of the optimal classifiers on the four course evaluation data sets are shown in Table 3.
[0066] Table 3
[0067]
[0068] When determining the sentiment score of each course evaluation based on the topic list, there may be a problem of missing features caused by the lack of the topic element in a certain evaluation. To solve this problem, the present invention selects the performance difference predicted by the best classifier on the course evaluation dataset to calculate the contribution of each evaluation feature in each classifier as the performance result, and uses the Kullback-Leibler (KL) divergence to estimate the difference. The specific process is shown in formula (10).
[0069] (10);
[0070] in represents the value of KL divergence, Indicates evaluation characteristics The prediction result on the kth classifier is: To evaluate the characteristics The real result. The contribution ZD of each evaluation feature based on the classifier is obtained as the performance result.
[0071] The game theory-based feature calculation method is used to obtain the teaching global importance estimation of the entire input data set. This embodiment uses the Shapley Additive Global Explanation (SAGE) algorithm to represent the Shapley Additive Global Importance Algorithm. By considering all possible feature subsets to calculate the marginal contribution of each feature, an accurate assessment of feature importance is achieved, and K importance values in K classifiers are estimated. Let Represents the review features x of the training set i The importance value on the classifier Z is obtained by weighted average of the marginal contribution of each evaluation feature in all possible feature orders, and the specific calculation process is shown in formula (11):
[0072] (11);
[0073] Where |X| and |Q| are the sets of all evaluation features and all subsets of evaluation features, Z(Q ∪ x i ) represents the evaluation feature set with order, and Z(Q) represents the contribution of the evaluation feature with order to the set.
[0074] For each evaluation feature x in the course evaluation dataset i , the global importance value of each evaluation feature is calculated by the K classifier sequence, let represents the normalized performance of the k-th classifier on the test set. The calculation process is shown in formula (12):
[0075] (12);
[0076] Among them, w k represents the F1 value of the kth classifier on the test set. By combining the K classifiers with the importance value of each evaluation feature, the global importance value of each evaluation feature based on the classifier performance is obtained. The specific process is shown in formula (13):
[0077] (13);
[0078] Get the global importance value of each evaluation feature , and normalize the global importance value of each evaluation feature, as shown in formula (14):
[0079] (14);
[0080] According to the normalized global importance value of each evaluation feature obtained , and compare it with the topic list The={the 1,the 2 ,…,the j} to iterate and obtain the global importance value of each evaluation feature in different topic dimensions. 1 ,imp 2 ,…,imp j}, imp 1 ,imp 2 ,…,imp j They represent the importance values of the evaluation features in the 1st, 2nd,…,jth topic dimensions respectively.
[0081] Step 7: Combine the global importance value and performance of each evaluation feature in different subject dimensions, and use the importance-performance technique to analyze the impact of each subject on different online course platforms. 1 ,imp 2 ,…,imp j}, it is known that the performance result of the optimal classifier ZD={zd 1 ,zd 2 ,…,zd i}, zd 1 ,zd 2 ,…,zd i Represent the performance results of the 1st, 2nd, …, i-th evaluation features respectively, and average them together to form the topic list The={the 1 ,the 2 ,…,the jThe importance and performance of each topic in the four online course platforms were obtained, and the Importance Performance Analysis (IPA) importance-performance technique was used for analysis. It is known that IPA usually includes four areas, among which the upper right area represents the (high importance, high performance) area. This quadrant represents topics that perform well in both performance and importance, and are usually regarded as core concerns and priority content; the upper left area represents the (high importance, low performance) area. This quadrant represents topics that score high in importance but perform poorly in performance. They are usually regarded as key tasks or strategic goals, but require special attention and improvement due to poor performance; the lower left area represents the (low importance, low performance) area. This quadrant contains topics that perform poorly in both performance and importance; it usually represents secondary or unimportant content, and does not perform well in achieving goals. These topics may require further evaluation to see if they need to be retained or have their goals adjusted; the lower right area represents the (low importance, high performance) area. This quadrant typically indicates topics that perform well in terms of performance but score low in importance. Although they perform well in achieving their goals, they may not be the top priority at the moment due to their low importance. These may require further evaluation to see if they deserve more resources.
[0082] In order to enable users to quickly understand and compare the performance and importance of each topic, a comprehensive evaluation strategy for the teaching quality of platform courses is carried out for different topics, and improvement plans are designed purposefully. The specific IPA result analysis and improvement measures of the platform topic list of each course are shown in Table 4:
[0083] Table 4
[0084]
[0085] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.
Claims
1. A big data analysis method for online course quality evaluation based on knowledge graph, characterized in that: Here are the steps: Step 1: Use crawler technology to crawl user course evaluation data from the online course platform, and perform data cleaning and preprocessing to obtain a course evaluation data set; Step 2: Use the topic modeling model to conduct topic mining on the course evaluation dataset, calculate the confidence to determine the optimal number of topics and model configuration, and obtain the topic list of different online course platforms; Step 3: Combine the sentiment segmentation processing library to calculate the sentiment score of each user course evaluation under each topic in the topic list, and obtain the sentiment score matrix as the evaluation feature data of the input data set of the classifier sequence; Step 4: Use knowledge graph embedding to modify the original sentiment labels of the course evaluation dataset, map the evaluation text information into the vector embedding of entity triples, and combine the text embedding model to enhance the semantic information to obtain the final embedding of the user's course evaluation; Step 4 includes the following processes: Use syntactic dependency analysis to extract entity triples from the course evaluation dataset; Use the TransE algorithm to map entity triples into a dense low-dimensional space; Initialize low-dimensional vector representations of entities and relationships by traversing entity triples, obtain low-dimensional head entity set, low-dimensional entity relationship vector set and low-dimensional tail entity set, construct the low-dimensional head entity set, low-dimensional entity relationship vector set and low-dimensional tail entity set as training sets, define a minimization loss function to learn the embedding vectors of entities and relationships, obtain the embedding vectors of entity triples through repeated iterative training, and obtain the entity triple embedding after averaging; The BERT model is combined with the FastText model to perform semantic enhancement processing on the word and part-of-speech level of the course evaluation dataset, and the word matrix and part-of-speech matrix of the user course evaluation are constructed. The word vector embedding and part-of-speech vector embedding are obtained by pre-training the BERT model, and the entity triple embedding is averaged to obtain the final embedding of the user course evaluation. Step 5: The final embedding of the user course evaluation is passed into the sentiment classifier, and the secondary sentiment label is obtained through the sentiment classifier prediction. Finally, the original sentiment label and the secondary sentiment label are integrated using the support vector machine to obtain the final sentiment label of each user course evaluation as the label data of the input data set of the classifier sequence; Step 6: Establish a classifier sequence, pass in the obtained input data set for K-fold cross-validation training, compare the performance differences of each classifier in the classifier sequence, calculate the contribution of each evaluation feature in each classifier as performance, and combine the calculation algorithm based on game theory to obtain the global importance value of each evaluation feature in different topic dimensions; Step 7: Combine the global importance value and performance of each evaluation feature under different subject dimensions, and use the importance-performance technique to analyze the impact of each subject on different online course platforms.
2. The big data analysis method for online course quality evaluation based on knowledge graph according to claim 1 is characterized in that: In step 2, the latent Dirichlet distribution model is used to perform topic mining on the course evaluation dataset, and the user course evaluation is embedded to obtain the noun matrix, sentence matrix and word matrix. By setting the hyperparameter combination of the latent Dirichlet distribution model, the noun matrix is used to build a corpus, the number of occurrences of each noun is recorded, and the consistency measure is obtained. The optimal number of topics for the course evaluation dataset is determined based on the consistency measure, and finally a list of topics for the course evaluation datasets of different online course platforms is obtained.
3. The big data analysis method for online course quality evaluation based on knowledge graph according to claim 1 is characterized in that: In step three, based on the acquired topic list, each sentence vector in the sentence matrix is traversed and compared with the word vectors in the word matrix. The sentiment segmentation processing library is used to calculate the sentence vectors that meet the topic list to obtain the sentiment score of each topic in each user's course evaluation.
4. The big data analysis method for online course quality evaluation based on knowledge graph according to claim 1 is characterized in that: In step 5, the final embedding of the user course evaluation obtained in step 4 is passed into the BERT model and combined with the bidirectional long short-term memory network to obtain the secondary sentiment label of each user course evaluation.
5. The method for analyzing big data of online course quality evaluation based on knowledge graph according to claim 4 is characterized in that: In the process of label integration of the original emotion label and the secondary emotion label using support vector machine, the original emotion label L is assumed to be a positive sample, the secondary emotion label U is assumed to be a negative sample, LR={lr1,lr2,…,lr 2i } represents the set of all sample points, lr1,lr2,…,lr 2i The decision function used to determine the label assigned to the data point LR is represented by the 1st, 2nd, .., 2ith user course evaluation. The specific process is shown in the following formula: ; in, represents the weight vector, b represents the bias term, when f(LR)>0, the LR data point is assigned to class L; when f(LR)<0, the LR data point is assigned to class U, and finally the final sentiment label T={t f1 ,t f2 ,…,t fi }, t f1 ,t f2 ,…,t fi They represent the final sentiment labels of the 1st, 2nd,..,i-th user course evaluations respectively.
6. The big data analysis method for online course quality evaluation based on knowledge graph according to claim 1 is characterized in that: In step six, the acquired sentiment score matrix is combined with the final sentiment label to establish the input data set of the classifier sequence. The input data set is randomly classified into K training sets and test sets of equal size. K-1 sub-samples are set as training sets, and the remaining sub-samples are used as test sets to verify the classifier. K classifier sequences are constructed based on the K training sets.
7. The method for analyzing online course quality evaluation big data based on knowledge graph according to claim 6 is characterized in that: The best classifier in the classifier sequence is selected as the performance input for the global importance evaluation of teaching. The performance difference predicted by the best classifier on the course evaluation dataset is selected to calculate the contribution of each evaluation feature in each classifier as the performance result, and the difference is estimated using KL divergence: ; in represents the value of KL divergence, Indicates evaluation characteristics The prediction result on the kth classifier is: To evaluate the characteristics The real result of The contribution ZD of each evaluation feature based on the classifier is obtained as the performance result.
8. The method for analyzing big data for online course quality evaluation based on knowledge graph according to claim 7 is characterized in that: Use a computational feature method based on game theory to obtain the global importance estimate of the entire input data set, estimate the K importance values of the K classifiers, and weighted average the marginal contribution of each evaluation feature in all possible feature orders to obtain the importance value of each evaluation feature. Combine the K classifiers with the importance value of each evaluation feature to obtain the global importance value of each evaluation feature based on the classifier performance, and normalize it. According to the normalized global importance value of each evaluation feature, it is iterated with the topic list to obtain the global importance value of each evaluation feature corresponding to each topic.
Citation Information
Patent Citations
User-side commodity knowledge graph construction method based on e-commerce comment
CN113535901A
Book review text classification method based on knowledge graph word sense disambiguation
CN114064901A
Sentiment classification method and device based on knowledge graph, equipment and storage medium
CN115905518A
Intelligent logistics customer service processing method, system and device and storage medium
CN117668205A
Mooc learner learning effect prediction method based on multi-level enhanced contrast learning
CN118227791A
