Online learning intelligent evaluation method and system based on large model

Abstract

The invention discloses an intelligent evaluation method for supporting online learning based on a large model, and the method comprises the following steps: S1, obtaining online learning data of a learner, extracting corresponding learning features from the obtained data, and dividing the learning features into quantifiable evaluation features and non-quantified evaluation features; s2, based on the quantifiable evaluation characteristics, directly calculating to obtain corresponding quantifiable evaluation indexes; s3, analyzing and calculating the non-quantitative evaluation characteristics to obtain non-quantitative evaluation indexes reflecting the learning effect of the learner; and S4, taking the obtained score of each non-quantity evaluation index as input, and completing intelligent evaluation of the learning effect of the learner. And S5, the teacher can autonomously select whether to participate in the learning evaluation process, and finally, the final learning effect of the learner is calculated by fusing teacher evaluation, student learning data and intelligent evaluation. According to the invention, the intelligent evaluation task of the learning effect of the learner is completed on the basis of establishing the online learning effect evaluation index.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to an online learning intelligent evaluation method and system based on a large model. Background Art

[0002] With the development of related concepts of smart education, online education has become an important way for learners to acquire academic knowledge and expand professional skills. As online learning continues to develop, educational resources on online education platforms are also constantly enriched. When faced with massive course resources, learners may have the problem of "cognitive overload and easy to get lost". Characterizing learners' learning portraits through big data mining and other means, exploring learners' learning patterns, and recommending suitable educational resources to learners can better improve learners' learning efficiency. Similarly, because teachers cannot directly participate in learners' learning activities in online learning scenarios, learners may be unable to complete the course learning plan of the education platform on time due to lack of supervision. In addition, the online learning model has the characteristics of learners' great learning flexibility and a large number of course participants. Relying on teachers to conduct one-on-one supervision of learners is not in line with objective reality. Summary of the invention

[0003] The purpose of the present invention is to overcome the above-mentioned defects in the prior art and provide an online learning intelligent evaluation method based on a large model. On the basis of establishing online learning effectiveness evaluation indicators, an intelligent evaluation model corresponding to the evaluation indicators is designed and implemented to complete the task of intelligent evaluation of learners' learning effectiveness.

[0004] To achieve the above object, the present invention provides an online learning intelligent evaluation method based on a large model, comprising the following steps:

[0005] S1. Obtain online learning data of learners, extract corresponding learning features from the above-obtained data, and divide the learning features into quantifiable evaluation features and non-quantitative evaluation features;

[0006] S2. Based on the quantifiable evaluation features, directly calculate and obtain the corresponding quantifiable evaluation indicators;

[0007] S3. Analyze and calculate the non-quantitative evaluation features to obtain non-quantitative evaluation indicators that reflect the learners' learning effectiveness;

[0008] S4. The obtained scores of each non-quantitative evaluation indicator are used as input, and different weights are assigned to each indicator to form a total weighted evaluation score, thereby completing the intelligent evaluation of the learner's learning effectiveness;

[0009] S5. Teachers can choose whether to participate in the learning assessment process. When teachers participate in the assessment, their assessment results have a higher confidence weight. Ultimately, the teacher's assessment, student learning data and intelligent assessment are integrated to calculate the learner's final learning outcomes, and the learner's learning weight is updated based on the assessment results.

[0010] Furthermore, in step S1, the learning data includes the initial confidence weight value obtained from historical learning, and during the learning process, the teacher sets corresponding collaborative learning tasks according to teaching needs, and the learners can conduct learning discussions, task division, and submit learning outcomes during the learning process.

[0011] Furthermore, the learning outcomes of the learners in step S2 are specifically:

[0012] The learner learning outcomes are defined as a five-tuple:

[0013] CLE=<Lead,CA,LA,TC,SC> ;

[0014] The non-quantitative evaluation indicators included in the above five-tuple are:

[0015] Lead is the leadership ability of the learner, CA is the collaboration ability of the learner, and the leadership ability and collaboration ability are mainly evaluated around the interactive behavior of the learner in the learning process; TC is the task contribution of the learner in all tasks, and the task contribution of the learner mainly solves the problem of the individual contribution evaluation of the learner in the process; LA is the learning attitude of the learner; SC is the concentration of the learner, and the learning attitude and concentration of the learner are mainly evaluated around the learner's own learning performance in the process;

[0016] The learners’ learning outcomes are evaluated through the above five-tuple.

[0017] Furthermore, the five-tuple reflects the influence of learning attitude LA and concentration level SC through self-evaluation, mutual evaluation and attitude evaluation, which are as follows:

[0018] Construct a self-evaluation and mutual evaluation model. By collecting learners' evaluations of their own and other learners' behavior performance, calculate the confidence weight value of the learner's current evaluation index and the evaluation results. Specifically, the self-evaluation and mutual evaluation model is mainly calculated using the linear weighting method. Linear weighting is performed according to the confidence weight value of each evaluation. Finally, multiple goals can be expressed as follows:

[0019]

[0020] Among them, X is the final evaluation result, w i and x i Respectively represent the confidence weight value of the current evaluation indicator and the evaluation result of the current evaluation indicator,

[0021] In order to improve the accuracy of self-evaluation and mutual evaluation results, learners with good learning behavior in the past are given higher historical confidence weight values, and learners with poor learning behavior are given lower historical confidence weight values. i The final mutual evaluation score (i.e. evaluation result) in the mutual evaluation stage is taskScore i =x i The calculation formula is as follows:

[0022]

[0023] in, is the weight of learners’ mutual evaluation, E j is the initial mutual evaluation score, let L = (l1,l2,...,l n ) represents the candidate set of the learner’s confidence weight results, l res represents the final prediction result of the current learner confidence weight, l res The calculation is as follows:

[0024] l res =Classifier(B)

[0025] Among them, B is the learner’s learning behavior, Classifier(·) is the model classifier;

[0026] For the evaluation of interaction attitude, assuming that the initial value of the learner's daily interaction attitude is initVal, the weight value weight is calculated as follows:

[0027]

[0028] Among them, count represents the number of times the learner currently participates in learning, and α represents the weight adjusted with the number of discussions;

[0029] If the learner does not participate in the learning discussion of the day, the evaluation result of the learner's learning attitude on that day is the initial value; when the learner participates in the learning discussion process, the learner's discussion data is collected and stored, and the CLIP large model is used to extract the learner's attitude tag sequence from the data and record it as TagSeq = (tag1, tag2, ..., tag n ), the learner's positive and negative interaction attitude adjustment values ​​are posVal and negVal respectively, and the final learner's learning attitude AR is calculated as follows:

[0030]

[0031] The sequence of learner interaction attitude evaluation results is AttSeq = (att1, ..., attn ), take the mean value Avg of the attitude sequence Att As the final evaluation result of learners' interactive attitude.

[0032] Furthermore, the specific evaluation method of the learner's leadership ability is:

[0033] Construct a learner interaction network model, complete the mapping between learners and nodes in the network, and apply the partitioning rules to the network partitioning to divide the interaction network into different subspaces; the interaction behaviors between learners will be projected into the edges between nodes in the graph, and different types of interaction behaviors will be assigned different weights, and finally complete the construction of the interaction network;

[0034] The learner interaction network is a weighted undirected graph constructed based on the interaction behaviors between learners. The learner interaction relationship network is defined as follows: S represents the set of learners, A represents the interaction behaviors between learners, and G represents the interaction relationship network between learners, where V represents the set of points in the graph, E represents the set of edges in the graph, and each learner S i Corresponding to the node V in the point set V in the graph i , S i =(personalityType,groupRole,Attr), where Attr is the learner S i The remaining properties of

[0035] After building the learner interaction network, the leadership ability of the learner is quantified. The quantitative result of leadership ability is called the learner's leadership ability index. The Katz centrality concept is used as a quantitative method for the learner's leadership ability. i The Katz centrality meter formula is as follows:

[0036]

[0037] Among them, α represents the attenuation factor, A represents the adjacency matrix of graph G, and β represents the initial centrality.

[0038] Katz centrality calculates the relative influence of the current node by evaluating the current node's direct neighbor nodes and other nodes related to the direct neighbor nodes. When calculating the remote nodes, the attenuation coefficient is calculated as follows:

[0039] α i =α d

[0040] Among them, α i represents the attenuation coefficient of the current node, and d represents the distance between two nodes;

[0041] Therefore, the above Centrality(x i ) to obtain the leadership ability index of the learner, that is, to obtain the leadership ability assessment result;

[0042] At the same time, through the leadership ability assessment results Centrality (x i ) and learners’ learning behavior data i By comparing the correlation between learners' different behavioral performances in the learning process and the leadership ability assessment results Lead, the effectiveness of the model is verified.

[0043] Furthermore, the evaluation of the collaboration capability is specifically as follows:

[0044] First, define the basic behaviors in the learning process, evaluate the roles played by the learners through these behaviors, and finally complete the evaluation of the learners' collaborative ability CA through the proportion of different roles of the learners;

[0045] The roles played by learners are determined by analyzing the behavioral patterns of learners in the learning process: drifter, follower, regulator, contributor and instructor. Assume that the behavioral pattern vector of the learner is B = (b1, b2, ..., b n ), then the learner role type is calculated as follows:

[0046] groupRole=Evaluation(B)

[0047] Let M = (m1, m2, ..., m n ) represents the learner’s behavior pattern sequence, m i represents the total number of times the learner performs the i-th behavior in the behavior pattern. R = (r1, r2, ..., r n ) represents the prediction result of the learner role, r i represents the probability that the learner plays role i. The calculation method of the prediction result R of the naive Bayes classifier is as follows:

[0048]

[0049] The participants' collaboration ability was evaluated through the above methods.

[0050] Furthermore, the learner task contribution evaluation uses the Bert model to implement Key-Bert-based text task keyword extraction. The core information of the answer submitted by the current learner can be analyzed by extracting keywords from the text task, and plagiarism can be detected by combining the SimHash algorithm. The performance quality of the learner in a specific task can be analyzed to quantify the task contribution index.

[0051] Furthermore, the weight setting in step S4 is based on the model in the form of:

[0052]

[0053] Among them, w1,...,w n represents the weight value of the learning effectiveness evaluation indicator, x1,…,x n represents the evaluation result of the corresponding indicator, and α represents the coefficient reflecting the outstanding impact degree of the corresponding indicator;

[0054] When α = 1, the above model is the same as the linear weighted model;

[0055] The weight vector W for calculating the evaluation index is W=(w1,...,w n ), satisfying w i ≥0 and Learning effectiveness evaluation index vector X = (x1,...,x n ), where x i ∈[0,Max], Max represents the maximum value of the indicator value range.

[0056] According to some embodiments, an online learning intelligent evaluation system based on a large model includes:

[0057] An online learning effectiveness evaluation mode and indicator definition module is configured to evaluate the learner's online learning effectiveness mode and indicator definition in a hybrid intelligent mode;

[0058] The online learning effectiveness evaluation module is configured to improve the evaluation accuracy through the self-evaluation and mutual evaluation model based on the learner confidence weight, actively dispatch resources through the learner leadership ability evaluation model based on social network analysis to improve the overall learning effectiveness, evaluate the learner's collaborative ability through the evaluation model based on collaborative ability, and complete the evaluation of task contribution through the learner task contribution evaluation method;

[0059] The evaluation of the online learning effectiveness evaluation index system is configured to achieve intelligent evaluation based on the evaluation index weight establishment method and the evaluation index weighted evaluation method.

[0060] According to some embodiments, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned large-model-based online learning intelligent evaluation method.

[0061] According to some embodiments, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of an online learning intelligent evaluation method based on a large model as described in the first aspect above are implemented.

[0062] Compared with the prior art, the present invention has the following advantages:

[0063] The present invention adopts an intelligent evaluation method, calculates the evaluation confidence weight of the learner through various learning data in the learning process of the learner, and introduces a learner evaluation weight adjustment strategy to improve the reliability of the learner self-evaluation and mutual evaluation model results. On the basis of establishing learning effectiveness evaluation indicators, an intelligent evaluation model corresponding to the evaluation indicators is designed and implemented to complete the task of intelligent evaluation of learners' learning effectiveness. Finally, the teacher's evaluation results of the learner are used as the standard for model improvement to improve the accuracy of the intelligent evaluation results, and finally achieve the purpose of stimulating learners' learning enthusiasm and learning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0065] Figure 1 is a flow chart of intelligent evaluation in an embodiment of the present invention;

[0066] Figure 2 This is the workflow of the Key-Bert model in an embodiment of the present invention; DETAILED DESCRIPTION

[0067] The technical scheme in this embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in this embodiment of the present invention. Obviously, the described embodiment is one embodiment of the present invention, not all embodiments of the present invention. Based on this embodiment of the present invention, all other embodiments of the present invention obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as described in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0069] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features.

[0070] Embodiments of the present invention provide a method, system, medium and device for supporting online learning intelligent evaluation based on a large model.

[0071] Example 1

[0072] like Figure 1 and Figure 2 As shown, a method for online learning intelligent evaluation based on a large model for supporting learners, the present invention provides a method for online learning intelligent evaluation based on a large model, comprising the following steps:

[0073] S1. Obtain online learning data of learners, extract corresponding learning features from the above-obtained data, and divide the learning features into quantifiable evaluation features and non-quantitative evaluation features;

[0074] S2. Based on the quantifiable evaluation features, directly calculate and obtain the corresponding quantifiable evaluation indicators;

[0075] S3. Analyze and calculate the non-quantitative evaluation features to obtain non-quantitative evaluation indicators that reflect the learners' learning effectiveness;

[0076] S4. The obtained scores of each non-quantitative evaluation indicator are used as input, and different weights are assigned to each indicator to form a total weighted evaluation score, thereby completing the intelligent evaluation of the learner's learning effectiveness;

[0077] S5. Teachers can choose whether to participate in the learning assessment process. When teachers participate in the assessment, their assessment results have a higher confidence weight. Ultimately, the teacher's assessment, student learning data and intelligent assessment are integrated to calculate the learner's final learning outcomes, and the learner's learning weight is updated based on the assessment results.

[0078] The online learning data in step S1 may include:

[0079] 1. Establish a learner self-assessment and mutual assessment model: First, analyze the learner's historical learning data and calculate the learner's self-assessment and mutual assessment confidence weights based on this data. Learners obtain different historical confidence weight values ​​according to their own historical learning performance. The evaluation made by the learner with a higher historical confidence weight has a higher credibility. The reliability of the self-assessment and mutual assessment model results is improved through differentiated weight allocation;

[0080] 2. Teachers set corresponding collaborative learning tasks according to teaching needs, and learners can conduct learning discussions, divide tasks, and submit learning outcomes during the learning process.

[0081] 3. Learners use self-assessment and mutual assessment models to participate in assessment during learning. When a learner participates in learning for the first time, the learner’s historical learning behavior data is needed to calculate the learner’s learning weight value during the learning process; for learners who have already participated in learning, the individual learning weight value is read from the learner weight data.

[0082] In this embodiment, the quantifiable assessment features involved in step S1 include learner personality traits, learner task contribution, learner activity participation rate, learner task participation rate, and self-evaluation and mutual evaluation participation rate, which can be fully utilized by the machine to process by taking advantage of its fast computing speed and high accuracy of results.

[0083] The above-mentioned method for determining the learner's personality characteristics is as follows:

[0084] personalityType=argmax(neu,ext,opn,arg,con)

[0085] Among them, neu represents the probability that the learner has a neurotic personality, ext represents the probability that the learner has an extroverted personality, opn represents the probability that the learner has an open personality, arg represents the probability that the learner has an agreeable personality, and con represents the probability that the learner has a conscientious personality. Assessing the personality characteristics of learners can be used to recommend learning resources and learning paths that are suitable for their personality in the future, thereby improving learning motivation and effectiveness.

[0086] The learner's task contribution is calculated as follows:

[0087]

[0088] Among them, Workload i Represents the learner Student i The amount of work completed in the task, The learner's performance in the task will affect the evaluation results of the learner's task contribution.

[0089] Assume that the teacher releases n different learning tasks during the learning process, and the evaluation result of the learner's contribution to each task is taskScore i , then the sum and average of each task contribution evaluation result is used to represent the overall task contribution TR of the learner in a single task. The calculation method is as follows:

[0090]

[0091] The learner activity participation rate APR is calculated as follows:

[0092]

[0093] Among them, NA is the number of activities in which the learner participates, and MA is the number of activities published during the learning process.

[0094] The learner task participation rate TPR is calculated as follows:

[0095]

[0096] Among them, NT is the number of tasks in which the learner participates, and MT is the number of tasks issued during the learning process.

[0097] Assuming there are n learners, the vector representation of learners' self-evaluation and mutual evaluation is EVA = (self, g1, g2, ..., g n ), where self is a Boolean value indicating whether to participate in self-evaluation, g i For the evaluation of other members, the self-evaluation participation rate (EPR) is calculated as follows:

[0098]

[0099] The non-quantitative evaluation features involved in step S1 are relatively abstract features that cannot be directly evaluated quantitatively. Therefore, it is necessary to combine the research theoretical results in the field of education to extract features that can reflect the learners' learning outcomes for research and design effective intelligent evaluation models. Intelligent evaluation uses a large model to complete the extraction and evaluation of features related to learners' learning outcomes. The above model processes various data collected in the teaching platform and selects data samples with complete data as data sets for model training.

[0100] At present, the learning effectiveness of learners usually focuses more on process evaluations such as whether learners can put forward new ideas in the learning process, whether members can reach a consensus, and whether different learners can actively communicate. Therefore, the non-quantitative evaluation features of this embodiment include learner leadership, learner collaboration, learner's task contribution in all tasks, learner's learning attitude, and learner's concentration. After extracting the above five evaluation features and converting them into intuitive non-quantitative evaluation indicators through data model calculation, the learning effectiveness of learners can be more accurately reflected. The specific evaluation method is as follows:

[0101] Define the learner learning effectiveness (Collaborative Learning Effectiveness) as a five-tuple:

[0102] CLE=<Lead,CA,LA,TC,SC> ;

[0103] The non-quantitative evaluation indicators included in the above five-tuple are:

[0104] Lead is the leadership ability of the learner, CA is the collaboration ability of the learner, and the leadership ability and collaboration ability are mainly evaluated around the interactive behavior of the learner in the learning process; TC is the task contribution of the learner in all tasks, and the task contribution of the learner mainly solves the problem of the individual contribution evaluation of the learner in the process; LA is the learning attitude of the learner; SC is the concentration of the learner, and the learning attitude and concentration of the learner are mainly evaluated around the learner's own learning performance in the process;

[0105] The learners’ learning outcomes are evaluated through the above five-tuple.

[0106] Assessment of learning attitude LA and concentration SC

[0107] Specifically, this embodiment reflects the influence of learning attitude LA and concentration level SC through self-evaluation, mutual evaluation and attitude evaluation. First, a self-evaluation and mutual evaluation model is constructed. By collecting the evaluation made by the learner on his or her own and other learners' behavior performance, the confidence weight value of the learner's current evaluation index and the evaluation result are calculated. Specifically, the self-evaluation and mutual evaluation model is mainly calculated by the linear weighting method, and linear weighting is performed according to the confidence weight value of each evaluation. Finally, multiple objectives can be expressed as follows:

[0108]

[0109] Among them, X is the final evaluation result, w i and x i Respectively represent the confidence weight value of the current evaluation indicator and the evaluation result of the current evaluation indicator,

[0110] In order to improve the accuracy of self-evaluation and mutual evaluation results, learners with good learning behavior in the past are given higher historical confidence weight values, and learners with poor learning behavior are given lower historical confidence weight values. i The final mutual evaluation score (i.e. evaluation result) in the mutual evaluation stage is taskScore i =x i The calculation formula is as follows:

[0111]

[0112] in, is the weight of learners’ mutual evaluation, E j is the initial mutual evaluation score, let L = (l1,l2,...,l n ) represents the candidate set of the learner’s confidence weight results, l res represents the final prediction result of the current learner confidence weight, l res The calculation is as follows:

[0113] lres =Classifier(B)

[0114] Among them, B is the learner’s learning behavior, Classifier(·) is the model classifier;

[0115] For the evaluation of interaction attitude, assuming that the initial value of the learner's daily interaction attitude is initVal, the weight value weight is calculated as follows:

[0116]

[0117] Among them, count represents the number of times the learner currently participates in learning, and α represents the weight adjusted with the number of discussions;

[0118] If the learner does not participate in the learning discussion of the day, the evaluation result of the learner's learning attitude on that day is the initial value; when the learner participates in the learning discussion process, the learner's discussion data is collected and stored, and the CLIP large model is used to extract the learner's attitude tag sequence from the data, which is recorded as TagSeq = (tag1, tag2, ..., tag n ), the learner's positive and negative interaction attitude adjustment values ​​are posVal and negVal respectively, and the final learner's learning attitude AR is calculated as follows:

[0119]

[0120] The sequence of learner interaction attitude evaluation results is AttSeq = (att1, ..., att n ), take the mean value Avg of the attitude sequence Att As the final evaluation result of learners' interactive attitude.

[0121] Assessment of learners' leadership abilities

[0122] Construct a learner interaction network model, complete the mapping between learners and nodes in the network, and apply the partitioning rules to the network partitioning to divide the interaction network into different subspaces; the interaction behaviors between learners will be projected into the edges between nodes in the graph, and different types of interaction behaviors will be assigned different weights, and finally complete the construction of the interaction network;

[0123] The learner interaction network is a weighted undirected graph constructed based on the interaction behaviors between learners. The learner interaction relationship network is defined as follows: S represents the set of learners, A represents the interaction behaviors between learners, and G represents the interaction relationship network between learners, where V represents the set of points in the graph, E represents the set of edges in the graph, and each learner S i Corresponding to the node V in the point set V in the graph i , S i=(personalityType,groupRole,Attr), where Attr is the learner S i The remaining properties of

[0124] After building the learner interaction network, the leadership ability of the learner is quantified. The quantitative result of leadership ability is called the learner's leadership ability index. The Katz centrality concept is used as a quantitative method for the learner's leadership ability. i The Katz centrality meter formula is as follows:

[0125]

[0126] Among them, α represents the attenuation factor, A represents the adjacency matrix of graph G, and β represents the initial centrality.

[0127] Katz centrality calculates the relative influence of the current node by evaluating the current node's direct neighbor nodes and other nodes related to the direct neighbor nodes. When calculating the remote nodes, the attenuation coefficient is calculated as follows:

[0128] α i =α d

[0129] Among them, α i represents the attenuation coefficient of the current node, and d represents the distance between two nodes;

[0130] Therefore, the above Centrality(x i ) to obtain the leadership ability index of the learner, that is, to obtain the leadership ability assessment result;

[0131] At the same time, through the leadership ability assessment results Centrality (x i ) and learners’ learning behavior data i By comparison, the two are positively correlated. Therefore, the effectiveness of the model can be verified by comparing the relationship between learners' different behavioral performances in the learning process and the leadership ability assessment results Lead.

[0132] Collaboration skills assessment

[0133] First, define the basic behaviors in the learning process, evaluate the roles played by the learners through these behaviors, and finally complete the evaluation of the learners' collaborative ability CA through the proportion of different roles of the learners;

[0134] The roles played by learners are determined by analyzing the behavioral patterns of learners in the learning process: drifter, follower, regulator, contributor and instructor. Assume that the behavioral pattern vector of the learner is B = (b1, b2, ..., bn ), then the learner role type is calculated as follows:

[0135] group Role=Evaluation(B)

[0136] Let M = (m1, m2, ..., m n ) represents the learner’s behavior pattern sequence, m i represents the total number of times the learner performs the i-th behavior in the behavior pattern. R = (r1, r2, ..., r n ) represents the prediction result of the learner role, r i represents the probability that the learner plays role i. The calculation method of the prediction result R of the naive Bayes classifier is as follows:

[0137]

[0138] The participants' collaboration ability was evaluated through the above methods.

[0139] Learner task contribution evaluation

[0140] like Figure 2 As shown in the figure, in order to use the Bert model to implement Key-Bert-based text task keyword extraction, the core information of the current learner's submitted answer can be analyzed by extracting keywords from the text task, combined with the SimHash algorithm to detect plagiarism, and analyze the learner's performance quality in a specific task to quantify the task contribution index, which is specifically:

[0141] First, load the result text that needs to be extracted for keywords, remove the stop words in the task text, and extract the keyword candidate set in the task text. Then embed the keyword candidate set and the task text into word vectors through Bert to complete the mapping between text content and vectors. The model finally extracts the candidate words that are closest to the document as the output result. The cosine similarity between vectors A and B will be used as an indicator to measure whether the two texts are similar. The calculation method of cosine similarity is as follows:

[0142]

[0143] Finally, the N candidate words with the highest similarity to the task text are selected as the output results of the model.

[0144] In addition to analyzing the quality of text-based learning outcomes, it is also necessary to be able to determine the plagiarism phenomenon in the text data. For the text data submitted by learners, the SimHash algorithm is used to first segment the data, extract the feature vectors in the words, and assign different weights to each feature vector. Then the hash value of each feature vector is calculated through the hash function. After completing the hash mapping, the hash value Hash is weighted as follows:

[0145] W=λ×Hash

[0146] The weighted data signature W is a string of digital sequences. The signature is processed by dimensionality reduction to form the SimHash signature of the data. The similarity between two different texts is calculated by comparing the Hamming distance represented by the binary bits of the two digital signatures. The Hamming distance is calculated as follows:

[0147]

[0148] Where dist represents the Hamming distance between two binary sequences, and codeLength represents the binary length of the signature. When S(dist) is less than 32, it is determined that there is plagiarism between the two texts. For high similarity caused by the report template, a corresponding filter word list can be constructed to eliminate the impact of the template content on the evaluation results.

[0149] The Key-Bert model is used to extract task keywords, combined with the SimHash algorithm to detect plagiarism, and the performance quality of learners in specific tasks is analyzed to quantify task contribution indicators.

[0150] The weight setting in step S4 is based on the model in the form of:

[0151]

[0152] Among them, w1,...,w n represents the weight value of the learning effectiveness evaluation indicator, x1,…,x n represents the evaluation result of the corresponding indicator, and α represents the coefficient reflecting the outstanding impact degree of the corresponding indicator;

[0153] When α = 1, the above model is the same as the linear weighted model;

[0154] The weight vector W for calculating the evaluation index is W=(w1,...,w n ), satisfying w i ≥0 and Learning effectiveness evaluation index vector X = (x1,...,x n ), where x i∈[0,Max], Max represents the maximum value of the indicator value range.

[0155] In step S5, teachers' participation in online learning assessment mainly includes two ways: evaluating learners' learning performance and adjusting learners' confidence weights. When teachers actively participate in learners' learning performance assessment, teachers' evaluations will be given higher confidence weights and participate in the calculation process of the final assessment results. In addition, teachers can also participate in the adjustment process of learners' confidence weights. Teachers can view the confidence weights of each member at the moment and can manually adjust the learners' confidence weights to make the results meet the teachers' expectations. At the same time, teachers' adjustments to confidence weights will also be recorded to form a high-quality training set of confidence weight calculation models.

[0156] Example 2

[0157] The embodiment provides an online learning intelligent evaluation system based on a large model, including:

[0158] An online learning effectiveness evaluation mode and indicator definition module is configured to evaluate the learner's online learning effectiveness mode and indicator definition in a hybrid intelligent mode;

[0159] The online learning effectiveness evaluation module is configured to improve the evaluation accuracy through the self-evaluation and mutual evaluation model based on the learner confidence weight, actively dispatch resources through the learner leadership ability evaluation model based on social network analysis to improve the overall learning effectiveness, evaluate the learner's collaborative ability through the evaluation model based on collaborative ability, and complete the evaluation of task contribution through the learner task contribution evaluation method;

[0160] The evaluation of the online learning effectiveness evaluation index system is configured to achieve intelligent evaluation based on the evaluation index weight establishment method and the evaluation index weighted evaluation method.

[0161] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0162] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0163] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0164] Example 3

[0165] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the online learning intelligent evaluation method based on a large model as described in the first embodiment above are implemented.

[0166] Example 4

[0167] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the large model-based online learning intelligent evaluation method as described in the first embodiment are implemented.

[0168] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0169] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0172] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0173] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for intelligent evaluation of online learning based on a large model, characterized in that: The steps include: S1. Obtain online learning data of learners, extract corresponding learning features from the above-obtained data, and divide the learning features into quantifiable evaluation features and non-quantitative evaluation features; S2. Based on the quantifiable evaluation features, directly calculate and obtain the corresponding quantifiable evaluation indicators; S3. Analyze and calculate the non-quantitative evaluation features to obtain non-quantitative evaluation indicators that reflect the learners' learning effectiveness; S4. The obtained scores of each non-quantitative evaluation indicator are used as input, and different weights are assigned to each indicator to form a total weighted evaluation score, thereby completing the intelligent evaluation of the learner's learning effectiveness; S5. Teachers can choose whether to participate in the learning assessment process. When teachers participate in the assessment, their assessment results have a higher confidence weight. Ultimately, the teacher's assessment, student learning data and intelligent assessment are integrated to calculate the learner's final learning outcomes, and the learner's learning weight is updated based on the assessment results.

2. According to claim 1, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: In step S1, the learning data includes the initial confidence weight value obtained from historical learning, and during the learning process, the teacher sets corresponding collaborative learning tasks according to teaching needs, and the learners can conduct learning discussions, task division, and submit learning results during the learning process.

3. According to claim 2, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: The learning outcomes of the learners in step S2 are specifically: The learner learning outcomes are defined as a five-tuple: CLE=<Lead,CA,LA,TC,SC> ; The non-quantitative evaluation indicators included in the above five-tuple are: Lead is the leadership ability of the learner, CA is the collaboration ability of the learner, and the leadership ability and collaboration ability are mainly evaluated around the interactive behavior of the learner in the learning process; TC is the task contribution of the learner in all tasks, and the task contribution of the learner mainly solves the problem of the individual contribution evaluation of the learner in the process; LA is the learning attitude of the learner; SC is the concentration of the learner, and the learning attitude and concentration of the learner are mainly evaluated around the learner's own learning performance in the process; The learners’ learning outcomes are evaluated through the above five-tuple.

4. According to claim 3, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: The five-tuple reflects the influence of learning attitude LA and concentration level SC through self-evaluation, mutual evaluation and attitude evaluation, which are as follows: Construct a self-evaluation and mutual evaluation model. By collecting learners' evaluations of their own and other learners' behavior performance, calculate the confidence weight value of the learner's current evaluation index and the evaluation results. Specifically, the self-evaluation and mutual evaluation model is mainly calculated using the linear weighting method. Linear weighting is performed according to the confidence weight value of each evaluation. Finally, multiple goals can be expressed as follows: Among them, X is the final evaluation result, w i and x i Respectively represent the confidence weight value of the current evaluation indicator and the evaluation result of the current evaluation indicator, In order to improve the accuracy of self-evaluation and mutual evaluation results, learners with good learning behavior in the past are given higher historical confidence weight values, and learners with poor learning behavior are given lower historical confidence weight values. i The final mutual evaluation score taskScore in the mutual evaluation stage i =x i The calculation formula is as follows: in, is the weight of learners’ mutual evaluation, E j is the initial mutual evaluation score, let L = (l1,l2,...,l n ) represents the candidate set of the learner’s confidence weight results, l res represents the final prediction result of the current learner confidence weight, l res The calculation is as follows: l res =Classifier(B) Among them, B is the learner’s learning behavior, Classsifer(·) is the model classifier; For the evaluation of interaction attitude, assuming that the initial value of the learner's daily interaction attitude is initVal, the weight value weight is calculated as follows: Among them, count represents the number of times the learner currently participates in learning, and α represents the weight adjusted with the number of discussions; If the learner does not participate in the learning discussion of the day, the evaluation result of the learner's learning attitude on that day is the initial value; when the learner participates in the learning discussion process, the learner's discussion data is collected and stored, and the CLIP large model is used to extract the learner's attitude tag sequence from the data, which is recorded as TagSeq = (tag1, tag2, ..., tag n ), the learner's positive and negative interaction attitude adjustment values ​​are posVal and negVal respectively, and the final learner's learning attitude AR is calculated as follows: The sequence of learner interaction attitude evaluation results is AttSeq = (att1, ..., att n ), take the mean value Avg of the attitude sequence Att As the final evaluation result of learners' interactive attitude.

5. According to claim 3, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: The specific assessment method of the learner's leadership ability is: Construct a learner interaction network model, complete the mapping between learners and nodes in the network, and apply the partitioning rules to the network partitioning to divide the interaction network into different subspaces; the interaction behaviors between learners will be projected into the edges between nodes in the graph, and different types of interaction behaviors will be assigned different weights, and finally complete the construction of the interaction network; The learner interaction network is a weighted undirected graph constructed based on the interaction behaviors between learners. The learner interaction relationship network is defined as follows: S represents the set of learners, A represents the interaction behaviors between learners, and G represents the interaction relationship network between learners, where V represents the set of points in the graph, E represents the set of edges in the graph, and each learner S i Corresponding to the node V in the point set V in the graph i , S i =(personalityType,groupRole,Attr), where Attr is the learner S i The remaining properties of After building the learner interaction network, the leadership ability of the learner is quantified. The quantitative result of leadership ability is called the learner's leadership ability index. The Katz centrality concept is used as a quantitative method for the learner's leadership ability. i The Katz centrality meter formula is as follows: Among them, α represents the attenuation factor, A represents the adjacency matrix of graph G, and β represents the initial centrality. Katz centrality calculates the relative influence of the current node by evaluating the current node's direct neighbor nodes and other nodes related to the direct neighbor nodes. When calculating the remote nodes, the attenuation coefficient is calculated as follows: α i ＝α d Among them, α i represents the attenuation coefficient of the current node, and d represents the distance between two nodes; Therefore, the above-mentioned Centrality (V i ) to obtain the leadership ability index of the learner, that is, to obtain the leadership ability assessment result; At the same time, through the leadership ability assessment results Centrality (V i ) and learners’ learning behavior data i By comparing the correlation between learners' different behavioral performances in the learning process and the leadership ability assessment results Lead, the effectiveness of the model is verified.

6. According to claim 2, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: The evaluation of the collaboration capability is specifically as follows: First, define the basic behaviors in the learning process, evaluate the roles played by the learners through these behaviors, and finally complete the evaluation of the learners' collaborative ability CA through the proportion of different roles of the learners; The roles played by learners are determined by analyzing the behavioral patterns of learners in the learning process: drifter, follower, regulator, contributor and instructor. Assume that the behavioral pattern vector of the learner is B = (b1, b2, ..., b n ), then the learner role type is calculated as follows: groupRole=Evaluation(B) Let M = (m1, m2, ..., m n ) represents the learner's behavior pattern sequence, m i represents the total number of times the learner performs the i-th behavior in the behavior pattern. R = (r1, r2, ..., r n ) represents the prediction result of the learner role, r i represents the probability that the learner plays role i. The calculation method of the prediction result R of the naive Bayes classifier is as follows: The participants' collaboration ability was evaluated through the above methods.

7. According to claim 2, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: The learner task contribution evaluation is to use the Bert model to implement Key-Bert-based text task keyword extraction. The core information of the answer submitted by the current learner can be analyzed by extracting keywords from the text task, and the SimHash algorithm is combined to detect plagiarism and analyze the performance quality of the learner in a specific task to quantify the task contribution index.

8. According to claim 1, a method for supporting online learning intelligent evaluation based on a large model is characterized in that: The weight setting in step S4 is based on the model in the form of: Among them, w1,...,w n represents the weight value of the learning effectiveness evaluation indicator, x1,…,x n represents the evaluation result of the corresponding indicator, and α represents the coefficient reflecting the outstanding impact degree of the corresponding indicator; When α = 1, the above model is the same as the linear weighted model; The weight vector W for calculating the evaluation index is W=(w1,...,w n ), satisfying w i ≥0 and Learning effectiveness evaluation index vector X = (x1,...,x n ), where x i ∈[0,Max], Max represents the maximum value of the indicator value range.

9. An online learning intelligent evaluation system based on a large model, comprising: An online learning effectiveness evaluation model and indicator definition module is configured to evaluate the learner's online learning effectiveness model and indicator definition in a hybrid intelligent mode; The online learning effectiveness evaluation module is configured to improve the evaluation accuracy through the self-evaluation and mutual evaluation model based on the learner confidence weight, actively dispatch resources through the learner leadership ability evaluation model based on social network analysis to improve the overall learning effectiveness, evaluate the learner's collaborative ability through the evaluation model based on collaborative ability, and complete the evaluation of task contribution through the learner task contribution evaluation method; The evaluation of the online learning effectiveness evaluation index system is configured to achieve intelligent evaluation based on the evaluation index weight establishment method and the evaluation index weighted evaluation method.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the online learning intelligent evaluation method based on a large model as described in any one of claims 1 to 7 are implemented.

Images