Multimodal unified intelligent learning diagnosis modeling method, system, medium, and terminal

Abstract

The present invention belongs to the technical field of educational big data mining, and discloses a multimodal unified intelligent learning diagnosis modeling method, system, medium, and terminal. By constructing a multi-channel cognitive diagnosis model, a preliminary diagnosis is made on the learner, and the parameters of the learning resources are estimated to obtain a learning resource parameter set and a learner parameter set; the learning resources and the learner are modeled to obtain deep representation features; a self-attention mechanism is introduced to fuse the learner features with the learning resource features; the fused features are used as the data basis for predicting the learner's performance, and a learner performance prediction network is constructed to obtain a predicted value of the learner's correct answer probability; the learner's overall knowledge point mastery is diagnosed through the characteristic information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained. The present invention is conducive to integrating the advantages of the multi-channel cognitive diagnosis model, and a neural network is designed to perform intelligent learning diagnosis on the learner, and it has scalability.

Description

Technical Field

[0001] The present invention belongs to the technical field of educational big data mining, and in particular relates to a multimodal unified intelligent learning diagnosis modeling method, system, medium, and terminal. Background Art

[0002] In recent years, with the rapid development of information technology such as artificial intelligence, the education industry has also been constantly leaping towards intelligence and informatization. The integration of modern information technology and traditional teaching models and education methods has formed a new service format of "Internet + Education". Internet education can realize new teaching methods such as personalized guidance and intelligent learning evaluation, provide teachers and learners with accurate and effective guidance information, and improve teaching and learning effects.

[0003] Cognitive diagnostic evaluation is an auxiliary method for realizing smart education. Specifically, it analyzes learners' performance in specific learning tasks to explore and diagnose learners' knowledge mastery status and cognitive level, so as to provide learners with targeted improvement suggestions. At present, relevant researchers have proposed a variety of cognitive diagnostic models from different perspectives, which can be divided into two categories according to their different theoretical foundations:

[0004] (1) Cognitive diagnosis method based on parameter estimation;

[0005] (2) Based on non-parametric artificial intelligence methods.

[0006] The cognitive diagnosis methods based on parameter estimation are mainly represented by latent trait models and latent classification models. The latent trait model is based on item response theory and assumes that learners have a "latent trait", which generally refers to continuous potential ability. The model believes that learners' reactions and scores on exercise questions are not only related to the factors of the questions themselves, but also have a special relationship with the learners' latent traits. The learners' knowledge status is diagnosed by modeling the relationship between the learners' latent traits, question factors and answer responses; the latent classification model assumes that the ability value is discontinuous, and the latent knowledge space consists of K binary variables, with a total of 2 K The latent classification model sometimes cannot fully fit the actual situation, and there are often some learners who cannot be accurately classified into a limited number of hypothetical categories.

[0007] The non-parametric artificial intelligence method introduces deep learning methods, which are based on practice questions and learners' answer records, and model the learner's learning process and knowledge mastery status through recurrent neural networks, convolutional networks, etc., with high accuracy. However, due to the black box characteristics of deep learning methods, non-parametric artificial intelligence methods are usually not very interpretable, and the results obtained are sometimes not completely convincing.

[0008] Through the above analysis, the problems and defects of the prior art are as follows:

[0009] (1) The modeling angles and parameters of traditional cognitive diagnosis models are relatively single, and they are unable to use different methods to comprehensively consider multiple dimensions, resulting in one-sided and inaccurate diagnostic results.

[0010] (2) Traditional cognitive diagnosis models have strict requirements on data scale. If the scale is small, the accuracy of the diagnostic results will be low, and if the scale is too large, the efficiency will be greatly reduced.

[0011] (3) Existing non-parametric artificial intelligence diagnostic methods are black box methods, which cannot give their results good educational interpretation significance and their usability is limited to a certain extent.

[0012] The difficulty of solving the above problems and defects is:

[0013] (1) How to flexibly integrate the ideas of multiple traditional cognitive diagnostic models to build a more inclusive diagnostic framework, thereby improving the accuracy of diagnostic results.

[0014] (2) How to utilize the high-dimensional spatial representation capabilities of deep learning methods to fully mine the available information in educational data, model learners and learning resources, and achieve accurate learner learning diagnosis.

[0015] (3) How to organically integrate deep learning methods with traditional statistical-based cognitive diagnosis methods, taking advantage of their respective strengths, while improving the accuracy of diagnostic results and taking into account their explanatory significance in the field of education.

[0016] The significance of solving the above problems and defects is:

[0017] (1) The present invention fully taps into the potential information in learners’ learning data, and can conduct in-depth and comprehensive cognitive diagnosis and performance prediction of learners, providing reference and guidance information for their subsequent learning.

[0018] (2) The present invention can provide a flexible diagnostic method, which can select different diagnostic channels according to different needs and is applicable to a variety of scenarios.

[0019] (3) The multimodal unified intelligent learning diagnosis modeling method proposed in the present invention uses a deep learning method to model learners and learning resources based on the traditional cognitive diagnosis model, thereby improving the accuracy of diagnosis results and the efficiency of diagnosis. Summary of the invention

[0020] In view of the problems existing in the prior art, the present invention provides a multi-modal unified intelligent learning diagnosis modeling method, system, medium and terminal.

[0021] The present invention is implemented as follows: a multi-modal unified intelligent learning diagnosis modeling method, the multi-modal unified intelligent learning diagnosis modeling method comprising the following steps:

[0022] Step 1: Based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed to form an extensible diagnostic framework; within this framework, a preliminary diagnosis is performed on the learner, and the parameters of the learning resources are estimated to obtain the learning resource parameter set and the learner parameter set;

[0023] Step 2: Combine the deep autoencoder to construct the learner feature representation network and the learning resource feature representation network, model the learning resource and learner respectively based on the original parameter set, and obtain the deep representation features;

[0024] Step 3: Introduce the self-attention mechanism to integrate learner features with learning resource features, explore the correlation and importance information between features of each dimension, assign different weights according to different importance, and pay more attention to the most effective features;

[0025] Step 4: Use the fused features as the data basis for predicting learner performance, build a learner performance prediction network to get the predicted value of the probability of the learner answering correctly; diagnose the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtain the parameter representation of the exercise questions.

[0026] Furthermore, in step 1, a multi-channel cognitive diagnosis model is constructed based on the learner's historical learning records and learning resource information to form an extensible diagnosis framework; within this framework, a preliminary diagnosis is performed on the learner, and parameters of the learning resources are estimated to obtain a learning resource parameter set and a learner parameter set, including:

[0027] (1) Construct a collection of learning resources and a collection of learners’ historical learning data:

[0028] S={s1,s2,...,s N}

[0029] E={e1,e2,...,e M}

[0030] K={k1,k2,...,k L};

[0031] Among them, S is the set of learners, E is the set of exercise questions, and K is the set of knowledge points.

[0032] (2) Use the answer matrix R to record the students' historical answer results. Each row represents a student and each column represents an exercise. If student s answers exercise e correctly, then R se =1, otherwise R se = 0; use the exercise-knowledge point matrix Q to represent the relationship between the exercises and the knowledge points they examine, with each row representing a question and each column representing a knowledge point; if exercise e examines knowledge point k, then Q ek =1, otherwise Q ek =0.

[0033] (3) Construct a multi-channel cognitive diagnosis model, use the Q matrix and R matrix to estimate the learner and learning resource parameters, obtain the original parameter representation information, and construct the learner and learning resource parameter set based on it;

[0034] The parameters include: the learner's mastery of knowledge points, overall ability level parameters, difficulty, discrimination, guessing rate and error rate of the exercise questions.

[0035] The learner parameter set and learning resource parameter set are defined as:

[0036] STUDENT={SF1,SF2,...,SF n}

[0037] EXERCISE={EF1,EF2,...,EF m};

[0038] Among them, SF i Represents a characteristic parameter of the learner, EF j Represents a characteristic parameter of a learning resource.

[0039] Further, in step 2, the deep autoencoder is combined to construct a learner feature representation network and a learning resource feature representation network, and the learning resource and the learner are modeled based on the original parameter set, and deep representation features are obtained, including:

[0040] (1) Preprocess the learner and learning resource parameters. First, all parameters are segmented and mapped into several discrete values ​​according to their size. Then, they are encoded using the One-Hot method to generate learner and learning resource parameter vectors:

[0041] student F=(SF1,SF2,...,SF n )

[0042] exercise F =(EF1,EF2,...,EF m );

[0043] Among them, SF i EF is the feature vector obtained by One-Hot encoding a parameter of the learner; j It is a feature vector obtained by One-Hot encoding a parameter of the learning resource.

[0044] (2) Design a learner feature deep autoencoder, which consists of an encoder and a decoder. The encoder encodes high-dimensional input into low-dimensional latent variables to learn the most informative features; the decoder restores the latent vector of the hidden layer to the initial dimension. The optimal state is that the output of the decoder can be completely or approximately restored to the original input, so that the latent vector represents the input information and achieves the effect of dimensionality reduction.

[0045] (3) Design a learning resource feature deep autoencoder, the composition structure of which is similar to the learner feature deep autoencoder.

[0046] Furthermore, in step (2), the step of obtaining a latent feature vector using a deep autoencoder includes:

[0047] 2.1) Construct an encoder to obtain the hidden learner feature vector:

[0048] student H1 =g(W s1 ×student F +b s1 )

[0049] student H2 =g(W s2 ×student H1 +b s2 );

[0050] Among them, g is the tanh activation function, W s1 , W s2 is the encoding layer node weight parameter, b s1 , b s2 is the corresponding bias parameter;

[0051] 2.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features:

[0052] student D1 =g(Ws3 ×student H2 +b s3 )

[0053] student D2 =g(W s4 ×student D1 +b s4 );

[0054] Among them, g is the tanh activation function, W s1 , W s2 is the node weight parameter of the decoding layer, b s1 , b s2 is the corresponding bias parameter;

[0055] 2.3) Extract the hidden layer feature vector student H2 As a learner deep representation feature.

[0056] In step (3), the design of the learning resource feature deep autoencoder includes:

[0057] 3.1) Construct an encoder to obtain the hidden learner feature vector:

[0058] exercise H1 =g(W e1 ×exercise F +b e1 )

[0059] exercise H2 =g(W e2 ×exercise H1 +b e2 );

[0060] Among them, g is the tanh activation function, W e1 , W e2 is the encoding layer node weight parameter, b e1 , b e2 is the corresponding bias parameter;

[0061] 3.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features:

[0062] exercise D1 =g(W e3 ×exercise H2 +b e3 )

[0063] exercise D2 =g(W e4 ×exerciseD1 +b e4 );

[0064] Among them, g is the tanh activation function, W e1 , W e2 is the node weight parameter of the decoding layer, b e1 , b e2 is the corresponding bias parameter;

[0065] 3.3) Extract the feature vector of the hidden layer H2 As a learning resource, deep representation features.

[0066] Furthermore, in step 3, the self-attention mechanism is introduced to fuse learner features with learning resource features, to mine the correlation and importance information between features of each dimension, to give different weights according to different importance, and to pay more attention to the most effective features, including:

[0067] (1) Features from two dimensions: learners and exercises:

[0068] F = {student D2 ,exercise D};

[0069] (2) effectively integrating learner and exercise question features by using a self-attention mechanism; wherein said integrating features by using a self-attention mechanism includes:

[0070] 1) Input the concatenated features into the convolution layer with a convolution kernel size of 1 to obtain the query vector matrix Query, the key vector matrix Key, and the value vector matrix Value:

[0071] Query = Conv(F)

[0072] Key=Conv(F)

[0073] Value = Conv(F);

[0074] 2) The weight of each data in the feature is calculated by calculating the dot product between the query vector matrix and the key vector matrix. The weight represents the correlation between the task to be queried and each input data:

[0075] similarity(Query,Key i )=Query·Key i ;

[0076] 3) SoftMax is introduced to convert the correlation into numerical values; the original calculated scores are normalized into a probability distribution with a weight sum of 1; the weights of important elements are highlighted through the internal mechanism of SoftMax:

[0077]

[0078] 4) Perform weighted summation on the input data to obtain the fused feature data:

[0079]

[0080] Further, in step 4, the fusion features are used as the data basis for predicting the learner's performance, and a learner performance prediction network is constructed to obtain a predicted value of the learner's correct answer probability; the learner's overall knowledge mastery is diagnosed through the feature information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained, including:

[0081] (1) Construct a convolutional network layer to predict the learner's answer results. Input the fused feature vector into the convolutional layer to extract its spatial information. The convolution kernel size is set to 3 and the step size is 1:

[0082] F c =Conv(F a );

[0083] (2) Add the Relu activation function to reduce the dependency between parameters and avoid overfitting:

[0084] F re =Relu(F c );

[0085] (3) Reduce feature dimensions through the maximum pooling layer:

[0086] F p =MaxPool(F re );

[0087] (4) Input the reduced features into the fully connected layer to predict the learner’s answer results for a specific learning task:

[0088] p=σ(W3×F p +b3);

[0089] Among them, W3 and b3 represent the parameters to be learned, W3 is the weight parameter, b3 is the corresponding bias parameter; σ is sigmoid as the activation function.

[0090] (5) The evaluation method for learners’ knowledge mastery is as follows:

[0091] student knowledge-proficiency =σ(W4×studentH +b4);

[0092] student H is the high-dimensional learner feature vector, W4, b4 represents the weight and bias parameters to be learned, and student knowledge-proficiency It is the evaluation result of learners’ knowledge mastery.

[0093] (6) The result of the exercise parameter representation is the vector exercise.

[0094] Another object of the present invention is to provide a multimodal unified intelligent learning diagnostic modeling system using the multimodal unified intelligent learning diagnostic modeling method, wherein the multimodal unified intelligent learning diagnostic modeling system comprises:

[0095] The parameter set construction module is used to construct a multi-channel cognitive diagnosis model based on the learner's historical learning records and learning resource information to form an extensible diagnosis framework; within this framework, the learner is preliminarily diagnosed and the learning resource parameters are estimated to obtain the learning resource parameter set and the learner parameter set;

[0096] A feature representation network construction module is used to combine the deep autoencoder to construct a learner feature representation network and a learning resource feature representation network, model the learning resource and learner respectively based on the original parameter set, and obtain deep representation features;

[0097] The feature fusion module is used to introduce the self-attention mechanism to fuse learner features with learning resource features, explore the correlation and importance information between features of each dimension, assign different weights according to different importance, and pay more attention to the most effective features;

[0098] The learning diagnosis and parameter representation module is used to use the fused features as the data basis for predicting the learner's performance, build a learner performance prediction network to obtain the predicted value of the learner's correct answer probability; diagnose the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtain the parameter representation of the exercise questions.

[0099] Another object of the present invention is to provide a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0100] Based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed to form an extensible diagnostic framework; within this framework, a preliminary diagnosis is made on the learner, and the parameters of the learning resources are estimated to obtain the learning resource parameter set and the learner parameter set; a deep autoencoder is combined to construct a learner feature representation network and a learning resource feature representation network, and the learning resources and learners are modeled based on the original parameter set, and deep representation features are obtained;

[0101] A self-attention mechanism is introduced to fuse learner features with learning resource features, to explore the correlation and importance information between features of each dimension, to assign different weights according to different importance, and to pay more attention to the most effective features; the fused features are used as the data basis for predicting learner performance, and a learner performance prediction network is constructed to obtain the predicted value of the learner's correct answer probability; the learner's overall knowledge mastery is diagnosed through the characteristic information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained to provide a reference for the learner's personalized learning.

[0102] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0103] Based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed to form an extensible diagnostic framework; within this framework, a preliminary diagnosis is made on the learner, and the parameters of the learning resources are estimated to obtain the learning resource parameter set and the learner parameter set; a deep autoencoder is combined to construct a learner feature representation network and a learning resource feature representation network, and the learning resources and learners are modeled based on the original parameter set, and deep representation features are obtained;

[0104] A self-attention mechanism is introduced to fuse learner features with learning resource features, to explore the correlation and importance information between features of each dimension, to assign different weights according to different importance, and to pay more attention to the most effective features; the fused features are used as the data basis for predicting learner performance, and a learner performance prediction network is constructed to obtain the predicted value of the learner's correct answer probability; the learner's overall knowledge mastery is diagnosed through the characteristic information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained to provide a reference for the learner's personalized learning.

[0105] Another object of the present invention is to provide an information data processing terminal, which is used to implement the multi-modal unified intelligent learning diagnosis modeling system.

[0106] Combining all the above technical solutions, the advantages and positive effects of the present invention are as follows:

[0107] The multimodal unified intelligent learning diagnosis modeling method provided by the present invention fully taps into the potential information in the learners' learning data, can conduct in-depth and comprehensive cognitive diagnosis and performance prediction for learners, and provide reference and guidance information for their subsequent learning;

[0108] The present invention is conducive to integrating the advantages of multi-channel cognitive diagnosis models, can formulate flexible diagnosis strategies, model learners and learning resources from different angles, and is applicable to a variety of scenarios;

[0109] The multimodal unified intelligent learning diagnosis modeling method proposed in the present invention uses a deep autoencoder to model learners and learning resources on the basis of the traditional statistical method-based cognitive diagnosis model, and introduces the deep features obtained by self-attention mechanism processing, which can perform learning diagnosis on learners more accurately and efficiently.

[0110] The present invention designs a convolutional neural network to predict learners' answers, extracts effective information from the spatial features of learners and learning resources, and thereby obtains a more accurate prediction value for learners' answers.

[0111] Comparative technical or experimental effects, including:

[0112] The present invention compares a multimodal unified intelligent learning diagnosis modeling method with other learning diagnosis analysis methods, comparing the indicators of area under the curve AUC and root mean square error RMSE. AUC and provide reliable indicators for predicting and evaluating learner performance. An AUC value of 0.5 represents a randomly obtainable score. The higher the AUC score, the more accurate the prediction result. The root mean square error (RMSE) is used to measure the deviation between the estimated value and the true value. The smaller the value, the closer the mining result is to the true value. The calculation formula is:

[0113]

[0114] The present invention compares this method with the traditional learning diagnosis method. In order to make an objective comparison, all methods are adjusted to the best state. The AUC and RMSE comparison results of a multimodal unified intelligent learning diagnosis modeling method and the traditional learning diagnosis method under the MATH data set are shown in Table 1.

[0115] Table 1 Comparison of experimental results

[0116] BRIEF DESCRIPTION OF THE DRAWINGS

[0117] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0118] Figure 1 It is a flow chart of the multimodal unified intelligent learning diagnostic modeling method provided by an embodiment of the present invention.

[0119] Figure 2 It is a schematic diagram of a multi-modal unified intelligent learning diagnostic modeling method provided by an embodiment of the present invention.

[0120] Figure 3 It is a structural block diagram of a multi-modal unified intelligent learning diagnosis modeling system provided by an embodiment of the present invention;

[0121] In the figure: 1. Parameter set construction module; 2. Feature representation network construction module; 3. Feature fusion module; 4. Learning diagnosis and parameter representation module.

[0122] Figure 4 It is a comparison chart of experimental results provided by the embodiments of the present invention. DETAILED DESCRIPTION

[0123] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0124] In view of the problems existing in the prior art, the present invention provides a multi-modal unified intelligent learning diagnosis modeling method and system, and the present invention is described in detail below in conjunction with the accompanying drawings.

[0125] like Figure 1 As shown, the multimodal unified intelligent learning diagnosis modeling method provided by the embodiment of the present invention includes the following steps:

[0126] S101, based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed. This embodiment specifically adopts a method of integrating the IRT model and the DINA model to make a preliminary diagnosis of the learner, and estimates the parameters of the learning resources to obtain a learning resource parameter set (specifically including difficulty, discrimination, guessing rate, and error rate) and a learner parameter set (specifically including the degree of mastery of knowledge points and overall ability level);

[0127] S102, combining the deep autoencoder to construct a learner feature representation network and a learning resource feature representation network, modeling the learning resource and the learner respectively based on the original parameter set, and obtaining deep representation features;

[0128] S103, introduce the self-attention mechanism to integrate learner features with learning resource features, explore the correlation and importance information between features of each dimension, give different weights according to different importance, and pay more attention to the most effective features;

[0129] S104, using the fused features as the data basis for predicting the learner's performance, constructing a learner performance prediction network to obtain a predicted value for the probability of the learner answering correctly; diagnosing the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtaining the parameter representation of the exercise questions.

[0130] The schematic diagram of the multi-modal unified intelligent learning diagnosis modeling method provided by the embodiment of the present invention is shown in Figure 2 .

[0131] like Figure 3 As shown, the multi-modal unified intelligent learning diagnosis modeling system provided by the embodiment of the present invention includes:

[0132] Parameter set construction module 1 is used to construct a multi-channel cognitive diagnosis model based on the learner's historical learning records and learning resource information to form an extensible diagnosis framework; within this framework, a preliminary diagnosis is made on the learner, and parameters of the learning resources are estimated to obtain a learning resource parameter set and a learner parameter set;

[0133] Feature representation network construction module 2, used to combine the deep autoencoder to construct a learner feature representation network and a learning resource feature representation network, model the learning resource and the learner respectively based on the original parameter set, and obtain deep representation features;

[0134] Feature fusion module 3 is used to introduce the self-attention mechanism to fuse learner features with learning resource features, explore the correlation and importance information between features of each dimension, give different weights according to different importance, and pay more attention to the most effective features;

[0135] The learning diagnosis and parameter representation module 4 is used to use the fused features as the data basis for predicting the learner's performance, construct a learner performance prediction network to obtain the predicted value of the learner's correct answer probability; diagnose the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtain the parameter representation of the exercise questions.

[0136] The technical solution of the present invention is further described below in conjunction with specific embodiments.

[0137] Example 1

[0138] like Figure 2 As shown, the multimodal unified intelligent learning diagnosis modeling method provided by the embodiment of the present invention includes the following steps:

[0139] (1) Based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed to form an extensible diagnostic framework. Within this framework, a preliminary diagnosis of the learner is performed and the parameters of the learning resources are estimated to obtain the learning resource parameter set and the learner parameter set.

[0140] (2) Combine deep autoencoders to construct learner feature representation networks and learning resource feature representation networks, model learning resources and learners respectively based on the original parameter set, and obtain deep representation features;

[0141] (3) Introduce the self-attention mechanism to integrate learner characteristics with learning resource characteristics, explore the correlation and importance information between characteristics of each dimension, assign different weights according to different importance, and pay more attention to the most effective features;

[0142] (4) The fused features are used as the data basis for predicting learners' performance, and a learner performance prediction network is constructed to obtain the predicted value of the learner's correct answer probability; the learner's overall knowledge mastery is diagnosed through the feature information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained.

[0143] In step (1), the embodiment of the present invention provides a multi-channel cognitive diagnosis model based on the learner's historical learning records and learning resource information to form an extensible diagnosis framework. In this framework, the learner is preliminarily diagnosed and the learning resource parameters are estimated, so that the learning resource parameter set and the learner parameter set include:

[0144] (1.1) Construct a learning resource collection and a learner's historical learning data collection:

[0145] S={s1,s2,...,s N}

[0146] E={e1,e2,...,e M}

[0147] K={k1,k2,...,k L}

[0148] Among them, S is the set of learners, E is the set of exercise questions, and K is the set of knowledge points.

[0149] (1.2) Use the answer matrix R to record the students' historical answer results. Each row represents a student and each column represents an exercise. If student s answers exercise e correctly, then R se =1, otherwise R se = 0; use the exercise-knowledge point matrix Q to represent the relationship between the exercises and the knowledge points they examine. Each row represents a question and each column represents a knowledge point. If exercise e examines knowledge point k, then Q ek =1, otherwise Q ek =0.

[0150] (1.3) Construct a multi-channel cognitive diagnosis model, use the Q matrix and R matrix to estimate the learner and learning resource parameters, obtain the original parameter representation information, and construct the learner and learning resource parameter set based on it. In this embodiment, the DINA and IRT models are used to construct a multi-channel cognitive diagnosis model. As an embodiment of the present invention, in the scalable cognitive diagnosis framework, the fusion IRT model and the Dina model are used as an example, and the specific method of parameter acquisition and network construction is as follows:

[0151] (1.2.1) In the DINA model, the Q matrix and R matrix are used as input to estimate the learner and test parameters. Including: Learner - the mastery vector s for each knowledge point α ; Exercises—Guessing Rate e guess , error rate slip .

[0152] (1.2.2) Each student is described as a knowledge point mastery vector, where each dimension corresponds to a knowledge point. When the student's knowledge point mastery vector is known, for the test questions that the student has not answered, the student's potential answer to the test questions can be obtained according to the following formula:

[0153]

[0154] That is, when students have mastered all the knowledge points required to answer the test questions correctly, η uv =1.

[0155] (1.2.3) Two question parameters, error rate s and guess rate g, are introduced and defined as:

[0156] The student has mastered all the knowledge points required to answer the test questions, but for some reason answered incorrectly. That is, the error parameters are:

[0157] S j =P(R uv =0|η uv =1)

[0158] The student did not master all the knowledge points required to answer the test question, or even did not master any of the knowledge points, but the answer was correct. That is, guessing the parameters:

[0159] g j =P(R uv =1|η uv =0)

[0160] (1.2.4) The probability model of the actual response matrix is:

[0161]

[0162] (1.2.5) Thus, the total likelihood function of the DINA model is obtained:

[0163]

[0164] Where L = 2 V .

[0165] (1.2.6) Introduce the EM algorithm and use the maximum marginal likelihood estimation method to solve:

[0166] Step E: Use the s obtained in the previous round v With g v Estimated value calculation matrix P(R|α)=[P(R u |α l )] U×L The value of , and use P(R|α) to calculate the matrix P(α|R)=[P(α l |R u )] L×U The value of .

[0167] Step M: respectively and Available in represents the expected number of students who lack at least one knowledge point required for question v among the students who belong to the lth knowledge point mastering mode, express The expected number of people who answer question v correctly is and The meaning and and Similar, but different in that and is the expectation under the condition that students master all the knowledge points required for question v. Therefore, we can calculate and The value of , and thus get the new s v With g v Estimated value.

[0168] (1.2.7) In the IRT model, learner and test parameters are estimated based on the R matrix. Including: learner-ability value s θ; Exercises—Difficulty e difficulty , Discrimination discrimination .

[0169] (1.2.8) Let the parameter of question v be δ v , the ability parameter of learner k is π k , and the total question parameter vector is δ, and the ability parameter of all learners is π k The value set of is π, then the learner k's response is r k The conditional probability is:

[0170]

[0171] Among them, P(π k |δ j ) is the probability that learner k answers question j correctly.

[0172] (1.2.9) Thus, the total likelihood function of the IRT model is obtained:

[0173]

[0174] Among them, p(π k |π) is the learner's ability value is π k probability.

[0175] (1.2.10) Introduce the EM algorithm for parameter estimation:

[0176] Step E: Use the δ and π obtained in the previous iteration to calculate the power level to π k Number of people n k and the level is π k The number of people who answered the question correctly jk Expected value

[0177] Step M: Let Available make Solving the equation Available

[0178] (1.2.11) Combining the two types of parameters, learners and learning resources, we get two parameter sets, student and exerics:

[0179] student={s θ ,s α}

[0180] exercise={e difficulty ,e discrimination ,e guess ,e slip}

[0181] In step (2), the embodiment of the present invention provides a method for combining a deep autoencoder to construct a learner feature representation network and a learning resource feature representation network, and models the learning resource and the learner based on the original parameter set, and obtains their deep representation features:

[0182] (2.1) Construct the learner parameter vector and learning resource parameter vector from the learner and learning resource parameter sets:

[0183] student F =(s θ ,s α )

[0184] exercise F =(e difficulty ,e discrimination ,e guess ,e slip )

[0185] (2.2) Preprocess the learner and learning resource parameter vectors. First, all parameters are segmented and the continuous parameter values ​​are mapped to several discrete values ​​according to their size. Then, they are encoded using the One-Hot method to generate 64-dimensional learner and 40-dimensional learning resource parameter vectors:

[0186] student F =(SF1,SF2)

[0187] exercise F =(EF1,EF2,EF3,EF4)

[0188] Among them, SF i EF is the feature vector obtained by One-Hot encoding a parameter of the learner; j It is a feature vector obtained by One-Hot encoding a parameter of the learning resource.

[0189] (2.3) Design a learner feature deep autoencoder, which consists of an encoder and a decoder. The encoder encodes the high-dimensional input into a low-dimensional latent variable to learn the most informative features; the decoder restores the latent vector of the hidden layer to the initial dimension. The best state is that the output of the decoder can be completely or approximately restored to the original input, so that the latent vector can well represent the input information and achieve dimensionality reduction:

[0190] The method of obtaining a latent feature vector by using a deep autoencoder specifically includes:

[0191] (2.3.1) Construct an encoder to obtain the hidden learner feature vector:

[0192] studentH1 =g(W s1 ×student F +b s1 )

[0193] student H2 =g(W s2 ×student H1 +b s2 )

[0194] Among them, g is the tanh activation function, W s1 , W s2 is the encoding layer node weight parameter, b s1 , b s2 is the corresponding bias parameter;

[0195] (2.3.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features:

[0196] student D1 =g(W s3 ×student H2 +b s3 )

[0197] student D2 =g(W s4 ×student D1 +b s4 )

[0198] Among them, g is the tanh activation function, W s1 , W s2 is the node weight parameter of the decoding layer, b s1 , b s2 is the corresponding bias parameter;

[0199] (2.3.3) Extract the 32-dimensional feature vector student of the hidden layer H2 As a learner deep representation feature.

[0200] (2.4) Similarly, we design a deep autoencoder for learning resource features, whose composition structure is similar to that of the learner feature deep autoencoder:

[0201] (2.4.1) Construct an encoder to obtain the hidden layer learner feature vector:

[0202] exercise H1 =g(W e1 ×exercise F +b e1 )

[0203] exerciseH2 =g(W e2 ×exercise H1 +b e2 )

[0204] Among them, g is the tanh activation function, W e1 , W e2 is the encoding layer node weight parameter, b e1 , b e2 is the corresponding bias parameter;

[0205] (2.4.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features:

[0206] exercise D1 =g(W e3 ×exercise H2 +b e3 )

[0207] exercise D2 =g(W e4 ×exercise D1 +b e4 )

[0208] Among them, g is the tanh activation function, W e1 , W e2 is the node weight parameter of the decoding layer, b e1 , b e2 is the corresponding bias parameter;

[0209] (2.4.3) Extract the 32-dimensional feature vector of the hidden layer H2 As a learning resource, deep representation features.

[0210] Further, in S103, the self-attention mechanism provided by the embodiment of the present invention is introduced to effectively integrate the learner features and the learning resource features, fully explore the correlation and importance information between the features of each dimension, give different weights to them according to different importance, and pay more attention to the most effective part of the features, including:

[0211] (3.1) By combining the features of learners and exercises, we get a 64-dimensional overall feature:

[0212] F = {student H2 ,exercise H2}

[0213] (3.2) Combine the self-attention mechanism to effectively integrate the learner and exercise characteristics;

[0214] The specific method of combining the self-attention mechanism to perform feature fusion includes:

[0215] (3.2.1) Input the concatenated features into the convolution layer with a convolution kernel size of 1 to obtain the query vector matrix Query, the key vector matrix Key, and the value vector matrix Value:

[0216] Query = Conv(F)

[0217] Key=Conv(F)

[0218] Value=Conv(F)

[0219] (3.2.2) The weight of each data in the feature is calculated by calculating the dot product between the query vector matrix and the key vector matrix. The weight represents the correlation between the task to be queried and each input data:

[0220] similarity(Query,Key i )=Query·Key i

[0221] (3.2.3) Introducing softMax to convert the correlation numerically can, on the one hand, normalize the original calculated scores and organize them into a probability distribution with a weight sum of 1; on the other hand, the weights of important elements can be more highlighted through the internal mechanism of SoftMax:

[0222]

[0223] (3.2.4) Perform weighted summation on the input data to obtain the fused feature data:

[0224]

[0225] In step (4), the embodiment of the present invention provides a method of using the fusion features as a data basis for predicting the learner's performance, constructing a learner performance prediction network to obtain a predicted value of the learner's correct answer probability; diagnosing the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtaining the parameter representation of the exercise questions, providing a reference for the learner's personalized learning, including:

[0226] (4.1) Construct a convolutional network layer to predict the learner's answer results. Input the fused feature vector into the convolutional layer to extract its spatial information. The convolution kernel size is set to 3 and the step size is 1:

[0227] F c =Conv(F a )

[0228] (4.2) Add Relu activation function to reduce the dependency between parameters and avoid overfitting:

[0229] F re =Relu(F c )

[0230] (4.3) Reduce feature dimensions through the maximum pooling layer:

[0231] F p =MaxPool(F re )

[0232] (4.4) Input the reduced-dimensional features into the fully connected layer to predict the learner’s answer results under a specific learning task:

[0233] p=σ(W3×F p +b3)

[0234] Among them, W3 and b3 represent the parameters to be learned, W3 is the weight parameter, b3 is the corresponding bias parameter; σ is sigmoid as the activation function.

[0235] (4.5) The evaluation method for learners’ knowledge mastery is:

[0236] student knowledge-proficiency =σ(W4×student H +b4)

[0237] student H is the high-dimensional learner feature vector, W4, b4 represents the weight and bias parameters to be learned, and student knowledge-proficiency It is the evaluation result of learners’ knowledge mastery.

[0238] (4.6) The result of the exercise parameter representation is the vector exercise.

[0239] Example 2

[0240] The multimodal unified intelligent learning diagnosis modeling method provided by the embodiment of the present invention specifically includes:

[0241] (1) Based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed to form an extensible diagnostic framework. In this framework, a preliminary diagnosis of the learner is performed and the parameters of the learning resources are estimated to obtain the learning resource parameter set and the learner parameter set;

[0242] (2) Combine deep autoencoders to construct learner feature representation networks and learning resource feature representation networks, model learning resources and learners respectively based on the original parameter set, and obtain their deep representation features;

[0243] (3) Introducing the self-attention mechanism to effectively integrate learner characteristics and learning resource characteristics, fully exploring the correlation and importance information between features of each dimension, giving them different weights according to their different importance, and paying more attention to the most effective features;

[0244] (4) The fused features are used as the data basis for predicting learners’ performance, and a learner performance prediction network is constructed to obtain the predicted value of the learner’s probability of answering the question correctly. The learner’s overall knowledge mastery is diagnosed through the feature information of the learner and the exercises, and the parameter representation of the exercises is obtained to provide a reference for the learner’s personalized learning.

[0245] As a preferred embodiment of the present invention, a multi-channel cognitive diagnosis model is constructed based on the learner's historical learning records and learning resource information to form an extensible diagnosis framework. In this framework, a preliminary diagnosis of the learner is performed, and the parameters of the learning resources are estimated, so that the learning resource parameter set and the learner parameter set are obtained, including:

[0246] (1.1) Construct a learning resource collection and a learner's historical learning data collection:

[0247] S={s1,s2,...,s N}

[0248] E={e1,e2,...,e M}

[0249] K={k1,k2,...,k L}

[0250] Among them, S is the set of learners, E is the set of exercise questions, and K is the set of knowledge points.

[0251] Use the answer matrix R to record the students' historical answer results. Each row represents a student and each column represents an exercise. If student s answers exercise e correctly, then R se =1, otherwise R se = 0; use the exercise-knowledge point matrix Q to represent the relationship between the exercises and the knowledge points they examine. Each row represents a question and each column represents a knowledge point. If exercise e examines knowledge point k, then Q ek =1, otherwise Q ek =0.

[0252] (1.2) Based on the Q matrix and R matrix and various cognitive diagnostic models, the learner and test questions parameters are estimated to obtain the original parameter representation. For example, the learner's knowledge mastery, overall ability level parameters, etc.; the difficulty, discrimination, guessing rate, error rate, etc. of the exercise questions.

[0253] (1.3) The MATH dataset collected from the online education platform Zhixue.com is used, and the data information it contains is shown in Table 2.

[0254] Table 2 Description of MATH dataset

[0255] Number of students 14209 Number of exercises 15 (Objective questions) Number of knowledge points 11

[0256] As a preferred embodiment of the present invention, in an extensible cognitive diagnosis framework, taking the fusion IRT model and the Dina model as an example, the specific method of parameter acquisition and network construction is as follows:

[0257] (1.2.1) In the DINA model, the Q matrix and R matrix are used as input to estimate the learner and test parameters. Including: Learner - the mastery vector s for each knowledge point α ; Exercises—Guessing Rate e guess , error rate slip .

[0258] (1.2.2) Each student is described as a knowledge point mastery vector, where each dimension corresponds to a knowledge point. When the student's knowledge point mastery vector is known, for the test questions that the student has not answered, the student's potential answer to the test questions can be obtained according to the following formula:

[0259]

[0260] That is, when students have mastered all the knowledge points required to answer the test questions correctly, η uv =1.

[0261] (1.2.3) Two question parameters, error rate s and guess rate g, are introduced and defined as:

[0262] The student has mastered all the knowledge points required to answer the test questions, but for some reason answered incorrectly. That is, the error parameters are:

[0263] S j =P(R uv =0|η uv =1)

[0264] The student did not master all the knowledge points required to answer the test question, or even did not master any of the knowledge points, but the answer was correct. That is, guessing the parameters:

[0265] g j =P(R uv =1|η uv =0)

[0266] (1.2.4) The probability model of the actual response matrix is:

[0267]

[0268] (1.2.5) Thus, the total likelihood function of the DINA model is obtained:

[0269]

[0270] Where L = 2 V .

[0271] (1.2.6) Introduce the EM algorithm and use the maximum marginal likelihood estimation method to solve:

[0272] Step E: Use the s obtained in the previous round v With g v Estimated value calculation matrix P(R|α)=[P(R u |α l )] U×L The value of , and use P(R|α) to calculate the matrix P(α|R)=[P(α l |R u )] L×U The value of .

[0273] Step M: respectively and Available in represents the expected number of students who lack at least one knowledge point required for question v among the students who belong to the lth knowledge point mastering mode, express The expected number of people who answer question v correctly is and The meaning and and Similar, but different in that and is the expectation under the condition that students master all the knowledge points required for question v. Therefore, we can calculate and The value of , and thus get the new s v With g v Estimated value.

[0274] (1.2.7) In the IRT model, learner and test parameters are estimated based on the R matrix. Including: learner-ability value s θ ; Exercises—Difficulty e difficulty , Discrimination discrimination .

[0275] (1.2.8) Let the parameter of question v be δ v , the ability parameter of learner k is π k , and the total question parameter vector is δ, and the ability parameter of all learners is π kThe value set of is π, then the learner k's response is r k The conditional probability is:

[0276]

[0277] Among them, P(π k |δ j ) is the probability that learner k answers question j correctly.

[0278] (1.2.9) Thus, the total likelihood function of the IRT model is obtained:

[0279]

[0280] Among them, p(π k |π) is the learner's ability value is π k probability.

[0281] (1.2.10) Introduce the EM algorithm for parameter estimation:

[0282] Step E: Use the δ and π obtained in the previous iteration to calculate the power level to π k Number of people n k and the level is π k The number of people who answered the question correctly jk Expected value

[0283] Step M: Let Available make Solving the equation Available

[0284] As a preferred embodiment of the present invention, a deep autoencoder is combined to construct a learner feature representation network and a learning resource feature representation network, and the learning resource and the learner are modeled based on the original parameter set, and their deep representation features are obtained, including:

[0285] (2.1) Construct the learner parameter vector and learning resource parameter vector from the learner and learning resource parameter sets:

[0286] student F =(s θ ,s α )

[0287] exercise F =(e difficulty ,e discrimination ,e guess ,e slip )

[0288] (2.2) Preprocess the learner and learning resource parameter vectors. First, all parameters are segmented and mapped into several discrete values ​​according to their size. Then, they are encoded using the One-Hot method to generate the learner and learning resource parameter vectors:

[0289] student F =(SF1,SF2)

[0290] exercise F =(EF1,EF2,EF3,EF4)

[0291] Among them, SF i EF is the feature vector obtained by One-Hot encoding a parameter of the learner; j It is a feature vector obtained by One-Hot encoding a parameter of the learning resource.

[0292] (2.3) Design a learner feature deep autoencoder, which consists of an encoder and a decoder. The encoder encodes high-dimensional input into low-dimensional latent variables to learn the most informative features; the decoder restores the latent vector of the hidden layer to the initial dimension. The best state is that the output of the decoder can be completely or approximately restored to the original input, so that the latent vector can well represent the input information and achieve dimensionality reduction. The use of the deep autoencoder to obtain the latent feature vector specifically includes:

[0293] (2.3.1) Construct an encoder to obtain the hidden learner feature vector:

[0294] student H1 =g(W s1 ×student F +b s1 )

[0295] student H2 =g(W s2 ×student H1 +b s2 )

[0296] Among them, g is the tanh activation function, W s1 , W s2 is the encoding layer node weight parameter, b s1 , b s2 is the corresponding bias parameter;

[0297] (2.3.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features:

[0298] student D1 =g(Ws3 ×student H2 +b s3 )

[0299] student D2 =g(W s4 ×student D1 +b s4 )

[0300] Among them, g is the tanh activation function, W s1 , W s2 is the node weight parameter of the decoding layer, b s1 , b s2 is the corresponding bias parameter;

[0301] (2.3.3) Extract the hidden layer feature vector student H2 As a learner deep representation feature.

[0302] (2.4) Similarly, we design a deep autoencoder for learning resource features, whose composition structure is similar to that of the learner feature deep autoencoder:

[0303] (2.4.1) Construct an encoder to obtain the hidden layer learner feature vector:

[0304] exercise H1 =g(W e1 ×exercise F +b e1 )

[0305] exercise H2 =g(W e2 ×exercise H1 +b e2 )

[0306] Among them, g is the tanh activation function, W e1 , W e2 is the encoding layer node weight parameter, b e1 , b e2 is the corresponding bias parameter;

[0307] (2.4.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features:

[0308] exercise D1 =g(W e3 ×exercise H2 +b e3 )

[0309] exercise D2 =g(We4 ×exercise D1 +b e4 )

[0310] Among them, g is the tanh activation function, W e1 , W e2 is the node weight parameter of the decoding layer, b e1 , b e2 is the corresponding bias parameter;

[0311] (2.4.3) Extract the feature vector of the hidden layer H2 As a learning resource, deep representation features.

[0312] As a preferred embodiment of the present invention, a self-attention mechanism is introduced to effectively integrate learner features and learning resource features, fully explore the correlation and importance information between features of each dimension, give different weights to them according to different importance, and pay more attention to the most effective part of the features, including:

[0313] (3.1) Features from two dimensions: learners and exercises:

[0314] F = {student H2 ,exercise H2}

[0315] (3.2) Combine the self-attention mechanism to effectively integrate the learner and exercise characteristics;

[0316] The specific method of combining the self-attention mechanism to perform feature fusion includes:

[0317] (3.2.1) Input the concatenated features into the convolution layer with a convolution kernel size of 1 to obtain the query vector matrix Query, the key vector matrix Key, and the value vector matrix Value:

[0318] Query = Conv(F)

[0319] Key=Conv(F)

[0320] Value=Conv(F)

[0321] (3.2.2) The weight of each data in the feature is calculated by calculating the dot product between the query vector matrix and the key vector matrix. The weight represents the correlation between the task to be queried and each input data:

[0322] similarity(Query,Key i )=Query·Key i

[0323] (3.2.3) SoftMax is introduced to convert the correlation into a numerical value. On the one hand, it can be normalized to organize the original calculated scores into a probability distribution with a weight sum of 1; on the other hand, the weight of important elements can be more highlighted through the internal mechanism of SoftMax:

[0324]

[0325] (3.2.4) Perform weighted summation on the input data to obtain the fused feature data:

[0326]

[0327] As a preferred embodiment of the present invention, the fusion features are used as the data basis for predicting the learner's performance, and a learner performance prediction network is constructed to obtain a predicted value of the learner's correct answer probability; the learner's overall knowledge mastery is diagnosed through the feature information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained to provide a reference for the learner's personalized learning, including:

[0328] (4.1) Construct a convolutional network layer to predict the learner's answer results. Input the fused feature vector into the convolutional layer to extract its spatial information. The convolution kernel size is set to 3 and the step size is 1:

[0329] F c =Conv(F a )

[0330] (4.2) Add Relu activation function to reduce the dependency between parameters and avoid overfitting:

[0331] F re =Relu(F c )

[0332] (4.3) Reduce feature dimensions through the maximum pooling layer:

[0333] F p =MaxPool(F re )

[0334] (4.4) Input the reduced-dimensional features into the fully connected layer to predict the learner’s answer results under a specific learning task:

[0335] p=σ(W3×F p +b3)

[0336] Among them, W3 and b3 represent the parameters to be learned, W3 is the weight parameter, b3 is the corresponding bias parameter; σ is sigmoid as the activation function.

[0337] (4.5) The evaluation method for learners’ knowledge mastery is:

[0338] student knowledge-proficiency =σ(W4×student H +b4)

[0339] student H is the high-dimensional learner feature vector, W4, b4 represents the weight and bias parameters to be learned, and student knowledge-proficiency It is the evaluation result of learners’ knowledge mastery.

[0340] (4.6) The result of the exercise parameter representation is the vector exercise.

[0341] In addition, the experimental platform uses the Pytorch framework in Pycharm, as well as toolkits such as numpy, pandas, and sklearn to implement functions such as data preprocessing, model building and training, and result output.

[0342] The present invention compares this method with the traditional learning diagnosis method. In order to compare the objectivity of the results, each method is adjusted to the best state. The AUC and comparison results of a multimodal unified intelligent learning diagnosis modeling method and a traditional learning diagnosis method under the MATH data set are shown in Table 3.

[0343] Table 3 Comparison of experimental results

[0344]

[0345]

[0346] It can be seen from the experimental results that the AUC of the multimodal unified intelligent learning diagnosis modeling method proposed in the present invention on the MATH data set is improved by 7.67%, 12.76%, and 10.01% respectively compared with the traditional method, and the RMSE is reduced by 0.0562, 0.0052, and 0.04, which is more accurate than the traditional method. It shows that the present invention is based on the learner's historical learning records and learning resource information, and in an expandable diagnostic framework that integrates multiple cognitive diagnostic models, it makes a preliminary diagnosis of the learner, estimates the parameters of the exercise questions, and then constructs a network from the two dimensions of learners and exercise questions, namely the learner parameter network and the question parameter network, which can better model the exercise questions and learners, and more fully explore the high-dimensional features of the two. Combined with the self-attention mechanism, the characteristics of both learners and exercise questions are effectively integrated, so as to more accurately predict the learner's performance and accurately diagnose the learner's knowledge mastery.

[0347] In the description of the present invention, unless otherwise specified, "plurality" means two or more than two; the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", "front end", "rear end", "head", "tail", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0348] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0349] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A multimodal unified intelligent learning diagnostic modeling method, characterized in that: The multimodal unified intelligent learning diagnosis modeling method comprises the following steps: Step 1: Based on the learner's historical learning records and learning resource information, a multi-channel cognitive diagnosis model is constructed to form an extensible diagnostic framework; within this framework, a preliminary diagnosis is performed on the learner, and the parameters of the learning resources are estimated to obtain the learning resource parameter set and the learner parameter set; Step 2: Combine the deep autoencoder to construct the learner feature representation network and the learning resource feature representation network, model the learning resource and learner respectively based on the original parameter set, and obtain the deep representation features; Step 3: Introduce the self-attention mechanism to integrate learner features with learning resource features, explore the correlation and importance information between features of each dimension, assign different weights according to different importance, and pay more attention to the most effective features; Step 4: Use the fused features as the data basis for predicting learner performance, build a learner performance prediction network to obtain the predicted value of the learner's correct answer probability; diagnose the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtain the parameter representation of the exercise questions; In step 1, a multi-channel cognitive diagnosis model is constructed based on the learner's historical learning records and learning resource information to form an extensible diagnostic framework; within this framework, a preliminary diagnosis is performed on the learner, and parameters of the learning resources are estimated to obtain a learning resource parameter set and a learner parameter set, including: (1) Construct a collection of learning resources and a collection of learners’ historical learning data: S={s1,s2,...,s N } E}{e1,e2,...,e M } K={k1,k2,...,k L }; Among them, S is the set of learners, E is the set of exercises, and K is the set of knowledge points; (2) Use the answer matrix R to record the students' historical answer results. Each row represents a student and each column represents an exercise. If student s answers exercise e correctly, then R se =1, otherwise R se = 0; use the exercise-knowledge point matrix Q to represent the relationship between the exercises and the knowledge points they examine, with each row representing a question and each column representing a knowledge point; if exercise e examines knowledge point k, then Q ek =1, otherwise Q ek =0; (3) Construct a multi-channel cognitive diagnosis model, use the Q matrix and R matrix to estimate the learner and learning resource parameters, obtain the original parameter representation information, and construct the learner and learning resource parameter set based on it; The parameters include: the learner's knowledge mastery, overall ability level parameters, difficulty, discrimination, guessing rate and error rate of the exercise questions; The learner parameter set and learning resource parameter set are defined as: STUDENT={SF1,SF2,...,SF n } EXERCISE={EF1,EF2,...,EF m }; Among them, SF i Represents a characteristic parameter of the learner, EF j Represents a characteristic parameter of learning resources; In step 2, the deep autoencoder is combined to construct a learner feature representation network and a learning resource feature representation network, and the learning resource and the learner are modeled based on the original parameter set, and deep representation features are obtained, including: (1) Preprocess the learner and learning resource parameters. First, segment all parameters separately, map the continuous parameter values ​​into several discrete values ​​according to their size, and then use One-Hot encoding to generate learner and learning resource parameter vectors: student F =(SF1,SF2,...,SF n ) exercise F =(EF1,EF2,...,EF m ); Among them, SF i EF is the feature vector obtained by One-Hot encoding a parameter of the learner; j It is the feature vector obtained by One-Hot encoding a parameter of the learning resource; (2) Design a learner feature deep autoencoder, which consists of an encoder and a decoder. The encoder encodes high-dimensional input into low-dimensional latent variables to learn the most informative features; the decoder restores the latent vector of the hidden layer to the initial dimension. The best state is that the output of the decoder can be completely or approximately restored to the original input, so that the latent vector represents the input information and achieves dimensionality reduction. (3) Design a learning resource feature deep autoencoder, the composition structure of which is similar to the learner feature deep autoencoder.

2. The multimodal unified intelligent learning diagnostic modeling method according to claim 1, characterized in that: In step (2), the step of obtaining a latent feature vector by using a deep autoencoder includes: 2.1) Construct an encoder to obtain the hidden learner feature vector: student H1 =g(W s1 ×student F +b s1 ) student H2 =g(W s2 ×student H1 +b s2 ); Among them, g is the tanh activation function, W s1 , W s2 is the encoding layer node weight parameter, b s1 , b s2 is the corresponding bias parameter; 2.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features: student D1 =g(W s3 ×student H2 +b s3 ) student D2 =g(W s4 ×student D1 +b s4 ); Among them, g is the tanh activation function, W s3 , W s4 is the node weight parameter of the decoding layer, b s3 , b s4 is the corresponding bias parameter; 2.3) Extract the hidden layer feature vector student H2 As a learner’s deep representation feature; In step (3), the design of the learning resource feature deep autoencoder includes: 3.1) Construct an encoder to obtain the hidden learner feature vector: exercise H1 =g(W e1 ×exercise F +b e1 ) exercise H2 =g(W e2 ×exercise H1 +b e2 ); Among them, g is the tanh activation function, W e1 , W e2 is the encoding layer node weight parameter, b e1 , b e2 is the corresponding bias parameter; 3.2) Construct a decoder to decode the encoded features and obtain the reconstructed features of the input features: exercise D1 =g(W e3 ×exercise H2 +b e3 ) exercise D2 =g(W e4 ×exercise D1 +b e4 ); Among them, g is the tanh activation function, W e3 , W e4 is the node weight parameter of the decoding layer, b e3 , b e4 is the corresponding bias parameter; 3.3) Extract the feature vector of the hidden layer H2 As a learning resource, deep representation features.

3. The multimodal unified intelligent learning diagnostic modeling method according to claim 1, characterized in that: In step 3, the self-attention mechanism is introduced to integrate learner features with learning resource features, explore the correlation and importance information between features of each dimension, give different weights according to different importance, and pay more attention to the most effective features, including: (1) Features from two dimensions: learners and exercises: F={student D2 ,exercise D }; (2) effectively integrating learner and exercise question features by using a self-attention mechanism; wherein said integrating features by using a self-attention mechanism includes: 1) Input the concatenated features into the convolution layer with a convolution kernel size of 1 to obtain the query vector matrix Query, the key vector matrix Key, and the value vector matrix Value: Query = Conv(F) Key=Conv(F) Value = Conv(F). 2) The weight of each data in the feature is calculated by calculating the dot product between the query vector matrix and the key vector matrix. The weight represents the correlation between the task to be queried and each input data: similarity(Query,Key i )=Query·Key i ; 3) SoftMax is introduced to convert the correlation into numerical values; the original calculated scores are normalized into a probability distribution with a weight sum of 1; the weights of important elements are highlighted through the internal mechanism of SoftMax: 4) Perform weighted summation on the input data to obtain the fused feature data:

4. The multimodal unified intelligent learning diagnostic modeling method according to claim 1, characterized in that: In step 4, the fused features are used as the data basis for predicting the learner's performance, and a learner performance prediction network is constructed to obtain the predicted value of the learner's correct answer probability; the learner's overall knowledge mastery is diagnosed through the feature information of the learner and the exercise questions, and the parameter representation of the exercise questions is obtained, including: (1) Construct a convolutional network layer to predict the learner's answer results. Input the fused feature vector into the convolutional layer to extract its spatial information. The convolution kernel size is set to 3 and the step size is 1: F c =Conv(F a ); (2) Add the Relu activation function to reduce the dependency between parameters and avoid overfitting: F re =Relu(F c ); (3) Reduce feature dimensions through the maximum pooling layer: F p =MaxPool(F re ); (4) Input the reduced features into the fully connected layer to predict the learner’s answer results for a specific learning task: p=σ(W3×F p +b3); Among them, W3 and b3 represent the parameters to be learned, W3 is the weight parameter, b3 is the corresponding bias parameter; σ is sigmoid as the activation function; (5) The evaluation method for learners’ knowledge mastery is as follows: student knowledge-proficiency =σ(W4×student H +b4); student H is the high-dimensional learner feature vector, W4, b4 represents the weight and bias parameters to be learned, and student knowledge-proficiency Provide assessment results of learners’ knowledge mastery; (6) The result of the exercise parameter representation is the vector exercise.

5. A multimodal unified intelligent learning diagnostic modeling system using the multimodal unified intelligent learning diagnostic modeling method according to any one of claims 1 to 4, characterized in that: The multimodal unified intelligent learning diagnostic modeling system includes: The parameter set construction module is used to construct a multi-channel cognitive diagnosis model based on the learner's historical learning records and learning resource information to form an extensible diagnosis framework; within this framework, the learner is preliminarily diagnosed and the learning resource parameters are estimated to obtain the learning resource parameter set and the learner parameter set; A feature representation network construction module is used to combine the deep autoencoder to construct a learner feature representation network and a learning resource feature representation network, model the learning resource and learner respectively based on the original parameter set, and obtain deep representation features; The feature fusion module is used to introduce the self-attention mechanism to fuse learner features with learning resource features, explore the correlation and importance information between features of each dimension, assign different weights according to different importance, and pay more attention to the most effective features; The learning diagnosis and parameter representation module is used to use the fused features as the data basis for predicting the learner's performance, build a learner performance prediction network to obtain the predicted value of the learner's correct answer probability; diagnose the learner's overall knowledge mastery through the feature information of the learner and the exercise questions, and obtain the parameter representation of the exercise questions.

6. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multimodal unified intelligent learning diagnostic modeling method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the multimodal unified intelligent learning diagnostic modeling method as described in any one of claims 1 to 4.

8. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the multimodal unified intelligent learning diagnosis modeling system as described in claim 5.

Images