Learning evaluation method and device, electronic device, and storage medium

Abstract

The embodiment of the application provides a learning evaluation method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining exercise question text information, answer text information corresponding to the exercise question text information and answer record information corresponding to the answer text information of a to-be-tested user from a learning training system; inputting the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network respectively to output answer score information of the to-be-tested user; and performing information processing on the answer score information and the answer record information to obtain a learning evaluation result of the to-be-tested user. Based on this, the exercise question text information and the answer text information are respectively fed to an encoder and a decoder in the SAINT, so that the SAINT can capture the complex relationship between the exercise question and the answer in depth, and then the learning evaluation result of the to-be-tested user is obtained according to the answer score information and the answer record information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a learning evaluation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In the existing insurance agent training system, knowledge tracking is an important part, in order to track the state of understanding of each agent during learning, effective resource allocation can be made according to the ability and needs of the agent. Knowledge tracking specifically predicts whether the agent can pass the next simulation exercise of a certain category according to all past simulation contact situations of the agent (category of practice, whether the answer is correct, time spent, etc.).

[0003] Before the advent of deep learning, traditional knowledge tracking methods include Bayesian knowledge tracing (BKT) and collaborative filtering (CF). However, traditional knowledge tracking methods cannot capture the complex relationship between different exercises and answers, making it difficult to evaluate the learning results of the agent. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a learning evaluation method and device, electronic equipment and a storage medium, which can capture the complex relationship between different exercises and answers to obtain the learning evaluation result of the user to be tested.

[0005] To achieve the above purpose, the first aspect of the embodiments of the present application provides a learning evaluation method, which comprises:

[0006] Obtaining exercise text information of a user to be tested, answer text information corresponding to the exercise text information, and answer record information corresponding to the answer text information from a learning training system;

[0007] Respectively inputting the exercise text information and the answer text information into a preset SAINT-based deep knowledge tracking network, so that the SAINT-based deep knowledge tracking network outputs answer score information of the user to be tested, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network;

[0008] Performing information processing on the answer score information and the answer record information to obtain a learning evaluation result of the user to be tested.

[0009] In some embodiments, the separately inputting the exercise question text information and the answer text information into the preset SAINT-based deep knowledge tracking network, so that the SAINT-based deep knowledge tracking network outputs the test user's test score information, comprises:

[0010] inputting the exercise question text information into the encoder to obtain an exercise question feature vector;

[0011] inputting the answer text information into the decoder to obtain a test feature vector;

[0012] inputting the exercise question feature vector and the test feature vector into the decoder to obtain a test score vector;

[0013] determining the test user's test score information according to the test score vector.

[0014] In some embodiments, the information processing of the test score information and the test record information to obtain the test user's learning evaluation result, comprises:

[0015] inputting the test score information and the test record information into a preset gradient boosting decision tree model to obtain a prediction result feature output by the gradient boosting decision tree model;

[0016] obtaining the test user's learning evaluation result according to the prediction result feature.

[0017] In some embodiments, the inputting the test score information and the test record information into a preset gradient boosting decision tree model to obtain a prediction result feature output by the gradient boosting decision tree model, comprises:

[0018] training the gradient boosting decision tree model according to the test score information and the test record information to obtain a plurality of trained decision trees;

[0019] encoding the paths of the plurality of decision trees to generate the prediction result feature.

[0020] In some embodiments, the gradient boosting decision tree model is obtained by the following method:

[0021] configuring the maximum number of iterations and the loss function of the gradient boosting decision tree regression algorithm;

[0022] initializing a weak learner of the loss function;

[0023] performing iteration calculation on the weak learner for the maximum number of iterations to obtain the gradient boosting decision tree model.

[0024] In some embodiments, after determining the learning evaluation result of the to-be-tested user according to the answer record information corresponding to the answer score information and the answer text information, the method further comprises:

[0025] comparing the learning evaluation result of the to-be-tested user with previous learning evaluation results of the to-be-tested user to obtain a comparison evaluation result;

[0026] adjusting the learning plan of the to-be-tested user according to the comparison evaluation result.

[0027] In some embodiments, the answer record information comprises at least one of the following:

[0028] a category to which the question belongs;

[0029] a number of times of answering the question;

[0030] a time of answering the question.

[0031] To achieve the above object, a second aspect of the embodiments of the present application provides a learning evaluation device, which comprises:

[0032] an acquisition module configured to acquire, from a learning training system, exercise question text information of a to-be-tested user, answer text information corresponding to the exercise question text information, and answer record information corresponding to the answer text information;

[0033] a tracking module configured to input the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network, respectively, so that the SAINT-based deep knowledge tracking network outputs answer score information of the to-be-tested user, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, an output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network;

[0034] an evaluation module configured to perform information processing on the answer score information and the answer record information to obtain a learning evaluation result of the to-be-tested user.

[0035] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0036] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0037] The learning evaluation method and device, electronic equipment and storage medium provided by the present application obtain exercise question text information, answer text information corresponding to the exercise question text information and answer record information corresponding to the answer text information of a user to be tested from a learning training system; the exercise question text information and the answer text information are respectively input into a preset deep knowledge tracing network based on SAINT (Separated self-AttentIve Neural Knowledge Tracing), so that the deep knowledge tracing network based on SAINT outputs answer score information of the user to be tested, wherein the deep knowledge tracing network based on SAINT includes an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; the answer score information and the answer record information are processed to obtain a learning evaluation result of the user to be tested. Based on this, the present application inputs the exercise question and the answer into the deep knowledge tracing network based on SAINT independently to better construct input data. Since the SAINT is composed of the encoder and the decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of the multi-head self-attention module and the feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder can apply the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracing model, the present application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder of the SAINT respectively, so that the SAINT captures the complex relationship between the exercise question and the answer through the depth, and then obtains the learning evaluation result of the user to be tested according to the answer score information and the answer record information corresponding to the answer text information. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flowchart of the learning evaluation method provided by the embodiment of the present application;

[0039] Figure 2 is a flowchart of step S102 in Figure 1 ;

[0040] Figure 3 is a flowchart of step S103 in Figure 1 ;

[0041] Figure 4 is a flowchart of step S301 in Figure 3 ;

[0042] Figure 5 is a flowchart of generating a gradient boosting decision tree model;

[0043] Figure 6 is Figure 1 Fig. 9 is a flowchart of the process after step S103 in Fig. 8;

[0044] Figure 7 Fig. 1 is a structural schematic diagram of a learning evaluation device provided by an embodiment of the present application;

[0045] Figure 8 Fig. 6 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0047] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0049] First, the several terms involved in the present application are analyzed:

[0050] Artificial intelligence (AI): is a new technical science of researching, developing, simulating, extending and expanding human intelligence, and is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0051] Insurance agent: refers to the authorized insurance agent according to the insurance company, and the agent charges the agent fee. The insurance agent carries out business activities in the name of the insurance company within the scope of authorization, including the promotion and sales activities of the business, the acceptance of insurance, the establishment of temporary insurance or insurance, the collection of insurance fees, the agency of investigation and compensation, etc. The agent fee is usually paid according to the proportion of business volume. According to the scope of business, insurance agents can be divided into general agents, local agents and part-time agents, etc. The agent's way is to only agent for a company, and independent agent can agent for multiple companies.

[0052] Knowledge Tracing: is a technique that models students' knowledge mastery based on their past performance, resulting in a representation of students' current knowledge state.

[0053] Bayesian Knowledge Tracing (BKT): is a knowledge tracing model that proposes a hidden variable for students' knowledge state in the BKT model, which is represented by a binary tuple {master the knowledge point, not master the knowledge point}. The whole model structure is actually a HMM model, which predicts the next state according to the state transition matrix, and predicts the student's answer according to the current state. Moreover, in the BKT model, it is believed that once the knowledge is mastered, it will not be forgotten. Recently, some research has introduced the probability of guessing the right answer when the student does not master the knowledge and the probability of answering the wrong question when the student masters the knowledge, the student's prior knowledge and the difficulty of the question to extend the model. However, whether there are these extensions or not, the BKT model still has problems, the mapping between hidden state and exercise is fuzzy, and it is difficult to fully predict each exercise of a concept.

[0054] Collaborative filtering (CF): simply put, it is to use the preferences of a group of people with similar interests and common experiences to recommend information that users are interested in. Individuals give a certain degree of response (such as rating) to information through cooperative mechanisms and record it to achieve the purpose of filtering and help others filter information. The response is not necessarily limited to the information that is particularly interested in, and the record of information that is particularly not interested in is also very important.

[0055] Feedforward Neural Network (FNN): also known as feedforward neural network, is a kind of artificial neural network. In it, the parameters are propagated from the input layer to the output layer through the hidden layer in one direction. Unlike recurrent neural networks, it does not form a directed ring inside. FNN is composed of an input layer, one (shallow network) or multiple (deep network, hence the name deep learning) hidden layers, and an output layer. Each layer (except the output layer) is connected to the next layer. This connection is the key to the architecture of FNN, with two main features: weighted average and activation function.

[0056] Transformer: A deep learning model that uses a self-attention mechanism, which assigns different weights to different parts of the input data based on their importance. This model is mainly used in natural language processing (NLP) and computer vision (CV) fields. Like recurrent neural networks (RNNs), the Transformer model is designed to handle sequential input data such as natural language and can be applied to tasks such as translation and text summarization. Unlike RNNs, however, the Transformer model can process all input data at once. The attention mechanism provides context for any position in the input sequence. If the input data is natural language, the Transformer does not have to process one word at a time like an RNN, allowing for more parallel computation and reducing training time.

[0057] Gradient Boosting Decision Tree (GBDT): An iterative decision tree algorithm composed of multiple decision trees, which is an algorithm model for multi-classification. Gradient boosting constructs trees in a continuous manner, with each tree trying to correct the errors of the previous tree. By default, there is no randomization in the gradient boosting decision tree, but strong pre-pruning is used. Gradient boosting trees usually use a small depth, which makes the model occupy less memory and predict faster.

[0058] Based on this, the embodiment of the application provides a learning evaluation method and device, an electronic device and a storage medium. The learning training system is used to obtain exercise question text information of a to-be-tested user, answer text information corresponding to the exercise question text information and answer record information corresponding to the answer text information; the exercise question text information and the answer text information are respectively input into a preset SAINT-based deep knowledge tracking network, so that the SAINT-based deep knowledge tracking network outputs answer score information of the to-be-tested user, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; and the answer score information and the answer record information are processed to obtain a learning evaluation result of the to-be-tested user. Based on this, the embodiment of the application inputs the exercise question and the answer into the SAINT-based deep knowledge tracking network independently, so as to better construct the input data. Since the SAINT is composed of the encoder and the decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of the multi-head self-attention module and the feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder can apply the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracking model, the embodiment of the application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder of the SAINT respectively, so that the SAINT captures the complex relationship between the exercise question and the answer by depth, and then predicts the learning evaluation result of the to-be-tested user according to the answer record information corresponding to the answer score information and the answer text information, thereby realizing the prediction of the future possible learning score.

[0059] The learning evaluation method and device, the electronic device and the storage medium provided by the embodiment of the application are specifically described through the following embodiments. First, the learning evaluation method in the embodiment of the application is described.

[0060] The embodiment of the application can acquire and process related data based on artificial intelligence technology. The artificial intelligence (AI) is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge and using the knowledge to obtain the best results.

[0061] The artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0062] The learning evaluation method provided by the embodiments of the present application relates to the technical field of artificial intelligence. The learning evaluation method provided by the embodiments of the present application can be applied to a terminal, can be applied to a server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server side can be configured as a stand-alone physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application that implements the learning evaluation method, but is not limited to the above forms.

[0063] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0064] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed according to user information, user behavior data, user historical data, and user location information, etc. related to the identity or characteristics of the user, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, etc. After obtaining the separate permission or separate consent of the user, the necessary user-related data for the normal operation of the embodiments of the present application is obtained.

[0065] Figure 1 is an optional flowchart of the learning evaluation method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to steps S101 to S103.

[0066] Step S101, obtaining, from a learning training system, exercise question text information of a user to be tested, answer text information corresponding to the exercise question text information, and answer record information corresponding to the answer text information;

[0067] Step S102, inputting the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network respectively, so that the SAINT-based deep knowledge tracking network outputs answer scoring information of the user to be tested, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, an output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network.

[0068] Step S103, performing information processing on the answer scoring information and the answer record information to obtain a learning evaluation result of the user to be tested.

[0069] In step S101 of some embodiments, the user to be tested is various learning personnel participating in online training, for example, in an online training scene of the insurance industry, the learning training system can be an insurance agent learning training system, and the user to be tested can be an insurance agent. The exercise question text information is a set of exercise question topics that the user to be tested needs to answer in the online training system, the answer text information is a set of answers to the exercise question topics answered by the user to be tested in the online training system, and the answer record information includes but is not limited to the category to which the answer question topic belongs, the number of answers, and the answer time.

[0070] In step S102 of some embodiments, the exercise question and the answer are respectively independently input into the SAINT-based deep knowledge tracking network to better construct the input data. It should be noted that SAINT is a novel Transformer-based knowledge tracking model, and SAINT is specifically a separate self-attention neural knowledge tracking. SAINT has an encoder-decoder structure, an output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network. The exercise question text information and the answer text information are respectively input into the encoder and the decoder. The encoder applies the multi-head self-attention module to the exercise question text information, and the decoder applies the multi-head self-attention module to the answer text information. Since SAINT applies the deep multi-head self-attention module to the exercise question and the answer respectively, the structure fully utilizes the advantages of the multi-head attention module, so as to effectively capture the complex relationship between different exercise questions and answers. The SAINT-based deep knowledge tracking network outputs the answer scoring information of the user to be tested, that is, the answer score of the user to be tested.

[0071] In step S103 of some embodiments, the part of the answer record information is relatively discrete compared to the exercise text information and the answer text information, and therefore can be processed in a machine learning manner, such as a gradient boosting decision tree model. Thus, the learning evaluation result of the user to be tested is obtained according to the answer record information corresponding to the answer record information and the answer text information. Due to the introduction of the SAINT-based knowledge tracking model, the overall process reduces human intervention, and the knowledge tracking model deeply participates in the task selection and planning and provides quantifiable analysis results. Therefore, the present application can automatically track the state of understanding of each user to be tested, and can effectively allocate resources according to the ability and needs of the user to be tested. The SAINT-based knowledge tracking model undertakes most of the training records, evaluation and judgment, and predicts the future possible score (result) to provide data support for the user to be tested to formulate a more detailed learning plan task in the later stage.

[0072] In some embodiments, taking the online training scene of the insurance industry as an example, the deep knowledge tracking network based on the SAINT network mainly consists of an encoder and a decoder. The input of the encoder is the training question, and the decoder takes the output of the encoder plus the answer vector as input and outputs the score of the student. The structure of the SAINT network fully utilizes the advantages of the multi-head attention module and effectively captures the complex relationship between the exercise questions and the answers. In addition to the input exercise text information and the answer text information, the category of the answer question, the number of answers, the answer time, etc. belong to the important answer record information. For the answer record information, the GBDT model can be used for processing, that is, the vector output by the SAINT network is taken as input, and this part of the additional features is also taken as input. After training by the GBDT model, the final learning evaluation result is obtained. The present application can reform the original process on the basis of the existing knowledge tracking model. The result of the reform is the knowledge tracking system of the present application applied to the insurance industry, which can form the training life cycle management of insurance agents in the overall process by using the knowledge tracking network, and can predict the future learning result, learning attitude, time, etc. according to the learning evaluation result, to provide data support for the later planning task.

[0073] The steps S101 to S103 shown in the embodiments of the present application obtain the exercise question text information, the answer text information corresponding to the exercise question text information, and the answer record information corresponding to the answer text information from the learning training system; input the exercise question text information and the answer text information into the preset SAINT-based deep knowledge tracking network respectively, so that the SAINT-based deep knowledge tracking network outputs the answer score information of the user to be tested, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; and the answer score information and the answer record information are processed to obtain the learning evaluation result of the user to be tested. Based on this, the present application inputs the exercise question and the answer into the SAINT-based deep knowledge tracking network independently to better construct the input data. Since the SAINT is composed of the encoder and the decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder applies the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracking model, the present application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder of the SAINT respectively, so that the SAINT captures the complex relationship between the exercise question and the answer by depth, and then obtains the learning evaluation result of the user to be tested according to the answer record information corresponding to the answer score information and the answer text information.

[0074] Please refer to Figure 2 In some embodiments, step S102 can include but is not limited to steps S201 to S204:

[0075] Step S201, input the exercise question text information into the encoder to obtain an exercise question feature vector;

[0076] Step S202, input the answer text information into the decoder to obtain an answer feature vector;

[0077] Step S203, input the exercise question feature vector and the answer feature vector into the decoder to obtain an answer score vector;

[0078] Step S204, determine the answer score information of the user to be tested according to the answer score vector.

[0079] In step S201 of some embodiments, the exercise text information is a set of exercise questions that the user to be tested needs to answer in the online training system. The exercise text information is input into an encoder, which is a stack of multiple identical layers composed of a multi-head self-attention module and a feedforward network. The encoder can apply the multi-head self-attention module to the exercise text information, and the exercise text information is processed by the multi-head self-attention module and the feedforward network to obtain an exercise feature vector for representing the features of the exercise.

[0080] In step S202 of some embodiments, the answer text information is a set of answers to the exercise questions that the user to be tested answers in the online training system. The answer text information is input into a decoder, which is a stack of multiple identical layers composed of a multi-head self-attention module and a feedforward network. The decoder can apply the multi-head self-attention module to the answer text information, and the answer text information is processed by the multi-head self-attention module and the feedforward network to obtain an answer text information for representing the features of the answer.

[0081] In step S203 of some embodiments, the exercise feature vector and the answer feature vector output by the encoder are input into the decoder, which is processed by the multi-head self-attention module and the feedforward network to obtain an answer scoring vector for representing the features of the answer scoring.

[0082] In step S204 of some embodiments, the answer scoring vector is output by the SAINT-based deep knowledge tracking network to obtain the answer scoring information of the user to be tested, such as the score of the student. Based on this, since the SAINT applies a deep multi-head self-attention module to the exercise and the answer respectively, its structure fully utilizes the advantages of the multi-head attention module, thereby effectively capturing the complex relationships between different exercises and answers.

[0083] Please refer to Figure 3 In some embodiments, step S103 can include but is not limited to steps S301 to S302:

[0084] Step S301, input the answer scoring information and the answer record information into a pre-set gradient boosting decision tree model to obtain a predicted result feature output by the gradient boosting decision tree model;

[0085] Step S302, obtain the learning evaluation result of the user to be tested according to the predicted result feature.

[0086] In some embodiments, the answer record information includes, but is not limited to, the category to which the answer question belongs, the number of answer times, and the answer time. The answer record information is relatively discrete compared with the exercise text information and the answer text information, and therefore, a gradient boosting decision tree model can be used for processing to output a prediction result feature y, and the learning evaluation result of the to-be-tested user is predicted according to the prediction result feature y, so as to realize the prediction of the learning evaluation result of the to-be-tested user according to the answer record information corresponding to the answer scoring information and the answer text information.

[0087] In some embodiments, due to the introduction of the SAINT-based knowledge tracking model, the overall process reduces human intervention, and the knowledge tracking model is deeply involved in task selection and planning and provides quantifiable analysis results. Therefore, the present application can automatically track the state of understanding of each to-be-tested user, and can effectively allocate resources according to the ability and needs of the to-be-tested user. The SAINT-based knowledge tracking model undertakes most of the training records, evaluation and judgment, and predicts future possible scores (grades) to provide data support for formulating a more detailed learning plan task for the to-be-tested user in the later stage.

[0088] Referring to Figure 4 In some embodiments, step S301 can include, but is not limited to, steps S401 to S402:

[0089] Step S401, training a gradient boosting decision tree model according to the answer scoring information and the answer record information to obtain a plurality of trained decision trees;

[0090] Step S402, encoding the paths of the plurality of decision trees to generate a prediction result feature.

[0091] In some embodiments, the training of the gradient boosting decision tree model is performed through multiple iterations, and one decision tree is obtained in each iteration. The decision tree of each iteration is fitted on the basis of the loss function of the decision tree of the previous iteration, and the conclusions of all the decision trees are accumulated to obtain a prediction value. After obtaining the gradient boosting decision tree model, the paths of the decision trees in the gradient boosting decision tree model are encoded, and then the codes are combined to obtain a prediction result feature.

[0092] Referring to Figure 5 In some embodiments, the gradient boosting decision tree model can include, but is not limited to, steps S501 to S503:

[0093] Step S501, configuring the maximum number of iterations and the loss function of the gradient boosting decision tree regression algorithm;

[0094] Step S502, initializing a weak learner of the loss function;

[0095] Step S503, iteratively calculating the weak learner for a maximum number of iterations to obtain the gradient boosting decision tree model.

[0096] In some embodiments, a weak learner is initialized, a negative gradient is calculated according to the data in the sample set, a CART regression tree is fitted according to the calculated negative gradient, an optimal fitting value is calculated according to the leaf region of the CART regression tree, the strong learner is updated according to the calculated optimal fitting value to obtain an updated strong learner, the updated strong learner is taken as the weak learner of the next iteration, and the next iteration is performed; until the maximum number of iterations is reached, at which time the gradient boosting decision tree model is constructed.

[0097] Please refer to Figure 6 In some embodiments, after step S103, steps S601 to S602 can be included, but are not limited to:

[0098] Step S601, comparing the learning evaluation result of the to-be-tested user with the previous learning evaluation result of the to-be-tested user to obtain a comparison evaluation result;

[0099] Step S602, adjusting the learning plan of the to-be-tested user according to the comparison evaluation result.

[0100] In some embodiments, due to the introduction of the SAINT-based knowledge tracking model, the overall process reduces human intervention, and the knowledge tracking model deeply participates in task selection and plan making and provides quantifiable analysis results. Therefore, the present application can automatically track the state of understanding of each to-be-tested user, and can effectively allocate resources according to the ability and needs of the to-be-tested user. The SAINT-based knowledge tracking model undertakes most of the training records, evaluation and judgment, and predicts future possible scores (grades). After obtaining the learning evaluation result of the to-be-tested user, the learning evaluation result of the to-be-tested user is compared with the previous learning evaluation result of the to-be-tested user to obtain a comparison evaluation result, and the learning plan of the to-be-tested user is adjusted according to the comparison evaluation result, thereby providing data support for formulating a more detailed learning plan task for the to-be-tested user in the later period.

[0101] The learning evaluation method of the present application is further illustrated below in combination with specific embodiments.

[0102] Taking an online training scene of the insurance industry as an example, the deep knowledge tracking network based on the SAINT network mainly comprises an encoder and a decoder, the input of the encoder is a training question, the decoder takes the output of the encoder plus an answer vector as input, and outputs a score of a student. The SAINT network fully utilizes the advantages of the multi-head attention module, and effectively captures the complex relationship between the exercise question and the answer. In addition to the input exercise question text information and the answer text information, the category to which the question belongs, the number of answers, the answer time and the like are also important answer record information. The GBDT model can be used to process the answer record information, that is, the SAINT network output vector is taken as input, and the part of the extra features is also taken as input, after the GBDT model is trained, the final learning evaluation result is obtained. The present application can reform the original process on the basis of the existing knowledge tracking model, and the result after the reform is the knowledge tracking system of the present application applied to the insurance industry. In the overall process, the knowledge tracking network can form a training life cycle management of the insurance agent, and can predict the future learning result, learning attitude, time and the like according to the learning evaluation result, to provide data support for the later planning task.

[0103] Based on this, the present application inputs the exercise question and the answer into the deep knowledge tracking network based on SAINT respectively, in order to better construct the input data. Since the SAINT comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder can apply the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracking model, the present application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder of the SAINT respectively, so that the SAINT captures the complex relationship between the exercise question and the answer through deep self, and then obtains the learning evaluation result of the to-be-tested user according to the answer score information and the answer record information corresponding to the answer text information.

[0104] Please refer to Figure 7 The embodiment of the present application also provides a learning evaluation device, which can realize the learning evaluation method described above, and the device comprises:

[0105] The acquisition module 710 is configured to acquire, from a learning training system, exercise question text information of a to-be-tested user, answer text information corresponding to the exercise question text information, and answer record information corresponding to the answer text information;

[0106] The tracking module 720 is configured to input the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network respectively, so that the SAINT-based deep knowledge tracking network outputs the test score information of the user to be tested, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network.

[0107] The evaluation module 730 is configured to perform information processing on the test score information and the test record information to obtain the learning evaluation result of the user to be tested.

[0108] In some embodiments of the present application, the obtaining module 710 obtains the exercise question text information, the answer text information corresponding to the exercise question text information, and the test record information corresponding to the answer text information of the user to be tested from a learning training system, the identification module 720 inputs the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network respectively, so that the SAINT-based deep knowledge tracking network outputs the test score information of the user to be tested, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, and the evaluation module 730 performs information processing on the test score information and the test record information to obtain the learning evaluation result of the user to be tested.

[0109] In some embodiments of the present application, the obtaining module 710 obtains the exercise question text information and the answer text information corresponding to the exercise question text information of the user to be tested from a learning training system, wherein the user to be tested is various learning personnel participating in online training, for example, in an online training scene in the insurance industry, the user to be tested can be an insurance agent. The exercise question text information is a set of exercise question topics that the user to be tested needs to answer in the online training system, the answer text information is a set of answers to the exercise question topics answered by the user to be tested in the online training system, and the test record information includes but is not limited to the category to which the test question topic belongs, the test times, and the test time.

[0110] In some embodiments of the present application, the identification module 720 inputs the exercise questions and answers into the SAINT-based deep knowledge tracking network separately to better construct the input data. It should be noted that SAINT is a novel Transformer-based knowledge tracking model, and SAINT specifically refers to a separate self-attention neural knowledge tracking. SAINT has an encoder-decoder structure, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple identical layers composed of a multi-head self-attention module and a feedforward network. The exercise question text information and the answer text information are input into the encoder and the decoder, respectively. The encoder applies a multi-head self-attention module to the exercise question text information, and the decoder applies a multi-head self-attention module to the answer text information. Since SAINT applies a deep multi-head self-attention module to the exercise questions and answers, respectively, its structure fully utilizes the advantages of the multi-head attention module, thereby being able to effectively capture the complex relationships between different exercise questions and answers. The SAINT-based deep knowledge tracking network outputs the test user's answer score information, i.e., the test user's answer score.

[0111] In some embodiments of the present application, the evaluation module 730 obtains the learning evaluation result of the test user according to the answer score information and the corresponding answer record information of the answer text information, wherein the answer record information includes but is not limited to the category to which the answer question belongs, the number of answers, and the answer time. The answer record information is relatively discrete compared to the exercise question text information and the answer text information, so machine learning methods such as gradient boosting decision tree models can be used for processing. Thus, the learning evaluation result of the test user is obtained according to the answer score information and the corresponding answer record information of the answer text information. Due to the introduction of the SAINT-based knowledge tracking model, the overall process reduces human intervention, and the knowledge tracking model deeply participates in the task selection and planning and provides quantifiable analysis results. Therefore, the present application can automatically track the understanding state of each test user, and can effectively allocate resources according to the ability and needs of the test user. The SAINT-based knowledge tracking model undertakes most of the training records, evaluation and judgment, and predicts the future possible score (result) to provide data support for the test user to formulate a more detailed learning plan task in the later stage.

[0112] Based on this, the learning evaluation device in the embodiment of the present application, the acquisition module 710 acquires the exercise question text information of the user to be tested, the answer text information corresponding to the exercise question text information and the answer record information corresponding to the answer text information from the learning training system, the recognition module 720 respectively inputs the exercise question text information and the answer text information into the pre-set SAINT-based deep knowledge tracking network, so that the SAINT-based deep knowledge tracking network outputs the answer score information of the user to be tested, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, the evaluation module 730 processes the answer score information and the answer record information to obtain the learning evaluation result of the user to be tested. Based on this, the present application inputs the exercise question and the answer into the SAINT-based deep knowledge tracking network independently to better construct the input data. Since the SAINT is composed of an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder applies the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracking model, the present application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder in the SAINT respectively, so that the SAINT captures the complex relationship between the exercise question and the answer by depth, and then predicts the learning evaluation result of the user to be tested according to the answer record information corresponding to the answer score information and the answer text information, thereby realizing the prediction of the future possible learning score.

[0113] The specific implementation of the learning evaluation device is basically the same as that of the above-mentioned specific embodiment of the learning evaluation method, and will not be repeated here.

[0114] The embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor realizes the above-mentioned learning evaluation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer and the like.

[0115] Please refer to Figure 8 , Figure 8 The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0116] The processor 801 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0117] The memory 802 can be implemented by a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), and the like. The memory 802 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 802 and called and executed by the processor 801 to implement the learning evaluation method of the embodiments of the present application, that is, obtaining the exercise question text information of the to-be-tested user, the answer text information corresponding to the exercise question text information, and the answer record information corresponding to the answer text information from the learning training system; inputting the exercise question text information and the answer text information into the pre-set SAINT-based deep knowledge tracking network respectively, so that the SAINT-based deep knowledge tracking network outputs the answer score information of the to-be-tested user, wherein the SAINT-based deep knowledge tracking network includes an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; and performing information processing on the answer score information and the answer record information to obtain the learning evaluation result of the to-be-tested user. Based on this, the present application inputs the exercise questions and the answers into the SAINT-based deep knowledge tracking network independently to better construct the input data. Since the SAINT is composed of an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder applies the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracking model, the present application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder of the SAINT respectively, so that the SAINT captures the complex relationship between the exercise questions and the answers by depth, and then obtains the learning evaluation result of the to-be-tested user according to the answer record information corresponding to the answer score information and the answer text information.

[0118] The input / output interface 803 is configured to realize information input and output.

[0119] The communication interface 804 is configured to realize the communication interaction between the device and other devices. The communication can be realized in a wired manner (for example, a USB, a network cable, and the like) or in a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like).

[0120] The bus is configured to transmit information between various components (for example, the processor 801, the memory 802, the input / output interface 803, and the communication interface 804) of the device.

[0121] The processor 801, the memory 802, the input / output interface 803, and the communication interface 804 are connected to each other through the bus to realize the communication connection between the device.

[0122] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by the processor to realize the learning evaluation method.

[0123] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0124] The learning evaluation method, learning evaluation device, electronic equipment and storage medium provided by the embodiments of the present application obtain the exercise question text information of the user to be tested, the answer text information corresponding to the exercise question text information and the answer record information corresponding to the answer text information from a learning training system; input the exercise question text information and the answer text information into a preset deep knowledge tracking network based on SAINT respectively, so that the deep knowledge tracking network based on SAINT outputs the answer score information of the user to be tested, wherein the deep knowledge tracking network based on SAINT comprises an encoder and a decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; and the answer score information and the answer record information are processed to obtain the learning evaluation result of the user to be tested. Based on this, the present application inputs the exercise question and the answer into the deep knowledge tracking network based on SAINT independently to better construct the input data. Since the SAINT is composed of the encoder and the decoder, the output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network, therefore, the encoder can apply the multi-head self-attention module to the exercise question text information, and the decoder applies the multi-head self-attention module to the answer text information. Compared with the existing knowledge tracking model, the present application separates the exercise question text information and the answer text information, and feeds the exercise question text information and the answer text information into the encoder and the decoder of the SAINT respectively, so that the SAINT captures the complex relationship between the exercise question and the answer by depth, and then obtains the learning evaluation result of the user to be tested according to the answer record information corresponding to the answer score information and the answer text information.

[0125] Those skilled in the art can understand that all or some of the steps in the method disclosed above can be implemented by software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable programs, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, as known to those skilled in the art, communication media generally includes computer readable programs, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0126] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0127] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine some steps or different steps.

[0128] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, i.e., can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0129] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the functional modules / units in the system and the device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0130] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a given step or its integral presence in the process, method, system, article, or apparatus having been made with a wider scope. The use of notation such as "first", "second", "third", etc. does not generally limit the areas, but is used to connect like elements or to distinguish one claim from another. These terms can be used interchangeably when appropriate. Terms concerning the relative position of elements can be interpreted such that their use adheres to their normal meaning, but they can also be interpreted to mean the opposite according to specific claims.

[0131] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0132] In several embodiments provided by the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above-mentioned units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0133] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the application.

[0134] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0135] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0136] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A learning evaluation method characterized by comprising: The method comprises: obtaining exercise question text information of a to-be-tested user, answer text information corresponding to the exercise question text information and answer record information corresponding to the answer text information from a learning training system; inputting the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network respectively, so that the SAINT-based deep knowledge tracking network outputs answer score information of the to-be-tested user, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, an output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; performing information processing on the answer score information and the answer record information to obtain a learning evaluation result of the to-be-tested user; wherein the information processing of the answer score information and the answer record information to obtain the learning evaluation result of the to-be-tested user comprises: inputting the answer score information and the answer record information into a preset gradient boosting decision tree model to obtain the learning evaluation result; wherein the gradient boosting decision tree model is obtained by the following method: configuring a maximum iteration number and a loss function of a gradient boosting decision tree regression algorithm; initializing a weak learner of the loss function; performing iteration calculation on the weak learner for the maximum iteration number to obtain the gradient boosting decision tree model.

2. The method of claim 1, wherein, The inputting of the exercise question text information and the answer text information into the preset SAINT-based deep knowledge tracking network respectively to make the SAINT-based deep knowledge tracking network output the answer score information of the to-be-tested user comprises: inputting the exercise question text information into the encoder to obtain an exercise question feature vector; inputting the answer text information into the decoder to obtain an answer feature vector; inputting the exercise question feature vector and the answer feature vector into the decoder to obtain an answer score vector; determining the answer score information of the to-be-tested user according to the answer score vector.

3. The method of claim 1, wherein, The inputting of the answer score information and the answer record information into the preset gradient boosting decision tree model to obtain the learning evaluation result comprises: inputting the answer score information and the answer record information into a preset gradient boosting decision tree model to obtain a prediction result feature output by the gradient boosting decision tree model; obtaining the learning evaluation result of the to-be-tested user according to the prediction result feature.

4. The method of claim 3, wherein, The inputting of the answer score information and the answer record information into the preset gradient boosting decision tree model to obtain the prediction result feature output by the gradient boosting decision tree model comprises: training the gradient boosting decision tree model according to the answer score information and the answer record information to obtain a plurality of decision trees trained; encoding paths of the plurality of decision trees to generate the prediction result feature.

5. The method of claim 1, wherein, After the determination of the learning evaluation result of the to-be-tested user according to the answer record information corresponding to the answer score information and the answer text information, the method further comprises: The learning evaluation result of the to-be-tested user is compared with previous learning evaluation results of the to-be-tested user to obtain a comparison evaluation result; The learning plan of the to-be-tested user is adjusted according to the comparison evaluation result.

6. The method according to any one of claims 1 to 5, characterized in that, The answer record information includes at least one of the following: A category to which the answer question belongs; The number of times of answering the question; The time of answering the question.

7. A learning evaluation device characterized by comprising: The device comprises: An acquisition module configured to acquire, from a learning training system, exercise question text information of a to-be-tested user, answer text information corresponding to the exercise question text information, and answer record information corresponding to the answer text information; A tracking module configured to input the exercise question text information and the answer text information into a preset SAINT-based deep knowledge tracking network, respectively, so that the SAINT-based deep knowledge tracking network outputs answer score information of the to-be-tested user, wherein the SAINT-based deep knowledge tracking network comprises an encoder and a decoder, an output end of the encoder is connected to the decoder, and the encoder and the decoder are a stack of multiple same layers composed of a multi-head self-attention module and a feedforward network; An evaluation module configured to perform information processing on the answer score information and the answer record information to obtain a learning evaluation result of the to-be-tested user; The evaluation module is configured to perform information processing on the answer score information and the answer record information to obtain the learning evaluation result of the to-be-tested user, including inputting the answer score information and the answer record information into a preset gradient boosting decision tree model to obtain the learning evaluation result. The gradient boosting decision tree model is obtained by the following method: Configuring a maximum iteration number and a loss function of a gradient boosting decision tree regression algorithm; Initializing a weak learner of the loss function; Performing iteration calculation on the weak learner for the maximum iteration number to obtain the gradient boosting decision tree model.

8. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the learning evaluation method of any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the learning evaluation method of any one of claims 1 to 6.

Images