Online testing method and device

Abstract

The present application provides an online testing method and device for performing reinforcement learning from multiple dimensions, thereby selecting test questions for users from multiple dimensions, so that the test results can better reflect the user's actual ability. It includes: obtaining a test question bank, which includes multiple collected test questions; obtaining a test model based on the test question bank and a strategy optimization algorithm, and the test model can be used to select at least one test question from the test question bank during the online test process. The test model can specifically include a state encoder and a recommender. The state encoder is used to obtain the difference between the input test questions to generate a state code, and the recommender can be used to output the test question according to the state code and the optimization target. The optimization target includes at least one of novelty or diversity. The measurement factor of novelty includes exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

Description

Technical Field

[0001] The present application relates to the field of testing, and in particular to an online testing method and device. Background Art

[0002] With the rapid development of Internet technology, people have gradually gotten rid of the repetitive paper-and-pencil test model, and thus proposed Computerized Adaptive Testing (CAT). CAT is an online test that can accurately measure students' abilities by continuously providing them with the most appropriate test questions. CAT has been used in many large-scale educational examination scenarios, such as TOEFL and graduate school entrance examinations. The basic logic of CAT is: "Use the least number of questions to obtain the most comprehensive assessment of the test taker's ability level." For example, for a test taker with low ability, giving him or her difficult questions will not help assess the test taker's ability level. According to the test taker's ability level, giving questions of corresponding difficulty can obtain a more accurate test, avoiding questions that are far beyond their ability, wasting question opportunities, and avoiding the "question sea tactic."

[0003] Existing online testing methods have already achieved high-quality test question selection. However, if we only focus on the quality of the test questions, the selected test questions may not reflect the test taker's actual ability. Summary of the Invention

[0004] The present application provides an online testing method and device for performing reinforcement learning from multiple dimensions, thereby selecting test questions for users from multiple dimensions, so that the test results can better reflect the user's actual ability.

[0005] In view of this, in a first aspect, the present application provides an online testing method, comprising: obtaining a test question bank, the test question bank including a plurality of collected test questions; obtaining a test model based on the test question bank and a strategy optimization algorithm, wherein the test model can be used to select at least one test question from the test question bank during the online testing process, and the test model can specifically include a state encoder and a recommender, wherein the state encoder is used to obtain the difference between the input test questions to generate a state code, and the recommender can be used to output the test question according to the state code and the optimization target, wherein the optimization target includes at least one of novelty or diversity, wherein the measurement factor of novelty includes exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

[0006] In the implementation manner of the present application, in an online test scenario, test questions can be selected based on dimensions such as novelty and / or diversity, so as to select test questions with novelty and / or diversity for users, so that the selected test questions can more comprehensively test the user's answering ability.

[0007] In addition, novelty means that the knowledge points corresponding to the test questions selected by the test model have novel characteristics, which can be measured by the exposure rate of the knowledge points. For example, in each round of question selection, the test model needs to select test questions corresponding to knowledge points with low exposure rates for users; diversity means that the knowledge points selected by the test model for test questions have diverse characteristics, which can be measured by the coverage rate of knowledge points. For example, the test model can select test questions covering more knowledge points for users.

[0008] In one possible implementation, the optimization objective of the policy optimization algorithm may include a reward function for updating the test model. Therefore, when updating the test model, the desired optimization objective can be set for learning, thereby outputting a test model that is compatible with the optimization objective.

[0009] In one possible implementation, the aforementioned reward function may include a multi-dimensional reward function used to update the test model. The multiple dimensions may include, but are not limited to, at least two of quality, diversity, and novelty. Novelty indicates the exposure rate of the output test questions, and diversity indicates the coverage of a variety of knowledge points by the output test questions. In other words, the test model can be optimized to output novel or diverse test questions based on the exposure rate and knowledge point content of the test questions.

[0010] In the embodiment of the present application, during the reinforcement learning process, a reward function of multiple dimensions is set, so that the test model can be updated from multiple dimensions, so that the output results of the test model are better in multiple dimensions, thereby obtaining test questions that are more adapted to the actual ability of the user, and the selected test questions can better reflect the actual ability of the tester. In one possible embodiment, the aforementioned reward function of multiple dimensions may include but is not limited to quality rewards, diversity rewards and / or novelty rewards, the quality reward is determined based on the output accuracy of the test model in the test question bank, the diversity reward is determined based on whether the test question selected by the test model from the test question bank in the current time adds new knowledge points compared to the test question selected from the test question bank at least once before, and the novelty reward is determined based on whether the test question selected by the test model from the test question bank in the current time is a hot question. The test questions in the test question bank are divided into hot questions and non-hot questions, and the historical selection number of hot questions is higher than the historical selection number of non-hot questions.

[0011] Therefore, in the implementation manner of the present application, the test model can be updated from the dimensions of quality, novelty or diversity, so that the test questions output by the test model are of better quality, and the test model can output more novel and diverse test questions, thereby more comprehensively reflecting the ability of the tester.

[0012] In one possible implementation, the aforementioned test model may further include a relationship-aware aggregator, the input of which includes at least one of a premise graph or a correlation graph. The relationship-aware aggregator is configured to obtain, based on the input, an embedded representation of the relationship between knowledge points or an embedded representation of the relationship between test questions and knowledge points. The premise graph is configured to represent the sequential relationship between knowledge points in the input test questions, and the correlation graph is configured to represent the correlation between the test questions and the knowledge points.

[0013] The state encoder is used to extract the association between test questions and knowledge points based on the data output by the relationship-aware aggregator, and generate state codes based on the association.

[0014] In the implementation mode of the present application, a relationship-aware aggregator is also provided in the test model to extract the association relationship between the test questions and the knowledge points or between the knowledge points from the graph structure, and to more fully explore the relationship between the test questions and the knowledge points or between the knowledge points, thereby improving the accuracy of subsequent selection of test questions.

[0015] In a possible implementation, the aforementioned obtaining of a test model based on a test question bank and a strategy optimization algorithm may include: selecting at least one test question from the test question bank through the test model; and performing reinforcement learning on the test model through answer records of at least one test question to obtain a test model after reinforcement learning.

[0016] In the implementation manner of the present application, during the reinforcement learning process, reinforcement learning can be performed based on the user's answer records to the test questions, thereby performing reinforcement learning based on the user's actual answering ability to improve the output accuracy of the test model.

[0017] In a possible implementation, the aforementioned method may further include: obtaining a user's answer record for at least one test question from a test question bank; or, receiving online answer data obtained by the user performing operations on at least one test question, and obtaining the user's answer record for at least one test question based on the online answer data.

[0018] In the implementation manner of the present application, the user answer records used in the reinforcement learning process can be collected offline or obtained by the user answering questions online. Therefore, both offline learning and online learning can be achieved, and the test model can be adaptively updated based on the user's answering ability, thereby improving the output accuracy of the test model.

[0019] In one possible implementation, the aforementioned test question bank can be divided into a candidate set and a meta-question set. The test questions selected by the user are the test questions in the candidate set, and the test questions selected by the user are also used to train the test model, and the meta-question set is used to calculate rewards in multiple dimensions; reinforcement learning includes a training phase and a testing phase, the candidate set is used to test the test model in the training phase, and the meta-question set is used to calculate rewards in multiple dimensions in the verification phase.

[0020] Therefore, in the implementation manner of the present application, rewards can be calculated from multiple dimensions during the verification phase, thereby updating the test model from multiple dimensions, so that the test model can consider multiple dimensions when selecting test questions for the user, thereby outputting test questions that can better reflect the user's actual ability.

[0021] In one possible implementation, the aforementioned reinforcement learning may specifically include: in the testing phase, selecting at least one test question from a candidate set through a test model; after receiving a user's response to the at least one test question, obtaining an ability evaluation value based on the user's response to the at least one test question, the ability evaluation value being used to indicate the degree of accuracy of the user's answers to the test questions that have been screened for the user; in the verification phase, calculating rewards in multiple dimensions based on the ability evaluation value and the verification set, updating the test model based on the rewards in multiple dimensions, and obtaining the test model after the current iterative learning.

[0022] Therefore, in the implementation manner of the present application, rewards can be calculated from multiple dimensions during the verification phase, thereby updating the test model from multiple dimensions, so that the test model can consider multiple dimensions when selecting test questions for the user, thereby outputting test questions that can better reflect the user's actual ability.

[0023] In one possible implementation, the aforementioned test model obtained based on the test question bank and the strategy optimization algorithm may also include: using the test question bank to perform supervised learning to obtain the test model, wherein the test question bank includes labeled data that annotates diversity and / or novelty, and the supervised learning includes using the labeled data to perform supervised learning on the initial test model to obtain a trained test model.

[0024] In the embodiments of this application, in addition to reinforcement learning, supervised learning can also be performed to obtain a test model with stronger generalization. Furthermore, training with labeled data that indicates diversity and / or novelty can produce output results that are more desirable to users, thereby increasing the novelty and / or diversity of the test model's output results.

[0025] In a possible implementation, the aforementioned state encoder is specifically used to obtain the differences between the input test questions and the ability evaluation value corresponding to at least one user to generate a state code. The ability evaluation value corresponding to the user can be specifically calculated based on the user's answer record.

[0026] Therefore, when encoding, the state encoder can also combine the user's ability to perform state encoding, so that the obtained state encoding is consistent with the user's actual answering ability, thereby improving the accuracy of subsequent multiple-choice test questions.

[0027] In a second aspect, the present application provides an online testing device, comprising:

[0028] An acquisition module is used to acquire a test question bank, which includes a plurality of collected test questions;

[0029] A processing module is used to obtain a test model based on a test question bank and a strategy optimization algorithm. The test model is used to select at least one test question from the test question bank. The test model includes a state encoder and a recommender. The state encoder is used to obtain the difference between the input test questions to generate a state code. The recommender is used to output the test questions according to the state code and the optimization target. The optimization target includes at least one of novelty or diversity, wherein the measurement factor of novelty includes exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

[0030] In one possible implementation, the optimization objective of the policy optimization algorithm includes a reward function in multiple dimensions for updating the test model.

[0031] In one possible implementation, the multi-dimensional reward function is used to update the test model. The multiple dimensions include quality, diversity, and novelty. Novelty controls the exposure of the output test questions, while diversity indicates whether the output test questions cover a variety of knowledge points. In other words, the test model can be optimized to output novel or diverse test questions based on the exposure and knowledge point content of the test questions.

[0032] In a possible implementation, the processing module is specifically configured to: select at least one test question from a test question bank using a test model; and perform reinforcement learning on the test model using answer records of the at least one test question to obtain a test model after reinforcement learning.

[0033] In one possible implementation, the acquisition module is further used to: obtain an answer record for at least one test question from a test question bank; or, receive online answer data obtained by a user performing an operation on at least one test question, and obtain the user's answer record for at least one test question based on the online answer data.

[0034] In one possible embodiment, the test model also includes a relationship-aware aggregator, the input of the relationship-aware aggregator includes at least one of a premise graph or a correlation graph, and the relationship-aware aggregator is used to obtain an embedded representation of the relationship between knowledge points or between test questions and knowledge points based on at least one of the premise graph or the correlation graph. The premise graph is used to represent the chronological relationship between knowledge points in the input test questions, and the correlation graph is used to represent the correlation between test questions and knowledge points; the state encoder is used to extract the association relationship between test questions and knowledge points based on the data output by the relationship-aware aggregator, and generate state codes based on the association relationship.

[0035] In one possible embodiment, the reward functions of multiple dimensions include functions corresponding to at least one of quality reward, diversity reward, or novelty reward. The quality reward is determined based on the output accuracy of the test model in the test question bank. The diversity reward is determined based on whether the test question selected by the test model from the test question bank currently adds new knowledge points compared with the test question selected from the test question bank at least once before. The novelty reward is determined based on whether the test question selected by the test model from the test question bank currently is a hot question. The questions in the test question bank are divided into hot questions and non-hot questions. The historical selection number of hot questions is higher than the historical selection number of non-hot questions.

[0036] In one possible implementation, the test question bank is divided into a candidate set and a meta-question set. The test questions selected by the test model are the test questions in the candidate set, and the test questions selected by the test model are also used to train the test model. The meta-question set is used to calculate rewards in multiple dimensions.

[0037] Reinforcement learning includes a training phase and a testing phase. The candidate set is used to train the test model in the training phase, and the meta-problem set is used to calculate rewards in multiple dimensions in the testing phase.

[0038] In one possible implementation, the aforementioned reinforcement learning includes:

[0039] In the testing phase, at least one test question is selected from the candidate set using the test model. After receiving a user's response to the at least one test question, an ability evaluation value is obtained based on the user's response to the at least one test question. The ability evaluation value is used to indicate the degree to which the user correctly answers the test question that has been selected for the user.

[0040] In the verification phase, rewards in multiple dimensions are calculated based on the ability evaluation value and the verification set, and the test model is updated based on the rewards in multiple dimensions to obtain the test model after the current iterative learning.

[0041] In one possible implementation, the processing module is specifically used to: perform supervised learning using a test question bank to obtain a test model, wherein the test question bank includes labeled data that annotates diversity and / or novelty, and the supervised learning includes using the labeled data to perform supervised learning on the initial test model to obtain a trained test model.

[0042] In a possible implementation, the state encoder is specifically configured to obtain the difference between input test questions and at least one ability assessment value to generate a state code.

[0043] In a third aspect, embodiments of the present application provide an online testing device, which may also be referred to as a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is configured to perform the processing-related functions described in the first aspect or any optional embodiment of the first aspect. Optionally, the online testing device may be a chip.

[0044] In fourth aspect, an embodiment of the present application provides an online testing device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform processing-related functions as described in the first aspect or any optional embodiment of the first aspect.

[0045] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional embodiment of the first aspect.

[0046] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of reinforcement learning in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the system architecture provided for this application;

[0049] Figure 3 Another system architecture diagram provided for this application;

[0050] Figure 4 A schematic diagram of a flow chart of an online testing method provided in this application;

[0051] Figure 5A schematic diagram of another online testing method provided in this application;

[0052] Figure 6 A schematic diagram of the relationship between knowledge points provided in this application;

[0053] Figure 7 A schematic diagram of another online testing method provided in this application;

[0054] Figure 8 A schematic diagram of an online test effect provided by this application;

[0055] Figure 9 Another online test effect diagram provided for this application;

[0056] Figure 10 A schematic structural diagram of an online testing device provided in this application;

[0057] Figure 11 A schematic structural diagram of another online testing device provided in this application;

[0058] Figure 12 A schematic diagram of the structure of a chip provided in this application. DETAILED DESCRIPTION

[0059] The following will describe the technical solutions in the embodiments of this application in conjunction with the drawings in the embodiments of this application. The embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] First, we describe the overall workflow of an AI system. Next, we elaborate on the aforementioned AI framework from the perspectives of the "intelligent information chain" and the "IT value chain." The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, this could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data to information to knowledge to wisdom." The "IT value chain," spanning AI's underlying infrastructure, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value AI brings to the information technology industry.

[0061] (1) Infrastructure

[0062] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. Communication with the outside world occurs through sensors; computing power is provided by intelligent chips, such as central processing units (CPUs), neural-network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or hardware acceleration chips like field programmable gate arrays (FPGAs). The foundational platform includes distributed computing frameworks and networks, along with other related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0063] (2) Data

[0064] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0065] (3) Data processing

[0066] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0067] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0068] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0069] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0070] (4) General ability

[0071] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0072] (5) Smart products and industry applications

[0073] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0074] The embodiments of the present application involve related applications of neural networks and online testing. In order to better understand the solutions of the embodiments of the present application, the following first introduces the related terms and concepts of neural networks and online testing that may be involved in the embodiments of the present application.

[0075] (1) Computerized Adaptive Testing (CAT):

[0076] CAT is an online test that accurately measures students' abilities by continuously providing them with the most appropriate test questions. CAT has been used in many large-scale educational testing scenarios, such as the TOEFL and the Graduate Record Examination. The basic logic of CAT is to "use the fewest questions to obtain the most comprehensive assessment of the test-taker's ability level." For example, giving a low-ability test-taker difficult questions will not help assess their ability level. By giving the test-taker questions of appropriate difficulty based on their ability level, a more accurate test can be obtained, avoiding the choice of questions that are significantly different from the test-taker's ability, wasting question opportunities, and preventing the "sea of ​​questions" tactic.

[0077] (2) Self-attention

[0078] Self-attention effectively encodes a sequence of data (such as the natural language sentence "Your phone is great.") into multiple multidimensional vectors for easy numerical computation. These vectors incorporate the similarity between each element in the sequence, a measure known as self-attention. The self-attention model can be understood as a mapping from a query to a series of key-value pairs.

[0079] (3) Neural Network

[0080] A neural network can be composed of neural units. A neural unit can refer to an operation unit with xs and intercept 1 as input. The output of the operation unit can be shown as formula (1-1):

[0081]

[0082] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s is x s The weight of the neural unit, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0083] (4) Feedforward neural network (FNN)

[0084] A feedforward neural network is a type of artificial neural network. It uses a unidirectional, multi-layer structure. Each layer contains several neurons. In this type of neural network, each neuron receives signals from neurons in the previous layer and generates outputs to the next layer. The first layer is called the input layer, the last layer is called the output layer, and the other intermediate layers are called hidden layers (or hidden layers). Hidden layers can be single or multiple. For example, deep neural networks (DNNs) and convolutional neural networks (CNNs) are examples of FNN-based neural networks.

[0085] (5) Deep neural network (DNN), also known as multi-layer neural network, can be understood as a neural network with multiple intermediate layers. According to the position of different layers, the neural network inside the DNN can be divided into three categories: input layer, intermediate layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all intermediate layers, or called hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer.

[0086] Although DNN looks complicated, each layer can be expressed as a linear relationship expression: in, is the input vector, is the output vector, is the offset vector or bias parameter, w is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The number of these parameters is also relatively large. The definitions of these parameters in DNN are as follows: Take the coefficient w as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0087] In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as

[0088] It's important to note that the input layer has no W parameter. In deep neural networks, more intermediate layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0089] (6) Convolutional Neural Networks

[0090] A convolutional neural network (CNN) is a deep neural network with a convolutional architecture. It consists of a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons that performs convolution processing on the input signal. In a convolutional layer of a CNN, a neuron can only connect to a subset of neurons in adjacent layers. A convolutional layer typically contains several feature planes, each of which can be composed of a rectangular arrangement of neurons. Neurons in the same feature plane share weights, which are referred to as convolution kernels. Shared weights can be understood as extracting image information in a position-independent manner. Convolution kernels can be initialized as matrices of random size, and during CNN training, they can learn to acquire reasonable weights. Furthermore, shared weights have the direct benefit of reducing the number of connections between layers of the CNN, thereby reducing the risk of overfitting.

[0091] Taking a convolutional layer as an example, a convolutional layer can include multiple convolution operators, also known as kernels. In image processing, they act as filters that extract specific information from the input image matrix. A convolution operator is essentially a weight matrix, which is usually predefined. During the convolution operation, the weight matrix is ​​typically applied horizontally to the input image, pixel by pixel (or two pixels by two pixels, depending on the stride), to extract specific features from the image. The size of the weight matrix should be proportional to the image size. It is important to note that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image; during the convolution operation, the weight matrix extends across the entire depth of the input image. Therefore, convolution with a single weight matrix produces a convolved output with a single depth dimension. However, in most cases, a single weight matrix is ​​not used; instead, multiple weight matrices of the same dimension are applied. The outputs of each weight matrix are stacked to form the depth dimension of the convolved image. Different weight matrices can be used to extract different features in the image. For example, one weight matrix is ​​used to extract image edge information, another weight matrix is ​​used to extract specific colors of the image, and another weight matrix is ​​used to blur unwanted noise in the image... The multiple weight matrices have the same dimensions, and the feature maps extracted by the multiple weight matrices with the same dimensions also have the same dimensions. The multiple feature maps with the same dimensions extracted are then merged to form the output of the convolution operation.

[0092] The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input image, thereby helping CNN to make correct predictions.

[0093] When CNN has multiple convolutional layers, the initial convolutional layer often extracts more general features, which can also be called low-level features. As the depth of CNN increases, the features extracted by the subsequent convolutional layers become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0094] (7) Graph Convolutional Network (GCN)

[0095] Graph Convolutional Network (GCNN) is a deep learning model that models and processes non-Euclidean spatial data (such as graph data). Its principle is to use pairwise message passing, so that graph nodes iteratively update their corresponding representations by exchanging information with their neighbors.

[0096] GCN is similar to CNN, but differs in that CNN typically takes two-dimensional structured data as input, while GCN typically takes graph-structured data as input. GCN cleverly designs a method for extracting features from graph data, allowing these features to be used for node classification, graph classification, link prediction, and graph embedding.

[0097] (8) Graph attention networks (GAT)

[0098] Graph Attention Networks (GANs) are a novel neural network architecture for graph-structured data. Graph Attention Networks combine graph neural networks with attention layers. Hidden attention layers can be used to address the shortcomings of previous methods based on graph convolutions or their approximations. By stacking these layers, nodes can learn about the characteristics of their neighbors, implicitly assigning different weights to different nodes in the neighborhood without requiring any expensive matrix operations (such as inversions) or prior knowledge of the graph structure.

[0099] (9) Autoencoder

[0100] An autoencoder is a neural network that uses a backpropagation algorithm to make its output equal to its input. It compresses the input data into a latent space representation and then reconstructs the output from this representation. An autoencoder typically consists of an encoder model and a decoder model.

[0101] (10) Reinforcement Learning

[0102] Reinforcement learning (RL), also known as reinforcement learning, evaluation learning or enhanced learning, is used to describe and solve the problem of how an agent can maximize rewards or achieve specific goals by learning strategies during its interaction with the environment.

[0103] Reinforcement learning involves an agent learning through trial and error. Rewards earned through interactions between actions and the environment guide the agent's behavior, with the goal of maximizing the reward. Reinforcement learning does not require a training dataset. Instead, the reinforcement signal (i.e., reward) provided by the environment evaluates the quality of the action, rather than instructing the reinforcement learning system on how to perform the correct action. Because the external environment provides little information, the agent must rely on its own experience to learn. In this way, the agent acquires knowledge in an action-reward (i.e., reward) environment and refines its action plan to adapt to the environment.

[0104] Reinforcement learning is a machine learning paradigm used to solve problems in which intelligent agents learn strategies to maximize rewards or complete specific machine learning tasks during their interactions with the environment. The agent learns through trial and error, earning rewards through interaction with the environment and using them to guide action selection. It finds the optimal strategy for the current state and selects appropriate actions based on that strategy to maximize its rewards.

[0105] For example, Figure 1 This is a diagram of the training process of reinforcement learning. Figure 1 As shown in the figure, reinforcement learning mainly includes four elements: agent, environment state, action and reward. Among them, the input of the agent is called state and the output is called action.

[0106] For example, the training process for reinforcement learning involves multiple interactions between the agent and the environment, obtaining the state, action, and reward for each interaction. These multiple sets of (state, action, reward) are used as training data for the agent to train once. The agent is trained again using this process until convergence conditions are met.

[0107] Among them, the process of obtaining the status, action, and reward of an interaction is as follows: Figure 1 As shown, the current state of the environment s(t) is input to the agent, and the agent outputs an action a(t). The reward r(t), or return, for this interaction is calculated based on the relevant performance indicators of the environment under the influence of action a(t). Thus, the state s(t), action a(t), and reward r(t) of this interaction are obtained. The state s(t), action a(t), and reward r(t) of this interaction are recorded for subsequent training of the agent. The next state s(t+1) of the environment under the influence of action a(t) is also recorded to facilitate the next interaction between the agent and the environment.

[0108] (11) Loss function

[0109] It can also be called a cost function, a measure of the difference between the predicted output of a machine learning model for a sample and the true value of the sample (also called the supervised value), that is, it is used to measure the difference between the predicted output of a machine learning model for a sample and the true value of the sample. The loss function can usually include loss functions such as squared error, cross entropy, logarithm, exponential, etc. For example, the squared error can be used as a loss function, defined as The specific loss function can be selected according to the actual application scenario.

[0110] (12) Back propagation (BP)

[0111] An algorithm that calculates the gradient of model parameters based on a loss function and updates the model parameters. Neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during training, reducing the reconstruction error loss of the neural network model. Specifically, forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The backpropagation algorithm is a backpropagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0112] (13) Gradient: The derivative vector of the loss function with respect to the parameters.

[0113] (14) Knowledge point: A relatively independent minimum unit of knowledge, theory, principle, thought, etc. In this application, a knowledge point can be understood as a unit of information included in a test question, and each test question can include one or more knowledge points.

[0114] First, the system architecture provided by this application is introduced. The method provided by this application can be deployed in a terminal or server (such as deployed in a cloud platform or other remote servers, etc.). When deployed in a terminal, the method provided by this application can be directly deployed in the terminal, and the terminal can directly select the appropriate test questions for the user from the test question bank. When deployed in a server, taking a cloud platform as an example, Figure 2 As shown, deployed on an education platform, it can provide services to users through the client. For example, the education platform can select test questions for users and present them to them through the client. During the user's answering process, the platform selects the next test question online based on the user's answer record. This allows users to select test questions that are appropriate for their abilities, more accurately reflecting their abilities.

[0115] Each user's client device can interact with the server cluster 310 through a communication network of any communication mechanism / communication standard, and the communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof. Specifically, the communication network can include a wireless network, a wired network, or a combination of a wireless network and a wired network. The wireless network includes but is not limited to: a fifth-generation mobile communication technology (5th-Generation, 5G) system, a long-term evolution (longterm evolution, LTE) system, a global system for mobile communication (global system for mobile communication, GSM) or a code division multiple access (code division multiple access, CDMA) network, a wideband code division multiple access (wideband code division multiple access, WCDMA) network, wireless fidelity (wireless fidelity, WiFi), Bluetooth (bluetooth), Zigbee protocol (Zigbee), radio frequency identification technology (radio frequency identification, RFID), long range (Lora) wireless communication, close field communication (near field communication, NFC) any one or more combinations. The wired network can include a fiber optic communication network or a network composed of coaxial cables, etc.

[0116] The aforementioned terminals may specifically include but are not limited to personal computers, computer workstations, smart phones, tablet computers, smart cameras, smart cars or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc.

[0117] Typically, a CAT scene consists of multiple parts. For example, a CAT scene can be Figure 3 As shown, it can specifically include a cognitive diagnosis model (CDM) and a selection algorithm;

[0118] CDM captures student ability through their responses to test questions. A common, simple CDM approach is item response theory (IRT), which uses item response functions (IRFs) to approximate students' true ability. Deep learning-based CDMs, such as Neural CDM (NCD), use neural networks to simulate the interaction between students and test questions.

[0119] The selection algorithm selects the most suitable test questions for students based on their historical answer records. The mainstream CAT question setting strategies in the industry mainly fall into two categories:

[0120] Heuristic rule-based question-setting algorithms: Based on information entropy theory, they calculate the information content of each question and use this ranking to determine the order of questions. Representative models include the Maximum Information Entropy (MFI) algorithm, the Kullback-Leibler Information (KLI) algorithm, and the Quality Meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing (MAAT) algorithm.

[0121] Parameterized training and learning strategies: Based on a neural network architecture, these strategies are optimized using techniques such as meta-learning and reinforcement learning. Representative models include the meta-learning-based BOBCAT and the reinforcement learning-based NCAT.

[0122] Traditional static selection algorithms typically use heuristics to select questions with the highest amount of information (MFI) or the highest expected model change (MAAT). These algorithms are often greedy for a single step but lack long-term vision. In recent years, data-driven selection algorithms have emerged that can learn from large datasets.

[0123] The selection algorithm is used as a question selector. Figure 3The CAT process plays a vital role. The commonly used selection algorithm is to use the maximum Fisher information (MFI) to select test questions, or to use the KL information (Kullback-Leibler Information, KLI) method to calculate the integral over an ability interval to select test questions. These heuristic algorithms can only be used for specific CDMs, such as IRT. In order to improve this defect, a CDM-independent algorithm MAAT was proposed. It is based on the rules of active learning and selects test questions that cause the greatest change in CDM. In addition, RAT also benefits the selection algorithm by capturing multiple aspects of students' abilities. In recent years, many scholars have proposed data-driven selection algorithms from the perspective of meta learning or reinforcement learning. For example, the bilevel optimization-based computerized adaptive testing method (BOBCAT) is a meta-learning-based method that combines CDM and selection algorithms in a bilevel optimization framework; the fully adaptive framework: neural computerized adaptive testing for online education (NCAT) is a reinforcement learning-based method that uses a deep Q-network (DQN) network based on the attention mechanism to select test questions.

[0124] For example, MAAT is an active learning-based algorithm whose design is not constrained by a specific CDM. Its selection algorithm consists of three parts, each with its own functions: a quality module that ranks questions based on information content (maximum information entropy); a diversity module that assigns weights to different knowledge points, with the more frequently they are selected, the more important they are; and an importance module that predicts the more similar a question is to others. However, the MAAT algorithm's selection strategy is greedy, focusing only on the importance of the current solution. As a result, the resulting solution is suboptimal and may fall into suboptimal solutions. Furthermore, its selection strategy is fixed, resulting in weak generalization capabilities.

[0125] For example, BOBCAT is a meta-learning-based method that introduces the MAML framework. Through alternating inner and outer layer training, it trains the CDM and selection strategy as a whole. The CDM is trained in the inner layer, and the selection strategy is trained in the outer layer. BOBCAT, for the first time, establishes a parameter connection between the CDM and CAT, enabling integrated training. However, BOBCAT focuses solely on quality objectives, and the resulting test questions may lack diversity and novelty, failing to fully reflect the test takers' abilities. Furthermore, BOBCAT fails to fully exploit the effective information between the test questions and knowledge points, potentially preventing it from selecting the optimal solution.

[0126] For example, NCAT is a reinforcement learning-based approach that proposes a learnable neural computer adaptive testing framework. This framework formally defines CAT as a reinforcement learning problem, learning the selection algorithm directly from real data. Specifically, NCAT reshapes the selection algorithm into a two-level optimization objective, transforming the loss value in the two-level optimization problem into maximizing the expected cumulative reward in reinforcement learning. In terms of model structure, NCAT uses a two-channel performance learning (PL) component to capture complex student performance information and a contradiction learning (CL) component to identify and extract contradictions in student scores, attempting to mitigate the impact of interference. Finally, NCAT makes the next step selection at the policy level, optimizing using Q-Learning methods. However, similar to BOBCAT, NCAT also focuses solely on quality objectives. The final selected test questions may lack diversity and novelty, failing to fully reflect the test subject's ability. Furthermore, it does not fully exploit the effective information between the test questions and knowledge points, and may not select the optimal solution.

[0127] Therefore, the present application provides an online testing method, which uses reinforcement learning to perform online learning on the test question selection algorithm, and updates the selection algorithm from multiple dimensions, so that test questions with higher quality, more novelty and diversity can be selected.

[0128] In the method provided in the present application, after obtaining a test question bank including multiple test questions, a test model can be obtained based on the test question bank and a strategy optimization algorithm, such as reinforcement learning or supervised learning. The test model can be used to select at least one test question from the test question bank during the online test process. The test model includes a state encoder and a recommender. The state encoder is used to obtain the differences between the input test questions to generate state codes, and the recommender is used to output test questions based on the state codes and optimization objectives; wherein the optimization objective includes one of novelty or diversity, wherein the measurement factor of novelty may include exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

[0129] The optimization objectives of the policy optimization algorithm include a reward function used to update the test model. The test model is updated through the reward function, and in the process of updating the test model, the output results of the test model are made to better meet the guidance or constraints of the reward function.

[0130] Specifically, a reward function of multiple dimensions can be set in the optimization objective, and the reward function of multiple dimensions is used to update the test model, so that when the test model selects test questions for the user, it can select test questions for the user from multiple dimensions, so that the selected test questions can more accurately reflect the user's ability.

[0131] When using a policy optimization algorithm to generate a test model, reinforcement learning or supervised learning can be used. For example, experts can label data for diversity or novelty, and then use supervised learning based on state-coded input to enable the test model to output test questions that meet the requirements of diversity or novelty. Alternatively, a test model can be generated with at least one of novelty or diversity as the optimization objective, thereby outputting novel or diverse test questions through reinforcement learning.

[0132] That is, in one possible implementation, a test question bank can be used for supervised learning to obtain a test model, wherein the test question bank includes labeled data that annotates diversity and / or novelty, and supervised learning includes using the labeled data to perform supervised learning on the initial test model to obtain a trained test model. In the implementation of the present application, in addition to reinforcement learning, supervised learning can also be performed to obtain a test model, thereby having stronger generalization. And by using labeled data annotated with diversity and / or novelty for training, it is possible to obtain results that users more expect to be output, thereby improving the novelty and / or diversity of the output results of the test model.

[0133] It should be noted that in the following embodiments of this application, the policy optimization algorithm is taken as an example of the reinforcement learning algorithm. The reinforcement learning process mentioned below can also be replaced by supervised learning or other learning processes with at least one of novelty or diversity as the optimization goal. This application does not limit this.

[0134] See Figure 4 , a flow chart of an online testing method provided in this application is as follows.

[0135] 401. Get the test question bank.

[0136] The test question bank may include multiple test questions set for the user, each of which may include one or more knowledge points. Generally, the test questions may be pre-set, generated based on pre-set knowledge points, or collected from big data. The test question bank may be the same or different for each user.

[0137] It should be noted that the users mentioned in this application may also be referred to as subjects, students, testers or other roles participating in the test.

[0138] For example, taking a grammar test of a certain language as an example, the test question bank may include multiple test questions of various types, such as multiple single-choice questions containing grammar knowledge points, multiple multiple-choice questions, or translation test questions.

[0139] For another example, taking a math test as an example, the test question bank may include addition test questions, subtraction test questions, multiplication test questions, division test questions, or test questions combining multiple algorithms, etc.

[0140] The test question bank may contain different data for different scenarios. For example, in offline training, the answer records of one or more users can be collected after they have completed the test. The test question bank can also include the answer records of one or more users for the test questions. For example, in an online test scenario, after the test questions are selected and displayed to the user, the user's answer data can be received, thereby obtaining the answer records for each test question.

[0141] 402. A test model is obtained based on the test question bank and the strategy optimization algorithm.

[0142] The test model can be used to output selected test questions. Specifically, the test model can include a state encoder and a recommender. The state encoder is used to generate a state code based on the differences between input test questions. The recommender can be used to output test questions based on the state code and an optimization objective. The optimization objective includes at least one of novelty or diversity. The novelty metric includes exposure, and the diversity metric includes the presence of new knowledge points.

[0143] The strategy optimization algorithm can learn with at least two of the optimization objectives of quality, novelty and / or diversity. Novelty indicates controlling the exposure rate of test questions, and diversity indicates covering a variety of knowledge points, thereby obtaining a test model that can output test questions with novelty or diversity.

[0144] For example, in the embodiments of this application, reinforcement learning is used as an example. During the reinforcement learning process, one or more test questions selected by the user can be first selected from the test question bank. After receiving the user's response feedback, further learning can be carried out based on the user's response, thereby selecting test questions that are adapted to the user's ability based on the updated test model. Alternatively, the user's historical answer records can be used for reinforcement learning.

[0145] In the process of reinforcement learning, refer to the above Figure 1 In the reinforcement learning process shown, the user's answering can be understood as the interaction between the intelligent agent and the environment, and the test model can be understood as the decision model. The test model may include a state encoder and a recommender. The state encoder can be used to obtain the difference between the input test questions to generate a state code, and the recommender is used to output the test questions selected by the user based on the state code. By having the user answer the questions, one or more sets of (state, action, reward) training data can be obtained, and the obtained one or more sets of training data can be used for learning to update the test model. In particular, when calculating the reward, the reward can be calculated from multiple dimensions, thereby updating the test model from multiple dimensions, so that the test model can select test questions for the user from multiple dimensions, so that the selected test questions can more comprehensively reflect the user's ability and improve the test effect.

[0146] Specifically, the state encoder can generate state codes based on the differences between test questions and the user's ability assessment, with at least two of quality, novelty, or diversity as optimization goals. The user's ability assessment can be specifically determined by the user's response to the test questions. Novelty includes a goal related to the exposure rate of the test questions, and diversity includes a goal related to whether the output test questions include new knowledge points.

[0147] Specifically, the optimization objectives of the strategy optimization algorithm also correspondingly include the reward functions of the aforementioned multiple dimensions, which may specifically include at least one corresponding function of quality reward, diversity reward or novelty reward. The quality reward is determined based on the output accuracy of the test model in the test question bank. The diversity reward is determined based on whether new knowledge points are added between the test question selected by the test model from the test question bank in the current time and the test question selected from the test question bank at least once before. The novelty reward is determined based on whether the test question selected by the test model from the test question bank in the current time is a hot question. The questions in the test question bank are divided into hot questions and non-hot questions. The historical number of selections of hot questions is higher than the historical number of selections of non-hot questions.

[0148] Therefore, in this embodiment, the test model can be rewarded and updated based on dimensions such as quality, diversity or novelty, so that the test model can select test questions for users based on dimensions such as quality, diversity or novelty, thereby reflecting the user's ability more comprehensively and accurately.

[0149] Reinforcement learning in this embodiment can be divided into multiple phases, such as a testing phase and a verification phase. Accordingly, the test question bank can be divided into a candidate set and a meta-question set. The test questions selected by the test model for the user are selected from the candidate set, while the test questions in the meta-question set are used to calculate the value of the reward function in the verification phase. In other words, when calculating reward functions for multiple dimensions, they are calculated based on the test questions in the meta-question set.

[0150] Specifically, during the testing phase, at least one test question can be selected from the candidate set through the test model. After receiving the user's response to the at least one test question, the user's corresponding ability evaluation value can be calculated based on the response of the at least one test question. The ability evaluation value can be used to indicate the degree of correctness of the user's answer to the test question selected for him.

[0151] In the verification phase, rewards in multiple dimensions can be calculated based on the user's ability evaluation value and the verification set, and the test model can be updated based on the rewards in multiple dimensions to obtain the test model after the current iterative learning.

[0152] In addition, the test model in this embodiment may also include a relationship-aware aggregator, the input of which may include a premise graph or a related graph. The relationship-aware aggregator can be used to extract the association relationship between knowledge points or the association relationship between test questions and knowledge points from the input premise graph or related graph, and convert the association relationship into an embedded representation through the embedding layer. The state encoder can extract the association relationship between test questions and knowledge points based on the embedded representation output by the relationship-aware aggregator, and generate a state code based on the association relationship.

[0153] The premise graph represents the execution order of knowledge points in the input test questions. For example, a premise relationship involving a pair of knowledge points means that one knowledge point should be learned logically before the other, such as multiplication being the successor of addition. The dependency graph represents the correlation between test questions and knowledge points, for example, indicating which knowledge points a test question includes.

[0154] Therefore, in the implementation manner of the present application, the association relationship between test questions and knowledge points or the association relationship between knowledge points can be represented by a graph structure. The present application sets up a relationship-aware aggregator for the graph structure, so that the association relationship between test questions and knowledge points or between knowledge points can be mined, and the performance of subsequent test questions in various dimensions can be improved, thereby increasing the probability of obtaining the optimal solution.

[0155] The above describes the method flow provided by this application. To facilitate understanding, the method provided by this application is described in more detail below in combination with specific application scenarios.

[0156] First, the method provided in this application can be understood as a graph-enhanced multi-objective method for CAT (GMOCAT). Specifically, the CAT process is represented as a multi-objective Markov decision process (MOMDP), and then a scalarized multi-objective reinforcement learning (MORL) framework is introduced. Compared to greedy algorithms, the RL framework has been shown to be able to explore test questions that are more suitable for the test subjects from a long-term perspective.

[0157] In the implementation of this application, multiple goals are set in the RL framework, such as:

[0158] Quality objectives: Accurately predict students' abilities;

[0159] Diversity goal: to diversify knowledge concepts when recommending test questions to more comprehensively reflect students' abilities;

[0160] Novelty goal: Control the exposure rate of test questions and avoid selecting test questions with high exposure rate.

[0161] To achieve the above goals, this application sets up multiple rewards in the RL framework, such as quality rewards, diversity rewards, or novelty rewards.

[0162] See Figure 5 The method flow provided in this application is introduced from multiple perspectives. For example, it can be divided into multi-objective rewards, relationship-aware aggregator, state encoder, and actor-critic recommender. The actor-critic recommender can also be called a recommender. Each of them is introduced below.

[0163] 1. Multi-target rewards

[0164] 1. Quality rewards

[0165] Generally, an effective selection algorithm can select the test questions that are most suitable for students, thereby accurately predicting their abilities. For any student, since her true ability is unknown, her meta-question set can be used to measure the error in ability estimation. Specifically, at test step t, we need to calculate θ t The prediction accuracy on its meta-problem set is denoted as ACC(θ t).ACC(θ t ) value, the more accurate the ability estimate is and the closer it is to the true ability. It can be understood that if the test questions selected by the algorithm help improve the accuracy of ability estimation, an incentive should be given. In contrast, if the selected questions reduce this accuracy, a penalty should be given. Therefore, the quality reward can be expressed as:

[0166] r qua =ACC(θ t )-ACC(θ t-1 )

[0167] Among them, the test questions in the meta-problem set are usually used to calculate quality rewards and will not be selected for students.

[0168] 2. Diversity Rewards

[0169] In large-scale comprehensive exams, questions should contain a wide range of knowledge points. Diversity objectives require coverage of a variety of knowledge points. Therefore, if the selection algorithm selects a question with a new knowledge point, a positive reward should be given. To simplify the algorithm, the reward value can be discretized. If a question involving a new knowledge point is selected, the diversity reward is 1, otherwise it is 0:

[0170]

[0171] Among them, c t It is q t Knowledge points covered.

[0172] 3. Novelty Rewards

[0173] Typically, lack of novelty leads to overexposure of test questions, which may affect students' test-taking behavior. Therefore, the selection algorithm needs to take novelty into account, and novelty rewards are used to control the exposure of test questions. Represents a pre-given set of popular test questions. It can encourage the selection of unpopular test questions, because these test questions are more likely to be novel in the future, thereby maintaining a balanced distribution of test question exposure. Therefore, if the selected test question is not In the example, the novelty reward is 1, otherwise it is 0, as expressed as:

[0174]

[0175] generally, It is predetermined and usually does not change during the CAT process. Of course, it can be updated as the number of tests increases and according to the number of times each test question is selected.

[0176] 2. Relationship-Aware Aggregator

[0177] Since knowledge points appear in both the premise graph (indicating the order of knowledge points) and the related graph (indicating the related relationships between knowledge points), such as Figure 6 As shown in , its embedding is affected by two relationships. GAT is applied to aggregate the relationship information represented in the premise graph or the related graph. For example, for knowledge point c, its corresponding original embedding is ε c ,set up The neighbor embeddings with attention weights are aggregated to obtain the premise relation-aware embedding g pre and the relation-aware embedding g cor :

[0178]

[0179] middle

[0180]

[0181] att. represents a linear layer with LeakyReLU activation function. [·] is the concatenation operation, W pre , W cor are trainable parameters.

[0182] g pre and g cor Contains different relationship information. In order to distinguish their importance, the following processing method is used. The weight of the premise relationship μ pre From the attention vector P and g pre It is calculated from the similarity between them, such as the formula can be expressed as:

[0183] μ pre =P T ·tanh(W·g pre +b)

[0184] The weight of the correlation μ cor The two weights are then subjected to a softmax operation. Finally, the relation-aware embedding of the knowledge point c is Obtained by the following formula:

[0185]

[0186] Question relation aggregation: GAT is used to aggregate question relations again. For question q, its original embedding is ε q ,set up is the set of neighbors in the correlation graph. Since the test questions only contain related relations, their relation-aware embedding Calculated by the following formula:

[0187]

[0188] 3. State Encoder

[0189] State encoder f se Taking historical response records as input, generate status:

[0190]

[0191] First, use the matrix W q Map each question q() to a real-valued embedding The dimension of the embedding vector is d. Apply the same operation to obtain the embedding vector ε of knowledge point c c , the embedding vector of the response y

[0192] In addition, the relationship information can be extracted by using the relationship-aware aggregator, and the original test questions and knowledge points are embedded as the input of the relationship-aware aggregator to obtain the relationship-aware embedding of the test questions and knowledge points. and

[0193] Got and Afterwards, the state encoder concatenates the embeddings corresponding to the triples of each historical time step, as expressed as:

[0194]

[0195] Correspondingly, the historical response record (also known as the user's answer record or learning record) {(q_{t′},c_{t′},y_{t′})|t′∈[1,t-1]} can be represented as a matrix:

[0196] E t =[e1,e2,…,e t-1 ] T

[0197] Different historical response records usually contain different information. For example, correctly answering a difficult question contains more information than correctly answering an easy question. In order to capture the differences between response records, t Apply the self-attention mechanism, as expressed as:

[0198]

[0199] Where W Q , W K , W V are trainable parameters, It is also possible to add a LayerNorm layer and a skip-connection layer after the self-attention layer to use the Dropout mechanism to avoid overfitting.

[0200] Because the student's actual ability is fixed or fluctuates within a small range during the CAT process, the order of each record is not important. Perform average pooling operation to generate state code s t .

[0201] 4. Actor-Critic Recommender

[0202] The Actor-Critic recommender can use state encoding to select the next question. The Actor-Critic recommender can be understood as including an Actor and a Critic. Actor is a parameter φ π The fully connected layer is used to extract the t |s t ;φ π ) in the sampling action. Critic is a parameter φ v Fully connected layer, given the state, Critic output:

[0203] V(s t ;φ v )=[V(s t ) qua ,V(s g ) div ,V(s t ) nov ]

[0204] It can be understood as a vector of predicted expected returns, where each element corresponds to the quality, diversity, and novelty goals respectively.

[0205] Use the multi-objective Proximal Policy Optimization (PPO) algorithm (which can also be replaced by other reinforcement learning algorithms, such as DQN or DDPG) to maximize the weighted sum of benefits Specifically, define the choice q t The advantage value A(s t ,q t ) is the actual payoff value of the state-action pair minus the expected return value of the state:

[0206]

[0207] The vectorized advantage value is converted into a scalar using the scalarization function w, and the Actor parameters are updated using the clipping proxy loss. The loss function can be defined as:

[0208]

[0209] The purpose of clip is to limit the update range. The loss function of the critic is to make the expected return as close as possible to the actual return. The loss function can be defined as:

[0210]

[0211] Subsequently, the loss of Multi-Objective PPO (MOPPO) is a weighted sum of two losses, using a hyperparameter α:

[0212]

[0213] The final loss value is used for reverse update to update the Actor-Critic recommender, state encoder, and relationship-aware aggregator.

[0214] In the reinforcement learning process of the embodiment of the present application, the CAT task can be modeled as a continuous decision problem and formalized as a multi-objective Markov decision process (MOMDP). This MOMDP can be represented by Tuple definition, where:

[0215] Represents a state set. During testing, the state is defined as Among them

[0216] f se Represents a state encoder that takes the historical response record of student i as input and outputs the state

[0217] is a finite set of actions, which can be understood as a candidate set. During testing, the selection algorithm selects a question from the action set (i.e., candidate set).

[0218] Indicates that the status Select the question After that, we reach the next state The conversion probability

[0219] R: Indicates that the selection algorithm is in state Select The immediate reward function of . The reward function is vectorized, that is,

[0220] denote quality, diversity, and novelty rewards, respectively.

[0221] γ∈[0,1]: is a discount factor that weighs immediate and future rewards.

[0222] Therefore, this application reconstructs the CAT process from the perspective of MORL: Let n represent the number of students. For student i, at test step t, the selection algorithm π selects a question from her candidate question set based on her state, i.e. Then, it pushes the test questions to student i and obtains multi-objective rewards Finally, the expectation is to maximize the weighted sum of the benefits

[0223]

[0224] Where w is a scalar function, which can be regarded as a weight vector whose elements represent the importance of each target.

[0225] For example, Figure 7 As shown in Figure 1, the CAT process using the data-driven selection algorithm includes two stages: training and testing. In order to train / test the selection algorithm, it is necessary to segment the sample containing the interaction records of student i into candidate question sets. Heyuan Question Set like Figure 7 The candidate question set and meta-question set are randomly selected and are different for each student. The training / testing phase of the CAT process in the embodiment of the present application is defined as follows:

[0226] Training phase: For each student i in the training set,

[0227] (1) At test step t, the selection algorithm selects a question from the candidate question set based on its historical response records;

[0228] (2) The student gives her / his response and the CDM updates its current estimate of the student's ability

[0229] (3) Use and meta-problem sets to compute reward feedback, measuring the precision of ability estimates;

[0230] (4) After T iterations of the above process, the selection algorithm can be trained to maximize the feedback.

[0231] Testing phase: For a new student j in the testing set, phases (1) and (2) are the same as the training phase. Phase (3) is to use We evaluate various performance metrics using meta-problem sets. During the testing phase, we only verify the output effect and do not train the selection algorithm.

[0232] The above describes the method flow provided by this application. The following describes the effects achieved by the method provided by this application in combination with specific application scenarios.

[0233] Take three educational datasets as examples: Eedi, ASSIST, and Junyi. Students with fewer than 40 interaction records are removed. The statistical information of the processed datasets is shown in Table 1.

[0234] Dataset Eedi ASSIST Junyi Number of students 4918 1360 20395 Test Questions 948 17751 2835 Knowledge Points 86 123 40 Answer record 1382727 239919 2537898 Prerequisites 334 1166 306 Knowledge points for each test question 4.0 1.2 1.0 Positive label probability 0.55 0.62 0.69

[0235] Table 1

[0236] Use 80%-10%-10% of students as training set, validation set and test set. Students in the training set will not appear in the validation / test set. Divide the samples containing student i's interaction records into candidate question sets Heyuan Question Set These two sets are unique for each student and are randomly generated in each training epoch to prevent overfitting. The experimental results are averaged over five runs. All experimental results are obtained on the test set. We compare static algorithms such as Random, MFI, KLI, and MAAT with learnable algorithms such as BOBCAT and NCAT.

[0237] Evaluation indicators can be set:

[0238] Quality Indicators: The prediction accuracy of student i’s final ability estimate on its meta-question set can be calculated. Therefore, this application uses the area under the ROC curve (AUC) and accuracy (ACC) as quality indicators.

[0239] Diversity index: Knowledge point coverage (Cov) can be used to measure diversity. Specifically, let is the knowledge point set, is the set of knowledge points covered by all selected test questions before step t. Cov is defined as the proportion of knowledge points covered by all selected test questions:

[0240]

[0241] Novelty index: Novelty can be measured by question exposure rate (such as the proportion of times a question is selected) and average overlap rate (the average overlap between the questions selected by any two students among all students):

[0242]

[0243] where N q is the count of question q being selected, is the set of all students, Q i is the set of questions that student i takes when taking the test.

[0244] The final quality indicator output effect can be shown in Table 2.

[0245]

[0246] Table 2

[0247] Among them, Metric represents the unit of measurement, static represents the static selection algorithm, and Learnable represents the learnable selection algorithm.

[0248] Therefore, on two different CDMs on three public datasets, the output of our proposed scheme outperforms all baselines. These results show that relational information and multi-objective strategies can improve the accuracy of capability estimation.

[0249] The output of diversity indicators can be as follows Figure 8 shown.

[0250] Obviously, the method provided by this application has a clear diversity goal and a relationship-aware selection algorithm in the MORL framework, and the Cov curve grows fastest on all data sets of all two CDMs.

[0251] The output results of the novelty index can be shown in Table 3.

[0252]

[0253] Table 3

[0254] While the random method achieves the lowest exposure rate, it doesn't necessarily mean it's the best approach. This is because randomly selected test questions lack personalization, defeating the purpose of CAT. The random methods listed here primarily serve to illustrate a practical lower bound for exposure rates. Of all methods except Random, the GMOCAT method presented in this application achieves the lowest exposure and overlap rates, demonstrating the effectiveness of novelty incentives. In contrast, the method presented in this application achieves even lower question exposure by directly optimizing exposure.

[0255] Therefore, the method provided in this application takes into account more comprehensive evaluation indicators and proposes a new question-setting strategy and a new multi-objective reinforcement learning optimization method. A multi-objective reinforcement learning CAT question-setting strategy method is proposed that unifies the three objectives of quality, diversity, and novelty, achieving a trade-off between these objectives. It can flexibly adapt to different practical needs. Secondly, the relationship between test questions and knowledge points is used to assist in question selection. Integrating relationship-aware embedding into the question selection strategy improves question quality.

[0256] In addition, desensitized learning data were collected from an educational center, as shown in Table 4.

[0257]

[0258]

[0259] Table 4

[0260] Similarly, the GMOCAT model provided by this application outperforms all baselines, which further verifies the effectiveness of the GMOCAT model. Therefore, this application has also achieved good results in the implementation examples of more complex industrial datasets. This is also because compared with existing solutions, the implementation examples of this application add a network module of the graph attention mechanism, explore the correlation between knowledge points and test questions, and incorporate the embedding of relationship perception into the topic selection strategy; multi-objective optimization helps this application fully consider multiple factors and achieve a balance between goals. Significantly improve the accuracy of recommendations.

[0261] Use the relationship information between the test questions and the knowledge points to help with question selection. Specifically, use relationship-aware embedding to incorporate such relationship information into the selection strategy. To illustrate its effectiveness, this application uses ablation experiments to verify. The experimental results are shown in the table below (GMOCAT-R means removing relationship-aware embedding. This means that this application ignores the relationship graph and replaces the relationship-aware embedding with the original embedding.). Figure 8 As can be observed, GMOCAT-R removes important relationship information between test questions and knowledge points, which greatly reduces its performance. This shows that it is reasonable to capture relevant information to select more appropriate test questions.

[0262] As shown in Table 5.

[0263]

[0264] Table 5

[0265] Multi-objective optimization in CAT helps this application take multiple factors into account and achieve a balance between objectives. To explore the role of different objectives, we explored the differences when GMOCAT focuses on three subsets of objectives. The first, second, and third elements of w correspond to the quality, diversity, and novelty objectives, respectively. The following w configurations can be used for experiments:

[0266] w∈{[1, 1, 1],[1, 0, 0],[0, 1, 0],[0, 0, 1],[1, 1, 0],[1, 0, 1],[0, 1, 1]}

[0267] The main consideration is the impact of the presence / absence of each target, so the value of w is 1 (indicates presence) or 0 (indicates absence).

[0268] The performance comparison of GMOCAT under different W configurations can be as follows Figure 9 shown.

[0269] Obviously, focusing on only one target will lead to a decrease in the performance of other metrics. For example, [1,0,0] gets a low coverage (Cov) value.

[0270] [0,1,0] and [0,0,1] obtain lower AUC / ACC values, which illustrates the importance and necessity of utilizing multiple objectives simultaneously.

[0271] exist Figure 8 In the AUC and ACC metrics, adding a diversity objective increases the AUC / ACC value (for example, [1, 1, 0] outperforms [1, 0, 0] in AUC / ACC). This phenomenon is consistent with intuition, as students' abilities are multifaceted. If the test questions contain different knowledge points, students' abilities can be predicted more accurately.

[0272] exist Figure 8 Adding a novelty objective to the AUC and ACC metrics slightly weakens quality performance (e.g., [1, 1, 0] outperforms [1, 1, 1] in AUC / ACC). This phenomenon suggests a potential conflict between quality and novelty objectives. The novelty objective attempts to achieve a balanced distribution of test questions, which results in low-quality questions being selected more frequently, thus hindering the prediction of student ability.

[0273] Obviously, there are both promotion and contradictions between the three goals. This application provides a flexible way to adapt to the needs of different scenarios.

[0274] The above describes the detailed process of the method provided by this application. The following describes the device provided by this application for executing the above method process.

[0275] See Figure 10 , the present application provides a structural diagram of an online testing device, which may include:

[0276] An acquisition module 1001 is configured to acquire a test question bank, wherein the test question bank includes a plurality of collected test questions;

[0277] Processing module 1002 is used to obtain a test model based on a test question bank and a strategy optimization algorithm. The test model is used to select at least one test question for the user from the test question bank. The test model includes a state encoder and a recommender. The state encoder is used to obtain the difference between the input test questions to generate a state code. The recommender is used to output the test questions selected by the user based on the state code and the optimization target. When the optimization target includes at least one of novelty or diversity, the measurement factor of novelty includes exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

[0278] In one possible implementation, the optimization objective of the policy optimization algorithm includes a reward function in multiple dimensions for updating the test model.

[0279] In one possible implementation, the multi-dimensional reward function includes quality, diversity, and novelty. Novelty controls the visibility of the output test questions, while diversity indicates that the output test questions cover a variety of knowledge points. In other words, the test model can be optimized to output novel or diverse test questions based on the test question's visibility and knowledge point content.

[0280] In one possible implementation, the processing module 1002 is specifically configured to: select at least one test question for the user from a test question bank using a test model; and perform reinforcement learning on the test model using the user's answer records for the at least one test question to obtain a test model after reinforcement learning.

[0281] In one possible implementation, the acquisition module 1001 is further used to: obtain the user's answer record for at least one test question from the test question bank; or, receive online answer data obtained by the user performing operations on at least one test question, and obtain the user's answer record for at least one test question based on the online answer data.

[0282] In one possible embodiment, the test model also includes a relationship-aware aggregator, the input of the relationship-aware aggregator includes at least one of a premise graph or a correlation graph, and the relationship-aware aggregator is used to obtain an embedded representation of the relationship between knowledge points or between test questions and knowledge points based on at least one of the premise graph or the correlation graph. The premise graph is used to represent the chronological relationship between knowledge points in the input test questions, and the correlation graph is used to represent the correlation between test questions and knowledge points; the state encoder is used to extract the association relationship between test questions and knowledge points based on the data output by the relationship-aware aggregator, and generate state codes based on the association relationship.

[0283] In one possible embodiment, the reward functions of multiple dimensions include functions corresponding to at least one of quality reward, diversity reward, or novelty reward. The quality reward is determined based on the output accuracy of the test model in the test question bank. The diversity reward is determined based on whether the test question selected by the test model from the test question bank currently adds new knowledge points compared with the test question selected from the test question bank at least once before. The novelty reward is determined based on whether the test question selected by the test model from the test question bank currently is a hot question. The questions in the test question bank are divided into hot questions and non-hot questions. The historical selection number of hot questions is higher than the historical selection number of non-hot questions.

[0284] In one possible implementation, the test question bank is divided into a candidate set and a meta-question set. The test questions selected by the user are test questions in the candidate set, and the test questions selected by the user are also used to train the test model. The meta-question set is used to calculate rewards in multiple dimensions.

[0285] Reinforcement learning includes a training phase and a testing phase. The candidate set is used to train the test model in the training phase, and the meta-problem set is used to calculate rewards in multiple dimensions in the testing phase.

[0286] In one possible implementation, the aforementioned reinforcement learning includes: in a testing phase, selecting at least one test question from a candidate set through a test model; after receiving a user's response to the at least one test question, obtaining an ability evaluation value based on the user's response to the at least one test question; the ability evaluation value is used to indicate the degree of accuracy of the user's answers to the test questions that have been screened for the user; in a verification phase, calculating rewards in multiple dimensions based on the ability evaluation value and the verification set, updating the test model based on the rewards in the multiple dimensions, and obtaining the test model after the current iterative learning.

[0287] In one possible implementation, the processing module 1002 is specifically used to: perform supervised learning using a test question bank to obtain a test model, wherein the test question bank includes labeled data that annotates diversity and / or novelty, and the supervised learning includes using the labeled data to perform supervised learning on the initial test model to obtain a trained test model.

[0288] In a possible implementation, the state encoder is specifically configured to obtain the difference between the input test questions and the ability evaluation value corresponding to at least one user to generate a state code.

[0289] See also Figure 11 , a structural schematic diagram of another online testing device provided by this application is described as follows.

[0290] The online testing device may include a processor 1101 and a memory 1102. The processor 1101 and the memory 1102 are interconnected via a circuit. The memory 1102 stores program instructions and data.

[0291] The memory 1102 stores the aforementioned Figure 4-Figure 9 The program instructions and data corresponding to the steps in .

[0292] Processor 1101 is used to execute the above Figure 4-Figure 9 The method steps are performed by the online testing device shown in any embodiment.

[0293] Optionally, the online testing device may further include a transceiver 1103 for receiving or sending data.

[0294] Optionally, the aforementioned Figure 11 The online test device shown in FIG is a chip.

[0295] The embodiment of the present application also provides an online test device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is used to execute the aforementioned Figure 4-Figure 9 The method steps are performed by the online testing device shown in any embodiment.

[0296] The present application also provides a computer-readable storage medium in which a program for generating a vehicle speed is stored. When the program is run on a computer, the computer executes the above-mentioned Figure 4-Figure 9 The illustrated embodiments describe steps in a method.

[0297] The present application also provides a digital processing chip. The digital processing chip integrates circuitry and one or more interfaces for implementing the aforementioned processor 1101 or the functions of processor 1101. When the digital processing chip integrates memory, it can perform the method steps of any one or more of the aforementioned embodiments. When the digital processing chip does not integrate memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the steps of the aforementioned embodiments based on program code stored in the external memory.

[0298] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the aforementioned Figure 4-Figure 9 The illustrated embodiment describes the steps performed by the layout strategy generating device in the method.

[0299] Optionally, the aforementioned memory or storage unit may be a storage unit within the chip, such as a register, a cache, etc., and the memory or storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0300] Specifically, the aforementioned processing unit or processor may be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0301] For example, see Figure 12 , Figure 12 This is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 120. NPU 120 is mounted on the host CPU as a coprocessor and is assigned tasks by the host CPU. The core of the NPU is arithmetic circuit 1203, which is controlled by controller 1204 to extract matrix data from memory and perform multiplication operations.

[0302] In some implementations, arithmetic circuit 1203 includes multiple processing engines (PEs). In some implementations, arithmetic circuit 1203 is a two-dimensional systolic array. Arithmetic circuit 1203 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, arithmetic circuit 1203 is a general-purpose matrix processor.

[0303] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 1202 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 1201 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 1208.

[0304] Unified memory 1206 is used to store input and output data. Weight data is directly transferred to weight memory 1202 through direct memory access controller (DMAC) 1205. Input data is also transferred to unified memory 1206 through DMAC.

[0305] The bus interface unit (BIU) 1210 is used for interaction between the AXI bus, the DMAC, and the instruction fetch buffer (IFB) 1209 .

[0306] The bus interface unit 1210 (BIU) is used for the instruction fetch memory 1209 to obtain instructions from the external memory, and is also used for the storage unit access controller 1205 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0307] DMAC is mainly used to move input data in the external memory DDR to the unified memory 1206 or move weight data to the weight memory 1202 or move input data to the input memory 1201.

[0308] The vector calculation unit 1207 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0309] In some implementations, the vector calculation unit 1207 can store the processed output vector to the unified memory 1206. For example, the vector calculation unit 1207 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1203, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1207 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1203, for example, for use in a subsequent layer in a neural network.

[0310] An instruction fetch buffer 1209 connected to the controller 1204 is used to store instructions used by the controller 1204;

[0311] Unified memory 1206, input memory 1201, weight memory 1202, and instruction fetch memory 1209 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0312] The operations of each layer in the recurrent neural network can be performed by the operation circuit 1203 or the vector calculation unit 1207.

[0313] The processor mentioned in any of the above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more processors for controlling the above Figure 4-Figure 9 The method is performed by programming an integrated circuit.

[0314] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0315] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0316] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0317] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0318] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

Claims

1. An online testing method, characterized in that: include: Obtaining a test question bank, wherein the test question bank includes a plurality of test questions; A test model is obtained based on the test question bank and the strategy optimization algorithm, and the test model is used to select at least one test question from the test question bank. The test model includes a state encoder and a recommender, and the state encoder is used to generate state codes based on the differences between test questions. The recommender is used to output test questions based on the state codes and optimization targets, and the optimization targets include at least one of novelty or diversity, wherein the measurement factor of novelty includes exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

2. The method according to claim 1, characterized in that The optimization objective of the policy optimization algorithm includes a reward function for updating the test model.

3. The method according to claim 2, characterized in that The reward function is a multi-dimensional reward function, and the multi-dimensional reward function includes at least two of quality, diversity, and novelty.

4. The method according to claim 3, characterized in that The reward functions of the multiple dimensions include quality rewards, diversity rewards and / or novelty rewards. The quality reward is determined by the output accuracy of the test model when testing in the test question bank. The diversity reward is determined by whether new knowledge points are added between the test question currently selected from the test question bank by the test model and the test question selected from the test question bank at least once before. The novelty reward is determined by whether the test question currently selected from the test question bank by the test model belongs to a popular question. The test questions in the test question bank are divided into the popular questions and non-popular questions. The historical selection number of the popular question is higher than the historical selection number of the non-popular question.

5. The method according to any one of claims 1 to 4, characterized in that The test model further includes a relationship-aware aggregator, wherein the input of the relationship-aware aggregator includes at least one of a premise graph or a correlation graph, and the relationship-aware aggregator is used to obtain an embedded representation of the relationship between knowledge points or an embedded representation of the relationship between test questions and knowledge points based on the input, wherein the premise graph is used to represent the sequential relationship between knowledge points in the input test questions, and the correlation graph is used to represent the correlation between the test questions and the knowledge points; The state encoder is used to extract the association relationship between the test questions and the knowledge points based on the data output by the relationship-aware aggregator, and generate the state code based on the association relationship.

6. The method according to any one of claims 1 to 4, characterized in that The test model is obtained based on the test question bank and the strategy optimization algorithm, including: Selecting at least one test question from the test question bank using a test model; Reinforcement learning is performed on the test model through the answer record of the at least one test question to obtain a test model after reinforcement learning.

7. The method according to claim 6, characterized in that The method further comprises: Obtaining an answer record of the at least one test question from the test question database; Alternatively, online answer data obtained by the user operating on the at least one test question is received, and an answer record of the at least one test question is obtained based on the online answer data.

8. The method according to any one of claims 1 to 4, characterized in that The test question bank is divided into a candidate set and a meta-question set. The test questions selected by the test model are test questions in the candidate set, and the test questions selected by the test model are also used to train the test model. The meta-question set is used to calculate rewards in the multiple dimensions. The policy optimization algorithm includes reinforcement learning, which includes a testing phase and a verification phase. The candidate set is used to train the testing model in the testing phase, and the meta-problem set is used to calculate rewards in the multiple dimensions in the verification phase.

9. The method according to claim 8, characterized in that The reinforcement learning includes: In the testing phase, at least one test question is selected from the candidate set using the test model, and after receiving a response to the at least one test question, an ability evaluation value is obtained based on the response to the at least one test question, the ability evaluation value being used to indicate the degree of accuracy in answering the test question; In the verification phase, the rewards of the multiple dimensions are calculated based on the capability evaluation value and the verification set, and the test model is updated based on the rewards of the multiple dimensions to obtain the test model after the current iterative learning.

10. The method according to any one of claims 1 to 4, characterized in that The test model is obtained based on the test question bank and the strategy optimization algorithm, including: The test question bank is used to perform supervised learning to obtain the test model, wherein the test question bank includes label data annotating the diversity and / or the novelty, and the supervised learning includes using the label data to perform supervised learning on the initial test model to obtain the trained test model.

11. The method according to any one of claims 1 to 4, characterized in that The state encoder is specifically used to obtain the difference between the input test questions and at least one ability evaluation value to generate the state code.

12. An online testing device, characterized in that: include: An acquisition module, configured to acquire a test question bank, wherein the test question bank includes a plurality of collected test questions; A processing module is used to obtain a test model based on the test question bank and the strategy optimization algorithm, and the test model is used to select at least one test question from the test question bank. The test model includes a state encoder and a recommender, and the state encoder is used to obtain the difference between the input test questions to generate a state code. The recommender is used to output the test question according to the state code and the optimization target, and the optimization target includes at least one of novelty or diversity, wherein the measurement factor of novelty includes exposure rate, and the measurement factor of diversity includes whether there are new knowledge points.

13. The device according to claim 12, characterized in that The optimization objective of the policy optimization algorithm includes a reward function for updating the test model.

14. The device according to claim 13, characterized in that The reward function is a multi-dimensional reward function, and the multi-dimensional reward function includes at least two of quality, diversity, and novelty.

15. An online testing device, characterized in that: The method comprises a processor coupled to a memory, wherein the memory stores a program, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 11 is implemented. 16 . A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the method according to claim 1 .

17. A device, characterized in that The method comprises a processing unit and a communication interface, wherein the processing unit obtains program instructions through the communication interface, and when the program instructions are executed by the processing unit, the method according to any one of claims 1 to 11 is implemented.

18. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images