Question difficulty prediction method and device, equipment, medium and product

Abstract

The invention provides a question difficulty prediction method and device, equipment, a medium and a product. According to one embodiment of the invention, the method comprises the following steps: obtaining a to-be-predicted question and a question attribute corresponding to the to-be-predicted question; inputting the to-be-predicted question and the question attributes into a difficulty prediction model, obtaining a target difficulty value of the to-be-predicted question output by the difficulty prediction model, the difficulty prediction model being obtained by performing question difficulty sorting on a plurality of training questions in the sample pair through learning, the multiple training questions in the sample pair have the same question attribute and different question difficulties; and determining a target difficulty level of the to-be-predicted question based on the target difficulty value.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment, medium and product for predicting the difficulty of a question. Background Art

[0002] In the field of education, question difficulty is an important indicator for measuring the characteristics of practice questions or test questions, and is crucial for the allocation of personalized learning resources and the setting of practice sequences. In traditional education, the difficulty labels of test questions rely on manual annotation by experts in the field of education. This process is not only time-consuming and laborious, but also has challenges in terms of consistency and accuracy of annotation, because different experts may have different judgments on the difficulty of the same question based on their personal experience and understanding.

[0003] With the development of natural language processing and machine learning, advanced technical means are used to automatically predict the difficulty of questions by analyzing factors such as the text content, sentence structure, and vocabulary complexity of the questions. This method has improved efficiency to a certain extent and attempts to achieve objectivity in difficulty labels.

[0004] However, most of the current difficulty prediction methods only rely on direct analysis of the question text and directly output a numerical value representing the absolute difficulty of the question, and the accuracy of difficulty prediction is insufficient. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present application provides a method, device, equipment, medium and product for predicting the difficulty of a question.

[0006] According to a first aspect of any embodiment of the present application, a method for predicting the difficulty of a question is provided, the method comprising:

[0007] Obtaining a topic to be predicted and a topic attribute corresponding to the topic to be predicted;

[0008] Inputting the to-be-predicted question and the question attribute into a difficulty prediction model, and obtaining a target difficulty value of the to-be-predicted question output by the difficulty prediction model; the difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in a sample pair, and the multiple training questions in the sample pair have the same question attributes and different question difficulties;

[0009] Based on the target difficulty value, a target difficulty level of the to-be-predicted question is determined.

[0010] According to a second aspect of any embodiment of the present application, a device for predicting the difficulty of a question is provided, the device comprising:

[0011] An acquisition module, used to acquire a topic to be predicted and a topic attribute corresponding to the topic to be predicted;

[0012] A prediction module, used for inputting the to-be-predicted question and the question attribute into a difficulty prediction model, and obtaining a target difficulty value of the to-be-predicted question output by the difficulty prediction model; the difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in a sample pair, and the multiple training questions in the sample pair have the same question attributes and different question difficulties;

[0013] A determination module is used to determine the target difficulty level of the to-be-predicted question based on the target difficulty value.

[0014] According to a third aspect of any embodiment of the present application, there is provided an electronic device, including:

[0015] processor;

[0016] a memory for storing processor-executable instructions;

[0017] The processor implements the method described in any embodiment of the present application by running the executable instructions.

[0018] According to a fourth aspect of any embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the method described in any embodiment of the present application is implemented.

[0019] According to a fifth aspect of any embodiment of the present application, a computer program product is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method described in any embodiment of the present application is implemented.

[0020] The technical solution provided by this application may have the following beneficial effects:

[0021] According to the above embodiments, by obtaining the question to be predicted and the question attributes corresponding to the question to be predicted, the question to be predicted and the question attributes are input into the difficulty prediction model, the target difficulty value of the question to be predicted output by the difficulty prediction model is obtained, and based on the target difficulty value, the target difficulty level of the question to be predicted is determined. Since the difficulty prediction model has learned to sort the difficulty of multiple training questions with the same question attributes but different difficulties in the sample pairs, it can fully consider the impact of the question attributes of the questions on the question difficulty, and better understand the relative difficulty of different questions with the same question attributes, so that the predicted target difficulty value and target difficulty level are more reasonable and accurate.

[0022] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in the specification and constitute a part of this application, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0024] Figure 1 is a flowchart of a method for predicting the difficulty of a question according to an exemplary embodiment of the present application;

[0025] Figure 2 is a structural schematic diagram of a prediction system according to an exemplary embodiment of the present application;

[0026] Figure 3 is a flowchart of another method for predicting the difficulty of a question according to an exemplary embodiment of the present application;

[0027] Figure 4 is a structural schematic diagram of an electronic device according to an exemplary embodiment of the present application;

[0028] Figure 5 It is a block diagram of a device for predicting question difficulty according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0031] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0032] The current difficulty prediction method only uses the semantic information of the question text to predict the difficulty of the question. However, if the attribute information of the question is not taken into account, for example, there is obviously a significant difference in difficulty between the math questions for the first grade of elementary school and the math questions for the first grade of high school. Ignoring the attribute information of the question will lead to unreasonable difficulty prediction results. In addition, directly predicting the absolute difficulty of a question will lead to insufficient accuracy in the prediction of the question difficulty.

[0033] In order to solve the above problems, the present application proposes a method for predicting the difficulty of a question. To further illustrate the present application, the following embodiments are provided:

[0034] See also Figure 1 , Figure 1 This is a flowchart of a method for predicting the difficulty of a question according to an exemplary embodiment of the present application. The method for predicting the difficulty of a question can be applied to scenarios such as online learning and educational assessment. The method can be executed by a prediction system, and the prediction system can be applied to electronic devices such as learning machines, mobile phones, computers, digital broadcast terminals, messaging devices, tablet devices, and personal digital assistants.

[0035] like Figure 1 As shown, the topic prediction method may include the following steps:

[0036] Step 102: Obtain the topic to be predicted and the topic attribute corresponding to the topic to be predicted.

[0037] In this step, the user can input the topic to be predicted and the topic attribute corresponding to the topic to be predicted in the prediction system by text input, taking a photo and uploading it. The prediction system obtains the topic to be predicted and the topic attribute input by the user.

[0038] Among them, the question to be predicted is the question text for question difficulty prediction, which is used to provide semantic information of the question. Question attributes are additional information related to the question to be predicted, such as: elementary school, junior high school, high school and other educational stages, as well as attribute information such as mathematics, physics, chemistry and other subjects. Question attributes are used to provide necessary contextual information for difficulty prediction. Different combinations of question attributes will lead to significant differences in question difficulty, which helps the model to judge the difficulty of questions more accurately.

[0039] Step 104: Input the to-be-predicted question and question attributes into the difficulty prediction model to obtain the target difficulty value of the to-be-predicted question output by the difficulty prediction model; the difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in the sample pair, and the multiple training questions in the sample pair have the same question attributes but different question difficulties.

[0040] In this step, the prediction system inputs the question to be predicted and the question attributes into the difficulty prediction model, and the difficulty prediction model predicts the target difficulty value of the question to be predicted based on the input information.

[0041] The difficulty prediction model is a trained machine learning model that is used to predict the difficulty value of a question based on the input question and question attributes. The difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in the sample pair.

[0042] A sample pair is a combination of at least two training questions with the same question attributes but different question difficulties. The multiple training questions in the sample pair have the same question attributes but different question difficulties. The sample pair includes multiple training questions, multiple question attributes corresponding to the multiple training questions, and difficulty value labels corresponding to the multiple training questions. The difficulty value labels are used to reflect the question difficulty of the training questions.

[0043] The difficulty prediction model gradually masters the ability to predict question difficulty based on questions and question attributes by learning the relationship between training questions, question attributes and question difficulty in sample pairs.

[0044] The training questions are question texts with difficulty value labels used to train the difficulty prediction model. The training questions can be selected from existing educational resources, and the difficulty value labels are determined by manually labeling the training questions and the question attributes corresponding to the training questions.

[0045] The target difficulty value is a quantitative indicator that reflects the difficulty of the question to be predicted. It is a difficulty value predicted based on objective factors such as the question to be predicted and question attributes, and is used to indicate the relative difficulty of the question to be predicted.

[0046] In one embodiment, the topic attributes may include textbook version, grade, and knowledge point. By including textbook version, grade, and knowledge point in the topic attributes, the difficulty prediction model fully considers the influence of the textbook version, grade, and knowledge point of the topic on the difficulty of the topic during the training and prediction process, ensuring that the target difficulty value and target difficulty level are more in line with the actual situation.

[0047] Step 106: Based on the target difficulty value, determine the target difficulty level of the question to be predicted.

[0048] In this step, the prediction system divides the questions to be predicted into corresponding target difficulty levels based on the target difficulty value output by the difficulty prediction model, and determines the target difficulty level of the questions to be predicted.

[0049] Among them, the target difficulty level is the difficulty range to which the target difficulty value of the question to be predicted belongs. The target difficulty level can be a specific difficulty level such as simple, medium, difficult, etc., or it can be the difficulty ranking of the question to be predicted, or it can be a pre-set specific difficulty value range. The embodiment of the present application does not limit this.

[0050] The method for predicting the difficulty of a question in this embodiment obtains the question to be predicted and the question attributes corresponding to the question to be predicted, inputs the question to be predicted and the question attributes into a difficulty prediction model, obtains a target difficulty value of the question to be predicted output by the difficulty prediction model, and determines a target difficulty level of the question to be predicted based on the target difficulty value. Since the difficulty prediction model has learned to sort the difficulty of multiple training questions with the same question attributes but different difficulties in a sample pair, it can fully consider the impact of the question attributes of the question on the question difficulty, better understand the relative difficulty of different questions with the same question attributes, and make the predicted target difficulty value and target difficulty level more reasonable and accurate.

[0051] In the above-mentioned embodiments, it is introduced that the difficulty prediction model is used to predict a reasonable and accurate target difficulty value and a corresponding target difficulty level based on the predicted questions and question attributes. In the following embodiments, the training process of the difficulty prediction model will be described in more detail and can be applied to any of the above embodiments.

[0052] In one embodiment, the prediction system obtains multiple question data, each of which includes the question text of each question, the subject to which each question belongs, the textbook version, grade, knowledge points and other question attributes and difficulty value labels.

[0053] In multiple question data, questions with the same question attributes and different difficulty value labels are combined in pairs to generate a sample pair, each sample pair includes a training question with a relatively high difficulty value and a training question with a relatively low difficulty value.

[0054] As described above, by obtaining multiple question data, each question data includes the question attributes and difficulty value labels of each question, and in the multiple question data, the questions with the same question attributes and different difficulty value labels are combined in pairs to generate sample pairs, which can increase the diversity of training data and make the sample pairs fully cover questions of different difficulty levels.

[0055] In one embodiment, the prediction system inputs each training question in the sample pair and the question attribute corresponding to each training question into the basic model to obtain the predicted difficulty value of each training question output by the basic model.

[0056] The prediction system ranks the multiple training questions according to the multiple predicted difficulty values ​​corresponding to the multiple training questions, and determines the predicted difficulty ranking of the multiple training questions. The difference between the predicted difficulty ranking and the actual difficulty ranking is determined by comparing the predicted difficulty ranking with the position change of the training questions in the actual difficulty ranking of the multiple training questions.

[0057] The prediction system calculates the loss value of loss functions such as mean square error and cross entropy based on the difference between the predicted difficulty ranking and the actual difficulty ranking. The difference between the two rankings is proportional to the size of the loss value. The smaller the difference between the two rankings, the smaller the loss value.

[0058] Exemplarily, the prediction system may calculate the loss value according to the following loss function formula 1:

[0059]

[0060] Among them, loss(θ) represents the loss value, σ represents the activation function, and r θ represents the basic model, θ is the parameter of the basic model, x represents the same topic attribute of multiple training topics in the sample pair, K represents the combination arrangement of the sample pair, E represents the expectation, y w is a relatively high-difficulty training question, y l It is a training question with a relatively low difficulty value. The training goal is to train the question y w The predicted difficulty values ​​are ranked in the training topic y l The predicted difficulty value is in front.

[0061] The prediction system adopts optimization algorithms such as gradient descent, stochastic gradient descent, and adaptive moment estimation (Adaptive Moment Estimation, Adam). It uses the training method of the reward model to minimize the loss value to train the basic model. It adjusts the parameters of the model to reduce the difference between the predicted difficulty ranking and the actual difficulty ranking until the preset training stop condition is met. The training process ends and the final difficulty prediction model is obtained.

[0062] During the model training process, in order to prevent the model from overfitting, the prediction system can also use a validation set including different sample pairs to evaluate the performance of the basic model and calculate evaluation indicators such as the accuracy and recall rate of the basic model.

[0063] The predicted difficulty value is a quantitative indicator of the difficulty of the training questions given by the basic model. The predicted difficulty ranking is used to reflect the relative difficulty relationship between multiple training questions as considered by the basic model. The real difficulty ranking is used to reflect the real relative difficulty relationship between multiple training questions.

[0064] The basic model is a model for building a difficulty prediction system, which can be a large language model (LLM). The large language model has strong text comprehension, task solving and reasoning capabilities, and can predict the difficulty of questions more accurately.

[0065] The loss value is used to reflect the degree of difference between the predicted difficulty ranking of the model and the actual difficulty ranking, and is used to guide the adjustment direction of the model parameters.

[0066] The training stop condition is the condition for stopping the training of the basic model, for example: reaching the specified number of iterations, the loss value is less than a certain threshold, the model performance no longer improves on the validation set, etc.

[0067] As described above, by inputting each training question and the corresponding question attributes into the basic model, the predicted difficulty value output by the basic model is obtained, and the predicted difficulty ranking of the multiple training questions is determined according to the multiple predicted difficulty values ​​corresponding to the multiple training questions. Based on the difference between the predicted difficulty ranking and the actual difficulty ranking, the loss value of the loss function is calculated, and the basic model is trained by minimizing the loss value using an optimization algorithm until the preset training stop condition is met, thereby obtaining a difficulty prediction model. Using a contrastive learning method, the model parameters are continuously adjusted according to the difference between the predicted difficulty ranking and the actual difficulty ranking, so that the basic model can gradually learn how to more accurately predict the relative difficulty of the questions, which helps the model to grasp the relative relationship of the difficulty of the questions as a whole, rather than just the absolute difficulty value of a single question.

[0068] In the above-mentioned embodiment, it is introduced that the predicted difficulty ranking is determined by the predicted difficulty values ​​of multiple training questions output by the basic model, and the model is trained according to the difference between the predicted difficulty ranking and the actual difficulty ranking, so that it can learn and understand the relative difficulty of the questions. In the following embodiment, the process of determining the target difficulty level will be described in more detail, and can be applied to any of the above embodiments.

[0069] In one embodiment, the target difficulty level may include a target difficulty grade. The prediction system uses the difficulty prediction model to perform difficulty prediction on a plurality of questions to be predicted, and obtains a plurality of target difficulty values ​​corresponding to the plurality of questions to be predicted.

[0070] The prediction system uses clustering algorithms such as K-means clustering algorithm and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) to perform cluster analysis on multiple target difficulty values, divide the multiple target difficulty values ​​into different difficulty levels, and obtain the target difficulty level corresponding to each question to be predicted.

[0071] Among them, the target difficulty level is the difficulty level of the questions divided according to the target difficulty value. The target difficulty level can be a discrete difficulty level such as easy, medium, difficult, etc., or it can be a continuous difficulty level such as 0.1 to 1.0.

[0072] As described above, by obtaining multiple target difficulty values ​​corresponding to multiple questions to be predicted, a clustering algorithm is used to perform cluster analysis on the multiple target difficulty values ​​to obtain the target difficulty level corresponding to each question to be predicted. The clustering algorithm can classify the questions to be predicted with similar target difficulty values ​​into the same category, thereby clearly dividing accurate and reasonable target difficulty levels.

[0073] In one embodiment, the prediction system uses the number of target difficulty levels as the number of clusters of cluster centers. For example, the number of target difficulty levels is K, and K initial cluster centers are randomly selected from multiple target difficulty values.

[0074] The prediction system uses metrics such as Euclidean distance or Manhattan distance to calculate the distance between the target difficulty value of each question to be predicted and each cluster center. Based on the distance between the target difficulty value of each question to be predicted and each cluster center, each question to be predicted is assigned to the cluster center with the closest distance based on the principle of minimizing distance.

[0075] After the allocation is completed, the prediction system calculates the average value of all target difficulty values ​​in each cluster center, and uses the average value corresponding to each cluster center as the new K cluster centers, so that the cluster centers gradually approach the true distribution of the target difficulty value.

[0076] Based on the new cluster center, multiple target difficulty values ​​are redistributed, and the cluster center update process is repeated until the preset clustering stop condition is met to determine the final cluster center.

[0077] The prediction system assigns different target difficulty levels to each final cluster center based on the average value of all target difficulty values ​​in each final cluster center and other information. The target difficulty level corresponding to each final cluster center is used as the target difficulty level of the to-be-predicted question in each final cluster center.

[0078] The clustering stop condition is the condition for stopping the clustering analysis of multiple target difficulty values, for example: the cluster center no longer changes or changes very little, the preset maximum number of iterations is reached, the clustering quality index no longer decreases significantly, etc.

[0079] As described above, by taking the number of target difficulty levels as the number of clusters, randomly selecting initial cluster centers from multiple target difficulty values, assigning each question to be predicted to the nearest cluster center, taking the average of all target difficulty values ​​in each cluster center as the new cluster center, and allocating multiple target difficulty values ​​based on the new cluster center until the clustering stop condition is met, determining the final cluster center, assigning different target difficulty levels to each final cluster center, and taking the target difficulty level corresponding to each final cluster center as the target difficulty level of the question to be predicted in each final cluster center. By changing the number of clusters, the degree of refinement of the question difficulty grading can be flexibly adjusted to meet the needs of different educational scenarios, and the questions to be predicted that can adapt to different difficulty distributions can all find the final cluster center through iterative calculation to ensure that the difficulty of the questions to be predicted in each cluster center is similar.

[0080] In one embodiment, the target difficulty level may include a target difficulty ranking. The prediction system ranks multiple target difficulty values ​​corresponding to multiple to-be-predicted questions, obtains a list of questions arranged from low to high or from high to low according to the target difficulty values, and generates a target difficulty ranking of multiple to-be-predicted questions.

[0081] Among them, the target difficulty ranking is the difficulty gradient information of multiple questions to be predicted based on the target difficulty value, which is used to intuitively display the difficulty differences between the various questions to be predicted.

[0082] As described above, by sorting multiple target difficulty values ​​corresponding to multiple questions to be predicted, a target difficulty sorting of multiple questions to be predicted is generated. The target difficulty sorting can provide learners with a clearer relative difficulty relationship and help to allocate learning resources more reasonably.

[0083] To further introduce the prediction system and the problem difficulty prediction process, Figure 2 A schematic diagram of the structure of a prediction system is shown. Figure 2 As shown, the prediction system 20 may include: a question acquisition module 201 , a difficulty prediction module 202 , a question difficulty ranking module 203 and a question difficulty clustering module 204 .

[0084] See also Figure 3 , Figure 3 A flowchart of another method for predicting the difficulty of a question is shown. The parking direction detection method can be performed by the prediction system 20. The method may include the following steps:

[0085] Step 302: Obtain multiple topic data.

[0086] In this step, the topic acquisition module 201 acquires a plurality of topic data collected in advance.

[0087] Step 304: Combine questions with the same question attributes and different difficulty value labels in pairs to generate sample pairs.

[0088] In this step, the difficulty prediction module 202 screens and combines multiple question data. Among the multiple question data, questions with the same question attributes such as textbook version, grade and knowledge point but different difficulty value labels are combined in pairs to generate sample pairs, each of which includes two training questions.

[0089] Step 306: Input each training question and the corresponding question attribute into the basic model to obtain the predicted difficulty value output by the basic model.

[0090] In this step, the difficulty prediction module 202 inputs each training question and the question attributes corresponding to each training question into the basic model. The basic model predicts the question difficulty based on the input training questions and question attributes to obtain the predicted difficulty value output by the basic model.

[0091] Step 308: Determine the predicted difficulty ranking of the multiple training questions according to the multiple predicted difficulty values ​​corresponding to the multiple training questions.

[0092] In this step, the difficulty prediction module 202 sorts the multiple training questions according to the multiple predicted difficulty values ​​corresponding to the multiple training questions, determines the predicted difficulty ranking of the multiple training questions, and reflects the prediction results of the basic model on the relative difficulty of the training questions, which is an important basis for the subsequent calculation of the loss value and the training model.

[0093] Step 310: Calculate a loss value of a loss function based on the difference between the predicted difficulty ranking and the actual difficulty ranking of the plurality of training questions.

[0094] In this step, the difficulty prediction module 202 calculates the loss value of the loss function based on the difference between the predicted difficulty ranking and the actual difficulty ranking of the plurality of training questions. The smaller the difference between the predicted difficulty ranking and the actual difficulty ranking, the smaller the loss value.

[0095] Step 312: Use an optimization algorithm to minimize the loss value to train the basic model and obtain a trained difficulty prediction model.

[0096] In this step, the difficulty prediction module 202 uses an optimization algorithm to minimize the loss value to train the basic model until a preset training stop condition is met to obtain a trained difficulty prediction model.

[0097] Step 314: Input the question to be predicted and the question attributes into the difficulty prediction model to obtain the target difficulty value output by the difficulty prediction model.

[0098] In this step, the question acquisition module 201 obtains the question to be predicted input by the user and the question attributes such as the textbook version, grade and knowledge point corresponding to the question to be predicted. The difficulty prediction module 202 inputs the question to be predicted and the question attributes corresponding to the question to be predicted into the difficulty prediction model. The difficulty prediction model predicts the question difficulty based on the input question to be predicted and the question attributes, and obtains the target difficulty value of the question to be predicted output by the difficulty prediction model.

[0099] Step 316: sorting the multiple target difficulty values ​​corresponding to the multiple questions to be predicted to generate a target difficulty ranking.

[0100] In this step, the question difficulty ranking module 203 ranks the multiple target difficulty values ​​corresponding to the multiple questions to be predicted, generates a target difficulty ranking of the multiple questions to be predicted, and can provide intuitive difficulty gradient information.

[0101] Step 318: Use a clustering algorithm to perform cluster analysis on multiple target difficulty values ​​to obtain a target difficulty level.

[0102] In this step, the question difficulty clustering module 204 uses the K-means clustering algorithm to perform cluster analysis on multiple target difficulty values ​​to obtain a target difficulty level corresponding to each question to be predicted.

[0103] The question difficulty clustering module 204 uses the number of target difficulty levels as the number of clusters of cluster centers and randomly selects initial cluster centers from multiple target difficulty values.

[0104] Traverse each target difficulty value and assign it to the nearest cluster center. After each assignment, recalculate the average of all target difficulty values ​​in the cluster center as the new cluster center until the clustering stop condition is met and the final cluster center is determined.

[0105] The question difficulty clustering module 204 assigns different target difficulty levels to each final cluster center, and uses the target difficulty level corresponding to each final cluster center as the target difficulty level of the to-be-predicted question in each final cluster center.

[0106] Figure 4 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. The electronic device may be, for example, a learning machine, a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, a car machine, etc. Figure 4At the hardware level, the electronic device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and may also include hardware required for other services. The processor 402 reads the corresponding computer program from the non-volatile memory 410 into the memory 408 and then runs it, forming a question difficulty prediction device at the logical level. Of course, in addition to the software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0107] Figure 5 is a block diagram of a device for predicting the difficulty of a question according to an exemplary embodiment of the present application. Figure 5 , the device may include: an acquisition module 502, a prediction module 504 and a determination module 506, wherein:

[0108] The acquisition module 502 is used to acquire the topic to be predicted and the topic attribute corresponding to the topic to be predicted;

[0109] The prediction module 504 is used to input the to-be-predicted question and the question attribute into a difficulty prediction model to obtain a target difficulty value of the to-be-predicted question output by the difficulty prediction model; the difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in a sample pair, and the multiple training questions in the sample pair have the same question attributes and different question difficulties;

[0110] The determination module 506 is used to determine the target difficulty level of the to-be-predicted question based on the target difficulty value.

[0111] In one example, the prediction module 504, before being used to input the to-be-predicted question and the question attributes into a difficulty prediction model to obtain a target difficulty value of the to-be-predicted question output by the difficulty prediction model, further includes: inputting each training question and the question attributes corresponding to each training question into a basic model to obtain a predicted difficulty value output by the basic model; determining a predicted difficulty ranking of the multiple training questions based on multiple predicted difficulty values ​​corresponding to the multiple training questions; calculating a loss value of a loss function based on a difference between the predicted difficulty ranking and an actual difficulty ranking of the multiple training questions; and training the basic model by minimizing the loss value using an optimization algorithm until a preset training stop condition is met to obtain the difficulty prediction model.

[0112] In one example, the target difficulty level includes a target difficulty grade; the determination module 506, when used to determine the target difficulty level of the question to be predicted based on the target difficulty value, includes: obtaining multiple target difficulty values ​​corresponding to multiple questions to be predicted; using a clustering algorithm to perform cluster analysis on the multiple target difficulty values ​​to obtain the target difficulty grade corresponding to each question to be predicted.

[0113] In one example, the target difficulty level includes a target difficulty ranking; the determination module 506, when used to determine the target difficulty level of the question to be predicted based on the target difficulty value, includes: sorting multiple target difficulty values ​​corresponding to multiple questions to be predicted, and generating a target difficulty ranking of the multiple questions to be predicted.

[0114] In one example, the prediction module 504, before being used to input each training question and the question attributes corresponding to each training question into the basic model to obtain the predicted difficulty value output by the basic model, further includes: obtaining multiple question data, each question data including the question attributes and difficulty value labels of each question; among the multiple question data, combining the questions with the same question attributes and different difficulty value labels in pairs to generate the sample pairs.

[0115] In one example, the determination module 506, when used to use a clustering algorithm to perform cluster analysis on multiple target difficulty values ​​to obtain the target difficulty level corresponding to each to-be-predicted question, includes: taking the number of levels of the target difficulty level as the number of clusters, and randomly selecting an initial cluster center from the multiple target difficulty values; assigning each to-be-predicted question to the nearest cluster center based on the distance between the target difficulty value of each to-be-predicted question and each cluster center; taking the average value of all target difficulty values ​​in each cluster center as a new cluster center; assigning the multiple target difficulty values ​​based on the new cluster center until a preset clustering stop condition is met, and determining the final cluster center; assigning different target difficulty levels to each final cluster center; and taking the target difficulty level corresponding to each final cluster center as the target difficulty level of the to-be-predicted question in each final cluster center.

[0116] In one example, the topic attributes include textbook version, grade, and knowledge point.

[0117] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0118] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0119] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions. The above instructions can be executed by a processor of the question difficulty prediction device to implement any method as described in the above embodiments.

[0120] The non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and the present application does not limit this.

[0121] In an exemplary embodiment, a computer program product including a computer program / instruction is also provided. The computer program / instruction can be executed by a processor of a question difficulty prediction device to implement any of the methods described in the above embodiments.

[0122] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the inventions claimed herein. The present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0124] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for predicting the difficulty of a question, characterized in that: The method comprises: Obtaining a topic to be predicted and a topic attribute corresponding to the topic to be predicted; Inputting the to-be-predicted question and the question attribute into a difficulty prediction model, and obtaining a target difficulty value of the to-be-predicted question output by the difficulty prediction model; the difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in a sample pair, and the multiple training questions in the sample pair have the same question attributes and different question difficulties; Based on the target difficulty value, a target difficulty level of the to-be-predicted question is determined.

2. The method according to claim 1, characterized in that: Before inputting the to-be-predicted question and the question attribute into a difficulty prediction model to obtain a target difficulty value of the to-be-predicted question output by the difficulty prediction model, the method further includes: Input each training question and the question attribute corresponding to each training question into the basic model to obtain the predicted difficulty value output by the basic model; Determining a predicted difficulty ranking of the plurality of training questions according to a plurality of predicted difficulty values ​​corresponding to the plurality of training questions; Calculating a loss value of a loss function based on a difference between the predicted difficulty ranking and the actual difficulty ranking of the plurality of training questions; The basic model is trained by minimizing the loss value using an optimization algorithm until a preset training stop condition is met, thereby obtaining the difficulty prediction model.

3. The method according to claim 1, characterized in that The target difficulty level includes a target difficulty grade; The step of determining the difficulty level of the to-be-predicted question based on the target difficulty value includes: Obtain multiple target difficulty values ​​corresponding to multiple questions to be predicted; A clustering algorithm is used to perform cluster analysis on the multiple target difficulty values ​​to obtain a target difficulty level corresponding to each question to be predicted.

4. The method according to claim 1, characterized in that: The target difficulty level includes a target difficulty ranking; Determining the target difficulty level of the to-be-predicted question based on the target difficulty value includes: A plurality of target difficulty values ​​corresponding to a plurality of questions to be predicted are sorted to generate a target difficulty sorting of the plurality of questions to be predicted.

5. The method according to claim 2, characterized in that: Before inputting each training topic and the topic attribute corresponding to each training topic into the basic model to obtain the predicted difficulty value output by the basic model, the method further includes: Acquire multiple question data, each of which includes question attributes and difficulty value labels of each question; Among the plurality of question data, questions with the same question attributes and different difficulty value labels are combined in pairs to generate the sample pairs.

6. The method according to claim 3, characterized in that The clustering algorithm is used to perform cluster analysis on multiple target difficulty values ​​to obtain the target difficulty level corresponding to each to-be-predicted question, including: Taking the number of target difficulty levels as the number of clusters, randomly selecting initial cluster centers from the multiple target difficulty values; According to the distance between the target difficulty value of each to-be-predicted question and each cluster center, each to-be-predicted question is assigned to the cluster center closest to the target difficulty value of each to-be-predicted question; Taking the average of all target difficulty values ​​in each cluster center as a new cluster center; Allocating the multiple target difficulty values ​​based on the new cluster center until a preset clustering stop condition is met, and determining a final cluster center; Assign different target difficulty levels to each final cluster center; The target difficulty level corresponding to each of the final cluster centers is used as the target difficulty level of the to-be-predicted question in each of the final cluster centers.

7. The method according to claim 1, characterized in that The topic attributes include textbook version, grade and knowledge point.

8. A device for predicting the difficulty of a question, characterized in that: The device comprises: An acquisition module, used to acquire a topic to be predicted and a topic attribute corresponding to the topic to be predicted; A prediction module, used for inputting the to-be-predicted question and the question attribute into a difficulty prediction model, and obtaining a target difficulty value of the to-be-predicted question output by the difficulty prediction model; the difficulty prediction model is obtained by learning to sort the difficulty of multiple training questions in a sample pair, and the multiple training questions in the sample pair have the same question attributes and different question difficulties; A determination module is used to determine the target difficulty level of the to-be-predicted question based on the target difficulty value.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images