Recommended methods, apparatus, equipment, and storage media for the topic.

Abstract

A method, apparatus, device, and storage medium for recommending questions are disclosed. The method includes: acquiring the current user's answer logs and the current user's mastery level obtained from the answer logs; filtering candidate questions from a question bank based on the current user's mastery level, wherein each question in the question bank has a proximal development interval, and the candidate questions are those corresponding to the proximal development intervals that match the mastery level; inputting the candidate questions into a pre-trained reinforcement learning model, obtaining the recommendation probability of each candidate question based on an optimal recommendation strategy, and selecting the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probabilities as recommended questions and recommending them to the current user. This invention filters candidate questions based on the proximal development intervals of questions, which is beneficial to improving the user's learning efficiency. Simultaneously, the reinforcement learning-based recommendation algorithm learns the optimal recommendation strategy from each user's answer logs, thereby satisfying various business indicators related to question recommendation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a method, apparatus, device and storage medium for recommending topics. Background Technology

[0002] With the continuous development of computer applications and network interconnection technologies, learning various skills through various learning platforms has become an important means for users to improve themselves. In addition to offering a series of learning courses, these platforms also provide users with questions related to the learning content after online study, allowing them to test their understanding of the knowledge points and review the material covered in the course. Furthermore, with the research and development of artificial intelligence technology, intelligent recommendation of educational products has gradually become a mainstream application technology.

[0003] Unlike other industries (such as video and advertising recommendations), educational product recommendations are not based on user preferences, but on user learning problems. The recommendation algorithm, as an "AI teacher," needs to be able to identify students' weak knowledge points and, based on the user's actual ability, assign the most suitable questions to the student to practice, so as to ensure the user's learning effectiveness.

[0004] Therefore, how to build recommendation systems using artificial intelligence algorithms has become an important part of personalized learning in the education industry. Summary of the Invention

[0005] The problem solved by the embodiments of the present invention is to provide a method, apparatus, device and storage medium for recommending questions, which improves the user's learning efficiency while meeting various business indicators related to question recommendation.

[0006] To address the aforementioned problems, this invention provides a question recommendation method, comprising: acquiring the current user's answer log and the current user's mastery level obtained from the answer log; filtering candidate questions from a question bank based on the current user's mastery level, wherein each question in the question bank has a corresponding proximal development interval, the proximal development interval representing the range of question mastery levels that can improve the user's mastery level, and the candidate questions being questions corresponding to the proximal development interval that matches the user's mastery level; inputting the candidate questions into a pre-trained ranking model, the ranking model obtaining the recommendation probability of each candidate question based on an optimal recommendation strategy, and selecting the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions, and recommending the recommended questions to the current user, wherein the ranking model is trained by learning the optimal recommendation strategy from the answer logs of each user.

[0007] Accordingly, this invention also provides a question recommendation device, comprising: an information acquisition module, configured to acquire the current user's answer log and the current user's mastery level obtained from the current user's answer log; a first filtering module, configured to filter candidate questions from a question bank based on the current user's mastery level, wherein each question in the question bank has a corresponding proximal development interval, the proximal development interval being used to represent the range of question mastery levels that can improve the user's mastery level, and the candidate questions being questions corresponding to the proximal development interval that matches the user's mastery level; and a second filtering module, configured to input the candidate questions into a pre-trained ranking model, wherein the ranking model obtains the recommendation probability of each candidate question based on an optimal recommendation strategy, and filters out the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions, and recommends the recommended questions to the current user, wherein the ranking model is trained by learning the optimal recommendation strategy from the answer logs of each user.

[0008] Accordingly, embodiments of the present invention also provide an apparatus including at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the question recommendation method described in the embodiments of the present invention.

[0009] Accordingly, embodiments of the present invention also provide a storage medium storing one or more computer instructions, which are used to implement the question recommendation method described in the embodiments of the present invention.

[0010] Compared with the prior art, the technical solution of the embodiments of the present invention has the following advantages:

[0011] In the question recommendation method provided in this embodiment of the invention, based on the Zone of Proximal Development (ZPD) of each question, questions corresponding to the ZPD that match the current user's mastery level are selected from the question bank as candidate questions. When the user's mastery level of a specific question is within the ZPD corresponding to that question, it indicates that recommending that question can help the user consolidate relevant knowledge, and the recommendation of that question is more meaningful. Therefore, first selecting questions corresponding to the ZPD that match the current user's mastery level as candidate questions is beneficial to improving the user's learning efficiency. At the same time, after obtaining the candidate questions, the candidate questions are input into a pre-trained ranking model. The ranking model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy and selects the recommended questions. The ranking model selects the candidate questions with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions and recommends these recommended questions to the current user. The ranking model learns the optimal recommendation strategy from the user's answer log, which helps to improve the overall effect of question recommendation and thus meet various business indicators related to question recommendation (e.g., improving the user's learning experience, user retention, and business conversion while enabling the user to make the most progress). Moreover, in actual question recommendation, since the input of the ranking model is candidate questions determined using the proximal development interval, recommending these candidate questions is more meaningful. Therefore, combining the proximal development interval and the ranking model can effectively improve the overall effect of question recommendation.

[0012] Optionally, the ranking model is a reinforcement learning model. Since reinforcement learning models are trained with long-term benefits as the reward value (i.e., cumulative reward), and the recommendation of questions needs to consider the long-term benefits of users (e.g., long-term learning effect, long-term user retention, etc.), the recommendation algorithm based on reinforcement learning can learn the optimal recommendation strategy from the user's answer log. Moreover, due to the robustness and scalability of the algorithm itself, reinforcement learning models are more suitable for engineering construction and iteration. Therefore, reinforcement learning models have stronger applicability. Attached Figure Description

[0013] Figure 1 This is a flowchart of an embodiment of the method recommended in the title of this invention;

[0014] Figure 2 This is a schematic diagram of the network structure of a reinforcement learning model according to an embodiment of the title recommendation method of this invention;

[0015] Figure 3 This is a flowchart of an embodiment of a training method for a reinforcement learning model;

[0016] Figure 4This is a functional block diagram of an embodiment of the device recommended in the title of this invention;

[0017] Figure 5 This is a hardware structure diagram of a device provided in an embodiment of the present invention. Detailed Implementation

[0018] As the background technology shows, how to build recommendation systems using artificial intelligence algorithms has become an important part of personalized learning in the education industry.

[0019] Existing question recommendation technologies mainly include recommendations based on similar questions, recommendations based on weak knowledge points, and recommendations based on students' ability levels.

[0020] Similarity-based recommendation suggests similar questions based on the user's incorrect answers, helping students solidify their understanding through generalization. However, this method only recommends questions associated with known incorrect answers. If a user has few or no incorrect answers, the number of recommended questions is limited. Furthermore, recommending questions related to previously answered incorrectly is not scientifically sound. Educational products should prioritize recommending questions that help students improve more quickly, rather than those they haven't mastered and continue to answer incorrectly. In other words, recommendation algorithms should first consider providing questions of appropriate difficulty for different users to improve learning efficiency. If users are consistently recommended questions they haven't mastered, the overall learning content will be too difficult, ultimately leading to a dislike of learning.

[0021] Recommendation based on weak knowledge points suggests relevant questions based on the user's weak knowledge areas, thereby reinforcing those areas. While this algorithm can determine a user's mastery level for each knowledge point, it cannot define the appropriate mastery range for each user. A common approach is to calculate the average mastery level across all users for each knowledge point and then define those below the average as unmastered. However, this method is difficult to apply to every user. Since different users have different abilities and foundations, their mastery ranges for different knowledge points will also vary. Using only the average mastery level as a threshold can easily lead to users performing poorly on certain questions.

[0022] Recommendations based on learning ability levels suggest questions according to different users' learning abilities. For example, users with a good foundation and strong learning ability are recommended more difficult questions, while users with a poor foundation and average learning ability are recommended easier questions. While recommendations based on learning ability levels match users' levels with question difficulty to recommend suitable questions to suitable users, for educational products, recommendations need to consider not only user learning efficiency but also business metrics such as user learning experience, user retention, and commercial conversion.

[0023] Based on the above analysis, there is an urgent need to provide a question recommendation method that can improve users' learning efficiency while meeting various business indicators related to question recommendation.

[0024] To address the aforementioned technical problem, embodiments of the present invention provide a method for recommending exam questions. (See reference...) Figure 1 The diagram shows a flowchart of an embodiment of the method recommended in the title of this invention.

[0025] In this embodiment of the invention, the question recommendation method includes the following basic steps:

[0026] Step S1: Obtain the current user's answer log and the current user's mastery level based on the answer log;

[0027] Step S2: Select candidate questions from the question bank based on the current user's mastery level. Each question in the question bank has a corresponding proximal development interval. The proximal development interval is used to represent the range of question mastery levels that can improve the user's mastery level. The candidate questions are the questions corresponding to the proximal development interval that matches the user's mastery level.

[0028] Step S3: Input the candidate questions into a pre-trained reinforcement learning model. The reinforcement learning model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and selects the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended questions, and recommends the recommended questions to the user. The ranking model is trained by learning the optimal recommendation strategy from the user's answer log.

[0029] In the question recommendation method of this invention, questions corresponding to the proximal development interval that match the user's mastery level are selected from the question bank as candidate questions, which helps improve the user's learning efficiency. At the same time, the candidate questions are input into a pre-trained ranking model. The ranking model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and selects the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions, and recommends the recommended questions to the user. The ranking model learns the optimal recommendation strategy from the user's answer log, which helps to make the overall effect of question recommendation better, thereby meeting the various business indicators related to question recommendation. Moreover, in actual question recommendation, since the input of the ranking model is the candidate questions determined by the proximal development interval, recommending the candidate questions is more meaningful. Therefore, combining the proximal development interval and the ranking model can effectively improve the overall effect of question recommendation.

[0030] To make the above-mentioned objects, features and advantages of the embodiments of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0031] refer to Figure 1 Execute step S1 to obtain the current user's answer log and the current user's mastery level based on the answer log.

[0032] The current user's answer log contains historical answer information, including the actual answer results corresponding to the questions and the predicted probability of the user answering correctly. By obtaining the current user's level of mastery, a first screening is performed to select questions that match the user's level of mastery and are conducive to the user's improvement as candidate questions.

[0033] For example, for any user, the user's answer log is all historical answer information from time 1 to t. The historical answer information includes the question number and user information, including the answer result (correct or incorrect) and the answer time.

[0034] In this embodiment, when recommending questions online, the user's mastery level is updated in real time. Therefore, the user's mastery level information is stored separately online. The user's mastery level information is obtained based on their answer performance. For example, the user's answer performance over the past three days is analyzed to calculate the correct answer rate, thereby obtaining the user's mastery level.

[0035] Continue to refer to Figure 1In step S2, candidate questions are selected from the question bank based on the current user's mastery level. Each question in the question bank has a corresponding proximal development interval, which represents the range of question mastery levels that can improve the user's mastery level. The candidate questions are those corresponding to the proximal development interval that matches the user's mastery level.

[0036] Here, "matching" means that the user's level of mastery falls within the proximal developmental range. It should be noted that the proximal developmental range of a problem can be considered the user's comfortable learning range. Problems that match the user's comfortable learning range are suitable for recommendation for learning, thus avoiding recommending overly easy problems that lose their purpose and recommending overly difficult problems that may cause aversion to learning.

[0037] The questions in the question bank are first filtered by using the proximal developmental intervals (PDIs) of the questions, thereby selecting questions that can improve the user's mastery as candidate questions. When the user's mastery of a specific question falls within the proximal developmental interval corresponding to that question, it means that recommending that question can help the user consolidate relevant knowledge, and the recommendation of that question is more meaningful. Therefore, in this embodiment, by first using the proximal developmental intervals to filter out questions corresponding to the proximal developmental intervals that match the user's mastery as candidate questions, it is beneficial to improve the user's learning efficiency, which is more conducive to improving the user's ability, and avoids recommending overly simple questions to the user, which would lose the purpose of practice, and recommending overly difficult questions to the user, which would cause the user to develop a dislike for learning.

[0038] The proximal development interval that matches the user's level of mastery refers to the user's level of mastery of the question falling within the proximal development interval of the question. For example, if the user's current level of mastery is 70%, and the proximal development interval of the question is 50% to 80%, then the proximal development interval of the question matches the user's level of mastery.

[0039] Understandably, user learning efficiency and user experience both affect various business metrics of the learning platform. It's worth noting that compared to using average mastery as a threshold to determine whether a user has mastered a question, selecting candidate questions based on the proximal developmental range of the question, which is suitable for the user's current mastery level, makes the method of obtaining candidate questions applicable to any user and ensures that the obtained candidate questions can help users consolidate their relevant knowledge.

[0040] Accordingly, before selecting candidate questions from the question bank based on the current user's mastery level, the process also includes: executing step S25 to construct a question bank, so that each question in the question bank obtains a corresponding proximal development interval.

[0041] By first constructing a question bank, each question in the question bank is assigned a corresponding proximal development interval. This allows for the direct selection of questions from the question bank that correspond to the proximal development interval that matches the current user's level of mastery when recommending questions.

[0042] In this embodiment, the proximal development interval of each question is obtained through offline training, thereby configuring the proximal development interval of each question for use when recommending questions online.

[0043] Specifically, the construction of the question bank includes: performing data processing on each question to obtain the proximal development interval corresponding to each question.

[0044] Each problem has a proximal development interval, and for each problem, there is a pair of optimal lower and upper bounds for that proximal development interval. Therefore, by processing the data for each problem, the proximal development interval for each problem can be determined.

[0045] In this embodiment, the data processing includes: acquiring the answer logs of each user. The answer logs of each user are used as samples to subsequently obtain the upper and lower limits of the proximal developmental interval.

[0046] As mentioned above, the answer log contains historical answer information, which includes the actual answer results corresponding to the questions and the predicted probability of the user answering correctly. The predicted probability of the user answering correctly is obtained from the historical answer information of each user. This is to prepare for the subsequent division of the answer data set corresponding to each question into a first data set, a second data set, and a third data set by combining the preset lower limit and preset upper limit of the proximal development interval corresponding to each question.

[0047] It should be noted that the proximal development interval for each question is obtained through offline training. Therefore, during offline training, each user's mastery level is recalculated based on their answer logs. For example, users' answer logs from the past three days can be analyzed to calculate their accuracy rate and thus their mastery level.

[0048] Specifically, when a predictive model is used to predict the probability of a user answering correctly based on the user's ability score and the question parameters, if the user has a good grasp of the question, the predicted probability of answering correctly is high; if the user has a poor grasp of the question, the predicted probability of answering correctly is low. Therefore, it can be assumed that each question has a corresponding proximal development interval. When the user's mastery level is within the proximal development interval of a specific question, the user can quickly master the knowledge points corresponding to the question through learning or guidance. When the user's mastery level is higher than the upper limit of the proximal development interval, the predicted probability of answering correctly is usually high, and the confidence level of the prediction (i.e., the prediction accuracy) is also high. This means that it can be determined that the user can easily master or recall the question, so there is no need to provide more learning or guidance. When the user's mastery level is lower than the lower limit of the proximal development interval, the predicted probability of answering incorrectly is usually high. This means that even with guidance and learning, it is difficult for the user to change their mastery level.

[0049] Each question has a lower and upper limit in its proximal development interval, which are both within the range of 0 to 1. If a user's mastery of a question is higher than the upper limit of the proximal development interval, the question is considered easy and the user is likely to answer correctly. If a user's mastery is lower than the lower limit of the proximal development interval, the question is considered difficult and the user is likely to answer incorrectly. If a user's mastery of a question is within the proximal development interval of that question, then recommending that question can help the user consolidate their knowledge.

[0050] Therefore, in this embodiment, the lower and upper limits of the proximal development interval for any question satisfy the following: for any question, the correlation between a user's answer to the question and the predicted probability of the user answering correctly is lower than the correlation between a user's answer to the question and the predicted probability of the user answering correctly is lower than the correlation between a user's answer to the question and the predicted probability of the user answering correctly is lower than the correlation between a user's answer to the question and the predicted probability of the user answering correctly is lower than the correlation between a user's answer to the question and the predicted probability of the user answering correctly is lower when the user's mastery level is within the proximal development interval of the question. In other words, when a user's mastery level is within the proximal development interval of the question, the prediction model has difficulty accurately predicting the probability of the user answering correctly, which indicates that the user's current mastery level of the question is unstable. Correspondingly, the user's mastery level of the question can be improved through more learning or guidance.

[0051] Specifically, for each question, once the preset lower and upper limits of the proximal development interval for that question are determined, the classification of each data pair in the answer dataset is determined by analyzing the relationship between the predicted probability of a user answering correctly and these preset lower and upper limits. That is, it's determined which of the three datasets (first, second, or third) each data pair should belong to. Furthermore, the actual answer results for each question are considered to subsequently calculate the probability of prediction errors. A prediction error refers to either a high predicted probability of a user answering correctly, but the actual answer is incorrect, or a low predicted probability of a user answering correctly, but the actual answer is correct.

[0052] It should be noted that the user's answer results include two types: correct and incorrect, represented by 1 and 0 respectively. The predicted probability of the user answering correctly is also within the range of 0 to 1 (that is, greater than or equal to 0 and less than or equal to 1). Therefore, the closer the predicted probability of the user answering correctly is to 1, the higher the probability that the user's answer result is correct; the closer the predicted probability of the user answering correctly is to 0, the higher the probability that the user's answer result is incorrect.

[0053] In this embodiment, the data processing further includes: utilizing the answer logs of each user to obtain the answer data set corresponding to the question, wherein the answer data set includes multiple data pairs obtained by each user answering the question at different times, and each data pair includes the predicted probability of the user answering correctly and the user's actual answer to the question; based on the predicted probability of the user answering correctly, and combined with the preset lower limit and preset upper limit of the proximal development interval, the answer data set corresponding to the question is divided into a first data set, a second data set, and a third data set, wherein the predicted probability of the user answering correctly in the first data set is higher than that in the third data set. The following parameters are defined: a preset upper limit value is set, where the predicted probability of a user answering correctly in the third data set is lower than the preset lower limit value, and the predicted probability of a user answering correctly in the second data set is within the proximal development interval of the question. The percentage of correct answers in the first data set, the probability of incorrect predictions in the second data set, and the percentage of incorrect answers in the third data set are calculated and used as the first ratio, second ratio, and third ratio, respectively. A set of preset lower and upper limits that maximizes the result of multiplying the first, second, and third ratios is then obtained and used as the lower and upper limits of the proximal development interval of the question. In other words, by exhaustively enumerating all preset lower and upper limits, a set of preset lower and upper limits that maximizes the result of multiplying the first, second, and third ratios is obtained and used as a pair of optimal lower and upper limits for the proximal development interval.

[0054] In this embodiment, the answer dataset is divided into a first dataset, a second dataset, and a third dataset using preset lower and upper limits of the proximal development interval as the dividing criteria. This allows for the application of different statistical standards to analyze the data in each dataset. Furthermore, when analyzing each dataset, the percentage of correct answers in the first dataset, the probability of incorrect predictions in the second dataset, and the percentage of incorrect answers in the third dataset are calculated separately.

[0055] As mentioned above, when a user's mastery level falls within the proximal developmental interval of a specific question, the user's mastery level is unstable, making it difficult to accurately predict the probability of the user answering correctly. Therefore, for the second dataset, it is necessary to calculate the probability of prediction errors. Specifically, the probability of prediction errors is calculated using formula (4):

[0056]

[0057] Among them, D zpd Ac represents the total number of samples in the second dataset. zpd This represents the number of correctly predicted samples. Accordingly, the formula (1-Ar) is used. Dzpd This gives the probability of a wrong prediction. Here, the number of correctly predicted samples refers to the sum of the number of samples where the user answers correctly and their actual answer is correct, and the number of samples where the user answers incorrectly and their actual answer is incorrect.

[0058] It should be noted that the answer dataset includes multiple data pairs; therefore, each data pair can be considered a sample. It should also be noted that the probability of prediction error in the second dataset should be as high as possible to compress the size of the proximal development interval as much as possible, thereby improving the accuracy of the proximal development interval. This means that the lower and upper limits can be accurately determined, which is beneficial for accurately identifying the interval where the user's mastery is unstable for each question. When the user's mastery is within the proximal development interval, the user's mastery can be changed, so that the user's ability can be improved by recommending questions that match the proximal development interval.

[0059] In this embodiment, calculating the prediction error rate of the second data set includes: determining a baseline value, wherein the baseline value is located within the proximal developmental interval; in the second data set, counting the number of data pairs in which the predicted probability of a user answering correctly is greater than the baseline value, and the number of data pairs in which the predicted probability of a user answering correctly is less than the baseline value, and the number of data pairs in which the actual answer is incorrect, and the number of data pairs in which the predicted probability of a user answering correctly is less than the baseline value, and the number of data pairs in which the actual answer is incorrect, and the sum of the first and second values ​​is divided by the total number of data pairs in the second data set to obtain the probability of accurate prediction in the second data set as the prediction accuracy rate; subtracting the prediction accuracy rate from the first value to obtain the probability of incorrect prediction in the second data set.

[0060] When a user's mastery level falls within the proximal developmental interval corresponding to a specific question, although the user's mastery level for that specific question is unstable, the higher the user's mastery level, the higher the probability of answering the question correctly. Therefore, a benchmark value is used as the standard for judging whether a user answers correctly or incorrectly in the second dataset. When the predicted probability of a user answering correctly is greater than the benchmark value, it can be considered that the user is predicted to answer the question correctly; when the predicted probability of a user answering correctly is less than the benchmark value, it can be considered that the user is predicted to answer the question incorrectly. Therefore, in data pairs where the predicted probability of a user answering correctly is greater than the benchmark value, the number of actual correct answers is the sample number where the user is predicted to answer correctly and the user's actual answer is correct. Similarly, in data pairs where the predicted probability of a user answering correctly is less than the benchmark value, the number of actual incorrect answers is the sample number where the user is predicted to answer incorrectly and the user's actual answer is incorrect.

[0061] It should be noted that the benchmark value should not be too small or too large. If the benchmark value is too small, that is, too close to the preset lower limit of the proximal development interval, it is easy to result in too few first data sets, leading to a large error in data statistics and consequently, an inaccurate probability of prediction errors in the statistical analysis of the second data set. If the benchmark value is too large, that is, too close to the preset upper limit of the proximal development interval, it is easy to result in too few second data sets, which is also likely to lead to an inaccurate probability of prediction errors in the statistical analysis of the second data set. Therefore, the benchmark value needs to be set reasonably. In this embodiment, the benchmark value is half of the sum of the preset lower limit and the preset upper limit of the proximal development interval. Therefore, this benchmark value is located in the middle of the proximal development interval to maximize the statistical accuracy of each data set as much as possible.

[0062] When a user's mastery level is higher than the upper limit of the proximal development interval, the probability of the user answering correctly is usually higher, and the confidence level of the prediction is also higher. Therefore, for the first dataset, it is necessary to calculate the percentage of correct answers. Specifically, the percentage of correct answers is calculated using formula (5):

[0063]

[0064] Among them, Cr Dup The numerator represents the percentage of correct answers, and the denominator represents the number of samples in the first dataset where the true answer is correct (a true correct answer is represented by 1). up This represents the total number of samples in the first data set.

[0065] When a user's mastery level is below the lower limit of the proximal development interval, the predicted probability of the user answering incorrectly is usually high, and the confidence level of the prediction is also high. Therefore, for the third dataset, it is necessary to calculate the percentage of incorrect answers. Specifically, the percentage of incorrect answers is calculated using formula (6):

[0066]

[0067] Among them, Wr Dlow The term represents the percentage of incorrect answers. The numerator represents the number of samples in the third dataset where the true answer is incorrect (the true answer is 0), and the denominator represents the total number of samples in the third dataset.

[0068] The percentage of correct answers in the first dataset and the percentage of incorrect answers in the third dataset should be as large as possible to compress the size of the proximal development interval and improve its accuracy. In summary, the probability of prediction errors in the second dataset, the percentage of correct answers in the first dataset, and the percentage of incorrect answers in the third dataset should all be as large as possible. Therefore, a set of preset lower and upper limits that maximizes the result of multiplying the first, second, and third ratios are used as the lower and upper limits of the proximal development interval for the question.

[0069] In this embodiment, a set of preset lower limit values ​​and preset upper limit values ​​that maximize the value of formula (7) are obtained as the lower limit value and upper limit value of the proximal development interval:

[0070]

[0071] Among them, L i U represents the preset lower limit of the proximal development zone. i Indicates the preset upper limit of the proximal development zone, Cr Dup This represents the first ratio, (1-Ar) Dzpd ) represents the second ratio, Wr Dlow Indicates the third ratio, Ar Dzpd Represents the probability of accurate prediction in the second dataset, 1-Ar DzpdD represents the probability of a prediction error in the second dataset, and D represents the total number of data pairs in the answer dataset. up D represents the total number of data pairs in the first data set. zpd D represents the total number of data pairs in the second data set. low This represents the total number of data pairs in the third data set.

[0072] It should be noted that, based on the predicted probability of users answering correctly, and combined with the preset lower and upper limits of the proximal development interval, the answer data set corresponding to the question is divided into a first data set, a second data set, and a third data set. If the sample size of any data set in the first, second, and third data sets is too small (for example, the sample size of some proximal development intervals is small), it is easy to cause the corresponding percentage of correct answers, the probability of prediction errors, or the percentage of incorrect answers to be inaccurate. This will cause formula (7) to become unstable due to the sample size of some intervals being too large or too small, thereby reducing the accuracy of the proximal development interval. Therefore, |D up |*|D low |*|D zpd | / D3 is used as a factor coefficient, which is the product of the proportion of the number of samples corresponding to each data set. The factor coefficient can play a normalization role. When the three intervals are evenly distributed, the value of the factor coefficient is the largest. Therefore, by adding the factor coefficient to the formula (7), the number of samples in the three intervals can be balanced as much as possible.

[0073] It is understandable that the proximal development interval is one of the intervals, and the intervals on both sides of the proximal development interval are the remaining two intervals, so there are a total of 3 intervals.

[0074] It should be noted that as the number of user answer logs increases or new users join, the proximal development intervals for each question can be updated based on more answer logs, as needed. For example, the proximal development intervals for each question can be updated every three months. The process of updating the proximal development intervals for each question is the same as the steps described above, and will not be repeated here.

[0075] Continue to refer to Figure 1 In step S3, the candidate questions are input into a pre-trained ranking model. The ranking model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and selects the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended questions, and recommends the recommended questions to the current user. The ranking model is trained by learning the optimal recommendation strategy from the answer logs of each user.

[0076] The ranking model, based on the first screening, ranks the recommendation probabilities of each candidate question according to the optimal recommendation strategy, and then performs a second screening to select the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions, which are then recommended to the user. The ranking model can learn the optimal recommendation strategy from the user's answer log, and each recommendation will try to recommend questions that are conducive to the user's progress, which helps to improve the overall effect of question recommendation. This improves the user's learning efficiency and meets various business indicators related to question recommendation. Moreover, since the input of the ranking model is candidate questions determined by the proximal development interval, the recommendation of these candidate questions is more meaningful. Therefore, combining the proximal development interval and the ranking model can effectively improve the overall effect of question recommendation.

[0077] In this embodiment, the ranking model is a reinforcement learning model. Reinforcement learning includes several key elements: agent, state, action, and reward. Reinforcement learning typically focuses on building an agent to determine which actions to take in an environment to maximize long-term reward. In other words, a characteristic of reinforcement learning models is that they are trained using long-term reward as the reward value (i.e., cumulative reward). Since question recommendation needs to consider the user's long-term benefits (e.g., long-term learning effectiveness, long-term user retention), a reinforcement learning-based recommendation algorithm can learn the optimal recommendation strategy from the user's answer logs. Furthermore, due to its inherent robustness and scalability, reinforcement learning models are more suitable for engineering construction and iteration, thus offering greater applicability. In this specific embodiment, agent represents the question recommendation engine, state represents the user's state characteristics, action represents the question recommended by agent to the user each time, and reward represents the user's feedback after being recommended a question (e.g., whether the user's mastery has improved for immediate benefits, and whether the user is retained for long-term benefits). Given the user's historical learning behavior sequence, the recommendation system recommends the next suitable question for the user, which is the action.

[0078] In this embodiment, the ranking model is a reinforcement learning model. (Refer to the reference...) Figure 2 , Figure 2This is a schematic diagram of the network structure of an embodiment of a reinforcement learning model. The reinforcement learning model includes an encoding layer 61, a recurrent network 62, a concatenation layer 63, an activation layer 64, a behavior network 65b, and a target network 65t. When recommending topics online, the target network 65t is used to rank the recommendation probabilities of candidate topics and select the candidate topic with the highest recommendation probability, or the top n candidate topics with the highest recommendation probabilities, as the recommended topic.

[0079] Specifically, the target network has a target policy. Based on the user's state, solving the target policy π ultimately yields the probability distribution (i.e., softmax) of the action space of the problem. The target policy π is expressed by the following formula:

[0080] Among them, s T Represents the user state vector, υ a This represents the action vector, where θ represents the parameters of the target policy π. θ (a|s) represents the probability of recommending the next question a in user state s, and A represents the set of all questions.

[0081] In this embodiment, during the process of obtaining the recommendation probability of each candidate topic based on the optimal recommendation strategy, the ranking model searches for an optimal target strategy π(a|s). This optimal target strategy recommends topics based on the user's state conditions, thereby maximizing the expected cumulative reward.

[0082]

[0083] Where R(τ) represents the cumulative reward gained from actions at all times, τ represents the trajectory formed by actions at all times, and r(s) t ,a t ) represents the state s at a certain time t. t Below, we recommend specific questions a. t The immediate reward received.

[0084] Therefore, before inputting the candidate questions into the pre-trained ranking model, the method further includes: executing S35, training the ranking model using the answer logs of each user. In this embodiment, the training process is explained using a reinforcement learning model as an example.

[0085] Specifically, in conjunction with reference Figure 3 , Figure 3This is a flowchart of an embodiment of a training method for a reinforcement learning model. The reinforcement learning model is trained in the following way: Step S351 is executed, inputting the answer logs of each user into the encoding layer 61, and converting the question number into question embedding vector features.

[0086] The encoding layer 61 serves as the input layer of the reinforcement learning model. At the same time, the encoding layer converts the question numbers into low-dimensional vectors to prepare for subsequent operations in the reinforcement learning model.

[0087] In step S352, the question embedding vector features are input into the recurrent network 62, and the user state features at the current time (i.e., time t+1) are obtained through the recurrent network 62. The user state features include the user state S after time t. t+1 The vector, obtained by acquiring the user state S t+1 This allows us to estimate the probability of recommending the next question based on each state-action pair (s, a). User state feature S t+1 A representation of the user's current state, for example, user state S. t+1 This includes the user's current level of mastery or ability.

[0088] In step S353, a concatenation layer 63 is used to concatenate the user state features output by the recurrent network 62 and the label context obtained through the answer logs. The concatenation layer 63 concatenates the vector of user state St+1 and the label context to form a new feature vector containing all user state features. This concatenated feature vector can then be used for neural network operations to obtain recommendation probabilities. The label context information includes user information, such as user profile data like unique user identifiers. By obtaining the user's label context, the model's feature input can be enriched, improving the model's prediction accuracy.

[0089] In step S354, activation layer 64 is used to perform a nonlinear transformation on the concatenated user state features and label context. This nonlinear transformation makes the reinforcement learning model nonlinear, allowing it to better fit the data and thus improving its training speed. In this embodiment, a rectified linear unit (ReLU) is used for the nonlinear transformation.

[0090] In step S355, the information output from activation layer 64 is input into behavior network 65b and target network 65t respectively. Behavior network 65b is trained to fit the recommendation strategy for the question and serve as the behavior strategy of behavior network 65b. Target network 65t is trained by updating the parameters of the target strategy of target network 65t through policy gradient ascent iteration to obtain the optimal target strategy that maximizes the expected cumulative benefit. In the process of updating the parameters of the target strategy of target network 65t, the behavior strategy is used for correction.

[0091] The behavioral network 65b and the target network 65t share information input into the reinforcement learning model. During training, an off-policy learning approach is used. This means that not only is the target network 65t trained, but the behavioral network 65b is also trained. The behavioral network 65b is used to fit the recommended strategy for the questions, allowing the behavioral policies of the behavioral network to correct data bias in the target policy when training the target network 65t. Specifically, the target network 65t learns from the current user's answer logs using offline policy correction. The answer logs received by the reinforcement learning model are collected from the trajectory set corresponding to previous target policies, not generated by the current target policy. Since the probability distributions of different target policies differ, when learning a new target policy, the target network 65t needs to combine the already learned behavioral network 65b to correct data bias, achieving importance sampling and improving the mismatch in probability distributions between different policies. This reduces the variance of each gradient term on the trajectory, thereby improving the user experience.

[0092] In this embodiment, the parameters of the target policy of the target network 65t are updated using an iterative approach based on the policy gradient and by utilizing the gradient of the expected cumulative reward of the parameters of each target policy. Specifically, the parameters of the target policy are updated using formulas (1) to (3):

[0093]

[0094]

[0095]

[0096] Where θ' represents the updated parameter, and θ represents the parameter before the update. Let λ represent the expected cumulative return, and λ represent the learning rate. Let R(τ) represent the gradient, R(t) represent the cumulative discount benefit, and R(t) represent the benefit generated by the action performed at the current time t' with the discount factor γ, where t is a historical time. μ represents importance weights, which are obtained by calculating the ratio of the recommendation probabilities of the same question obtained by using the target strategy and the action strategy respectively.

[0097] Referring to formula (2), importance sampling μ is added to the policy gradient formula, enabling the question recommendation engine (i.e., the agent) to learn parameters offline using users' historical data. That is, it uses the data generated by the behavioral policy of the behavioral network 65b to train the parameters of the target policy, thereby improving the consistency between the distribution of the target policy and the behavioral policy, and improving the consistency between the expectations of the target policy and the behavioral policy. Referring to formula (3), by adopting a discount factor γ, the question recommendation engine (i.e., the agent) can learn the importance of future returns, thus enabling the question recommendation engine to pay attention to both current returns and future returns.

[0098] Specifically, obtaining the importance sample includes: calculating the ratio of the recommendation probabilities of obtaining the same question using the target strategy and the behavioral strategy respectively; comparing the ratio with the suppression interval; wherein, when the ratio is within the suppression interval, the ratio is used as the importance sample; when the ratio is less than or equal to the lower limit of the suppression interval, the lower limit is used as the importance sample; when the ratio is greater than or equal to the upper limit of the suppression interval, the upper limit is used as the importance sample.

[0099] When the ratio of the recommendation probabilities of the target policy and the behavioral policy for the same question is outside the inhibition interval, it can easily lead to instability in the training process. Therefore, an inhibition interval is used to impose restrictions, and importance sampling is determined based on the comparison results. This maximizes the expected cumulative return while improving the consistency of the distributions of the target policy and the behavioral policy. The lower and upper limits are determined based on the actual data distribution. For example, in one embodiment, the inhibition interval is 0.8 to 1.2.

[0100] Specifically, using expressions Calculation in the same user state s t Below, regarding the same question a t The recommendation probability π obtained using the target strategy θ (a t |s t ) and the recommendation probability β(a) obtained by using the aforementioned behavioral strategy t |s t The ratio is obtained from the ratio of ).

[0101] Accordingly, in this embodiment, during the process of training the reinforcement learning model and using the reinforcement learning model for online question recommendation, the recommendation probability of each candidate question is obtained through the target network, and the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability are selected.

[0102] In this embodiment, the candidate question with the highest recommendation probability is selected as the recommended question. Therefore, recommending the candidate question with the highest probability is the optimal recommendation strategy, thereby significantly improving the overall effect of question recommendation to meet the effect requirements of the question recommendation method.

[0103] It should be noted that, to avoid the behavioral network interfering with the target strategy for generating user states, the backpropagation of gradients by the behavioral network is prevented. It should also be noted that in other embodiments, other ranking models based on optimal recommendation strategies can be used, for example, obtaining the optimal question recommendation sequence through a genetic algorithm.

[0104] Continue to refer to Figure 1 After recommending the recommended questions to the user, the question recommendation method further includes: executing step S4, updating the user's mastery level based on the user's answer results to the recommended questions.

[0105] By updating the user's level of understanding, the user's status is updated accordingly, so that in the next round of question recommendation, the updated user level of understanding can be used to filter out candidate questions and recommended questions.

[0106] Accordingly, embodiments of the present invention also provide a question recommendation device. Figure 4 This is a functional block diagram of an embodiment of the device recommended in the title of this invention.

[0107] The question recommendation device includes: an information acquisition module 10, used to acquire the current user's answer log and the current user's mastery level obtained from the current user's answer log; a first filtering module 20, used to filter candidate questions from the question bank based on the current user's mastery level, wherein each question in the question bank has a corresponding proximal development interval, the proximal development interval is used to represent the range of question mastery levels that can improve the user's mastery level, and the candidate questions are the questions corresponding to the proximal development interval that matches the user's mastery level; and a second filtering module 30, used to input the candidate questions into a pre-trained ranking model, the ranking model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and filters out the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions, and recommends the recommended questions to the current user, the ranking model is obtained by training by learning the optimal recommendation strategy from the answer logs of each user.

[0108] Each question has a corresponding proximal developmental interval (PDI). The first filtering module 20 selects questions from the question bank that correspond to the PDI intervals of each question and are matched to the user's level of mastery. When the user's mastery of a specific question falls within the PDI interval of that question, recommending that question is more meaningful as it helps the user solidify their knowledge. Therefore, selecting questions that match the user's level of mastery as candidate questions improves the user's learning efficiency. Simultaneously, the second filtering module 30 obtains the recommendation probability of each candidate question based on the optimal recommendation strategy and filters out… The candidate questions with the highest recommendation probability, or the top n candidate questions with the highest recommendation probability, are selected as recommended questions and recommended to the user. The ranking model learns the optimal recommendation strategy from the user's answer logs, which helps to improve the overall effect of question recommendation and thus meet various business indicators related to question recommendation (e.g., improving the user's learning experience, user retention, and business conversion while enabling the user to make the most progress). Moreover, since the input of the ranking model is candidate questions determined using the proximal development interval, recommending these candidate questions is more meaningful. Therefore, combining the proximal development interval and the ranking model can effectively improve the overall effect of question recommendation.

[0109] The current user's answer log contains historical answer information, including the actual answer results corresponding to the questions and the predicted probability of the user answering correctly. By obtaining the current user's level of mastery, a first-stage screening is performed to select questions that match the user's level of mastery and are conducive to the user's improvement as candidate questions. For example, for any user, their answer log contains all historical answer information from time 1 to t. This historical answer information includes the question number and user information, such as the answer result (correct or incorrect) and the answer time.

[0110] The question bank is first filtered by using the proximal developmental intervals (PDIs) of the questions to select candidates that can improve the user's mastery. When the user's mastery of a specific question falls within the corresponding PPI, it means that recommending that question can help the user consolidate relevant knowledge, making the recommendation more meaningful. Therefore, in this embodiment, by first using PPIs to filter out candidates that match the user's mastery, it is beneficial to improve the user's learning efficiency and thus better enhance the user's abilities. This also avoids recommending overly simple questions that lose their purpose and recommending overly difficult questions that may cause the user to develop a dislike for learning.

[0111] Accordingly, the question recommendation device also includes a question bank construction module 25, which is used to construct a question bank before selecting candidate questions from the question bank based on the current user's mastery level, so that each question in the question bank obtains a corresponding proximal development interval.

[0112] By first building a question bank, each question in the bank is assigned a corresponding proximal development interval. This allows for the direct selection of questions from the question bank that match the user's current level of proximal development when recommending questions.

[0113] In this embodiment, the question bank construction module 25 is an offline training module. It obtains the proximal development interval of each question and configures the proximal development interval of each question for use when recommending questions online.

[0114] Specifically, the question bank construction module 25 is used to process data for each question to obtain the proximal development interval for each question. Each question has a proximal development interval, and for each proximal development interval, each question has a pair of optimal lower and upper limits. Therefore, by processing data for each question, the proximal development interval for each question is determined.

[0115] In this embodiment, the question bank construction module 25 includes: an answer log collection unit, used to acquire the answer logs of each user. The answer logs of each user are used as samples to subsequently obtain the upper and lower limits of the proximal development interval.

[0116] It should be noted that the proximal development interval for each question is obtained through offline training. Therefore, during offline training, each user's mastery level is recalculated based on their answer logs. For example, users' answer logs from the past three days can be analyzed to calculate their accuracy rate and thus their mastery level.

[0117] Each question has a lower and upper limit in its proximal development interval, both ranging from 0 to 1. If a user's mastery of the question is higher than the upper limit, the question is considered easy, and the user is likely to answer correctly. Conversely, if the user's mastery is lower than the lower limit, the question is considered difficult, and the user is more likely to answer incorrectly. If the user's mastery of the question falls within its proximal development interval, recommending the question is recommended to help the user solidify their knowledge. Therefore, in this embodiment, the lower and upper limits of the proximal development interval for any question satisfy the following condition: when the user's mastery is within the proximal development interval, the prediction model struggles to accurately predict the probability of a correct answer, indicating that the user's current mastery of the question is unstable. Consequently, further learning or guidance can improve the user's mastery of the question.

[0118] In this embodiment, the question bank construction module 25 further includes: an answer data set acquisition unit, used to acquire the answer data set corresponding to the question using the answer logs of each user, wherein the answer data set includes multiple sets of data pairs obtained by each user answering the question at different times, and each set of data pairs includes the predicted probability of the user answering correctly and the user's actual answer to the question; and a data set partitioning unit, used to divide the answer data set into a first data set, a second data set, and a third data set according to the predicted probability of the user answering correctly and in combination with the lower limit and upper limit of the proximal development interval, wherein the predicted probability of the user answering correctly in the first data set is... The probability of a correct answer is higher than the upper limit, the probability of a user answering correctly in the third data set is lower than the lower limit, and the probability of a user answering correctly in the second data set is within the proximal development interval of the question; the calculation unit is used to calculate the percentage of correct answers in the first data set, the probability of incorrect predictions in the second data set, and the percentage of incorrect answers in the third data set, respectively, as the first ratio, the second ratio, and the third ratio; the filtering unit is used to obtain a set of preset lower limit values ​​and preset upper limit values ​​that maximize the result obtained after multiplying the first ratio, the second ratio, and the third ratio, as the lower limit and upper limit values ​​of the proximal development interval of the question.

[0119] When a user's mastery level falls within the proximal developmental interval of a specific question, their mastery is unstable, making it difficult to accurately predict the probability of a correct answer. Therefore, for the second dataset, it is necessary to calculate the probability of prediction errors. Specifically, the probability of prediction errors is obtained using formula (4):

[0120]

[0121] Among them, D zpdAc represents the total number of samples in the second dataset. zpd This represents the number of correctly predicted samples. Accordingly, the formula (1-Ar) is used. Dzpd This yields the probability of a prediction error. Here, the number of correctly predicted samples refers to the sum of the number of samples where the user answers correctly and their actual answer is correct, and the number of samples where the user answers incorrectly and their actual answer is incorrect. It should be noted that the answer dataset includes multiple data pairs; therefore, each data pair constitutes one sample.

[0122] It should also be noted that the probability of prediction error in the second dataset should be as high as possible to compress the size of the proximal development interval as much as possible, thereby improving the accuracy of the proximal development interval. That is, it is possible to accurately determine the lower and upper limits, which is beneficial for accurately determining the interval where the user's mastery is not stable for each question. When the user's mastery is within the proximal development interval, the user's mastery can be changed so that the user's ability can be improved by recommending questions that match the proximal development interval.

[0123] In this embodiment, in the second data set, the number of data pairs in which the predicted probability of a user answering correctly is greater than the benchmark value, and the number of data pairs in which the actual answer is correct, is taken as the first quantity. The number of data pairs in which the predicted probability of a user answering correctly is less than the benchmark value, and the number of data pairs in which the actual answer is incorrect, is taken as the second quantity. The probability of prediction error is obtained as follows: the sum of the first quantity and the second quantity is divided by the total number of data pairs in the second data set to obtain the probability of accurate prediction in the second data set as the prediction accuracy rate. The probability of prediction error in the second data set is obtained by subtracting the prediction accuracy rate from the first quantity.

[0124] In this embodiment, the benchmark value is half the sum of the preset lower limit and the preset upper limit of the proximal development interval. Therefore, this benchmark value is located in the middle of the proximal development interval to maximize the statistical accuracy of each dataset as much as possible.

[0125] When a user's mastery level is higher than the upper limit of the proximal development interval, the probability of the user answering correctly is usually higher, and the confidence level of the prediction is also higher. Therefore, for the first dataset, it is necessary to calculate the percentage of correct answers. Specifically, the percentage of correct answers is obtained through formula (5):

[0126]

[0127] Among them, Cr Dup The numerator represents the percentage of correct answers, and the denominator represents the number of samples in the first dataset where the true answer is correct (a true correct answer is represented by 1). upThis represents the total number of samples in the first data set.

[0128] When a user's mastery level is below the lower limit of the proximal development interval, the predicted probability of the user answering incorrectly is usually high, and the confidence level of the prediction is also high. Therefore, for the third dataset, it is necessary to calculate the percentage of incorrect answers. Specifically, the percentage of incorrect answers is obtained through formula (6):

[0129]

[0130] Among them, Wr Dlow The term represents the percentage of incorrect answers. The numerator represents the number of samples in the third dataset where the true answer is incorrect (the true answer is 0), and the denominator represents the total number of samples in the third dataset.

[0131] The percentage of correct answers in the first dataset and the percentage of incorrect answers in the third dataset should be as large as possible to compress the proximal development interval and improve its accuracy. In summary, the probability of prediction errors in the second dataset, the percentage of correct answers in the first dataset, and the percentage of incorrect answers in the third dataset should all be as large as possible.

[0132] Specifically, for any given problem, the lower and upper limits of the proximal development interval are configured to maximize the value of formula (7):

[0133]

[0134] Among them, L i U represents the preset lower limit of the proximal development zone. i Indicates the preset upper limit of the proximal development zone, Cr Dup This represents the first ratio, (1-Ar) Dzpd ) represents the second ratio, Wr Dlow Indicates the third ratio, Ar Dzpd Represents the probability of accurate prediction in the second dataset, 1-Ar Dzpd D represents the probability of a prediction error in the second dataset, and D represents the total number of data pairs in the answer dataset. up D represents the total number of data pairs in the first data set. zpd D represents the total number of data pairs in the second data set. low This represents the total number of data pairs in the third data set.

[0135] It should be noted that |D up |*|D low |*|D zpd| / D3 is the factor coefficient. The factor coefficient is the product of the proportion of the number of samples corresponding to each data set. The factor coefficient can play a normalization role. When the three intervals are evenly distributed, the value of the factor coefficient is the largest. Therefore, the factor coefficient is added to the formula (7) so that the number of samples in the three intervals can be as balanced as possible.

[0136] The first filtering module 20 performs a first filtering of questions in the question bank based on the proximal developmental interval, thereby selecting questions that can improve the user's mastery as candidate questions. When the user's mastery of a specific question is within the proximal developmental interval of the question, it means that recommending the question can help the user consolidate relevant knowledge, and the recommendation of the question is more meaningful. Therefore, in this embodiment, by first using the proximal developmental interval to filter out questions corresponding to the proximal developmental interval that match the user's mastery as candidate questions, it is beneficial to improve the user's learning efficiency, which is more conducive to improving the user's ability, and avoids recommending overly simple questions to the user, which would lose the purpose of practice, and recommending overly difficult questions to the user, which would cause the user to develop a dislike for learning. Here, the proximal developmental interval that matches the user's mastery means that the user's mastery of the question is located within the proximal developmental interval of the question.

[0137] It should be noted that by selecting candidate questions based on the proximal developmental range of the questions, questions suitable for the user's current level of mastery are selected. This ensures that the method of obtaining candidate questions is applicable to any user and that the obtained candidate questions can help users consolidate their relevant knowledge.

[0138] The ranking model in the second filtering module 30 is used to rank the recommendation probabilities of each candidate question based on the optimal recommendation strategy after the first filtering, and to perform a second filtering to select the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended question, and recommend the recommended question to the current user. The ranking model learns the optimal recommendation strategy from the user's answer log, and each recommendation will try to recommend questions that are more conducive to the user's further learning, which is conducive to the overall effect of question recommendation, thereby improving the user's learning efficiency and meeting the various business indicators related to question recommendation.

[0139] In this embodiment, the candidate question with the highest recommendation probability is selected as the recommended question. Therefore, recommending the candidate question with the highest probability is the optimal recommendation strategy, thereby significantly improving the overall effect of question recommendation to meet the effect requirements of the question recommendation method.

[0140] In this embodiment, the ranking model is a reinforcement learning model. A characteristic of reinforcement learning models is that they are trained using long-term returns as the reward value (i.e., cumulative returns). Since question recommendation needs to consider the user's long-term benefits (e.g., long-term learning effectiveness, long-term user retention), reinforcement learning-based recommendation algorithms can learn the optimal recommendation strategy from the user's answer logs. Furthermore, due to the robustness and scalability of its algorithms, reinforcement learning models are more suitable for engineering construction and iteration; therefore, reinforcement learning models have greater applicability.

[0141] Reference Figure 2 , Figure 2 This is a schematic diagram of the network structure of an embodiment of a reinforcement learning model, which includes an encoding layer 61, a recurrent network 62, a splicing layer 63, an activation layer 64, a behavior network 65b, and a target network 65t.

[0142] When recommending questions online, the target network 65t is used to rank the recommendation probabilities of candidate questions, thereby selecting the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended question. Specifically, the target network has a target policy, and based on the user's state, solving the target policy π ultimately yields the probability distribution of the action space of the questions. The target policy π is expressed by the following formula: Among them, s T Represents the user state vector, υ a Let θ represent the behavior vector, and let π represent the parameters of the target policy π. θ (a|s) represents the probability of recommending the next question a in user state s, and A represents the set of all questions.

[0143] In this embodiment, during the process of obtaining the recommendation probability of each candidate topic based on the optimal recommendation strategy, the ranking model searches for an optimal target strategy π(as). This optimal target strategy recommends topics based on the user's state conditions, thereby maximizing the expected cumulative revenue.

[0144]

[0145] Where R(τ) represents the cumulative reward gained from actions at all times, τ represents the trajectory formed by actions at all times, and r(s) t ,a t ) represents the state s at a certain time t. t Below, we recommend specific questions a. t The immediate benefits gained.

[0146] Accordingly, the question recommendation device further includes a ranking model training module 35, which is used to train the ranking model using the answer logs of each user before inputting the candidate questions into the pre-trained ranking model.

[0147] The reinforcement learning model is trained by learning the optimal recommendation strategy from the user's answer logs. When training the reinforcement learning model, the parameters of the target policy of the target network 65t are updated through policy gradient ascent iteration to obtain the optimal target policy that maximizes the expected cumulative reward. In the process of updating the parameters of the target policy of the target network 65t, the behavioral policies are used for correction, so that when training the target network 65t, the behavioral policies of the behavioral network can be used to correct the data bias of the target policy.

[0148] For a detailed description of the reinforcement learning model, please refer to the corresponding description in the foregoing embodiments, which will not be repeated here.

[0149] In this embodiment, the question recommendation device further includes an update module 40, used to update the user's mastery level based on the user's answers to the recommended questions. By updating the user's mastery level, the user's status is updated accordingly, so that in the next question recommendation process, the updated user mastery level can be used to filter out candidate questions and recommended questions.

[0150] This invention also provides a device that can implement the question recommendation method provided in this invention by loading a program.

[0151] refer to Figure 5 The diagram illustrates the hardware structure of a device according to an embodiment of the present invention. The device in this embodiment includes: at least one processor 01, at least one communication interface 02, at least one memory 03, and at least one communication bus 04.

[0152] In this embodiment, the number of processor 01, communication interface 02, memory 03 and communication bus 04 is at least one, and the processor 01, communication interface 02 and memory 03 communicate with each other through the communication bus 04.

[0153] The communication interface 02 can be an interface of a communication module used for network communication, such as the interface of a GSM module.

[0154] The processor 01 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the method recommended in this embodiment.

[0155] The memory 03 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0156] The memory 03 stores one or more computer instructions, which are executed by the processor 01 to implement the problem recommendation method provided in the foregoing embodiments.

[0157] It should be noted that the aforementioned terminal device may also include other devices (not shown) that may not be essential to understanding the content disclosed in the embodiments of the present invention; given that these other devices may not be essential for understanding the content disclosed in the embodiments of the present invention, the embodiments of the present invention will not describe them one by one.

[0158] This invention also provides a storage medium storing one or more computer instructions for implementing the question recommendation method provided in the foregoing embodiments.

[0159] In the question recommendation method provided in this embodiment, based on the proximal development interval of each question, questions corresponding to the proximal development interval that match the current user's mastery level are selected from the question bank as candidate questions. When the user's mastery level of a specific question is within the proximal development interval corresponding to that question, it indicates that recommending that question can help the user consolidate relevant knowledge, and the recommendation of that question is more meaningful. Therefore, first selecting questions corresponding to the proximal development interval that match the current user's mastery level as candidate questions is beneficial to improving the user's learning efficiency. At the same time, the candidate questions are input into a pre-trained ranking model, which is based on the optimal recommendation strategy. The algorithm obtains the recommendation probability of each candidate question and selects the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended question. The algorithm then recommends the recommended question to the current user. The ranking model learns the optimal recommendation strategy from the user's answer log, which helps to improve the overall effect of question recommendation and thus meet the various business indicators related to question recommendation. Moreover, in the actual recommendation of questions, since the input of the ranking model is the candidate questions determined by the proximal development interval, recommending the candidate questions is more meaningful. Therefore, combining the proximal development interval and the ranking model can effectively improve the overall effect of question recommendation.

[0160] The embodiments of the present invention described above are combinations of elements and features of the present invention. Unless otherwise stated, the elements or features described are optional. Individual elements or features may be practiced without combination with other elements or features. Furthermore, embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some constructions of any embodiment may be included in another embodiment and may be replaced by corresponding constructions of another embodiment. It will be apparent to those skilled in the art that claims in the appended claims that are not expressly referenced to each other may be combined to form embodiments of the present invention, or may be included as new claims in amendments made after the filing of this application.

[0161] Embodiments of the present invention can be implemented by various means, such as hardware, firmware, software, or combinations thereof. In a hardware configuration, the method according to an exemplary embodiment of the present invention can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0162] In firmware or software configuration, embodiments of the present invention can be implemented in the form of modules, processes, functions, etc. Software code can be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor and can send data to and receive data from the processor via various known means.

[0163] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is accorded the widest scope consistent with the principles and novel features disclosed herein.

[0164] While the embodiments of the present invention have been disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A question recommendation method, characterized in that, include: Get the current user's answer log and the current user's mastery level based on the answer log; Candidate questions are selected from the question bank based on the current user's mastery level. Each question in the question bank has a corresponding proximal development interval, which represents the range of question mastery levels that can improve the user's mastery level. The candidate questions are those corresponding to the proximal development interval that matches the user's mastery level. Before selecting candidate questions from the question bank based on the current user's mastery level, the method further includes: constructing a question bank, such that each question in the question bank obtains a corresponding proximal development interval. Constructing the question bank includes: performing data processing on each question to obtain the proximal development interval corresponding to each question; wherein, the data processing includes: acquiring the answer logs of each user; using the answer logs of each user, acquiring the answer data set corresponding to the question, wherein the answer data set includes multiple sets of data pairs obtained after each user answers the question at different times, and each set of data pairs includes the predicted probability of the user answering correctly and the user's actual answer to the question; Based on the predicted probability of a user answering correctly, and combined with the preset lower limit and preset upper limit of the proximal development interval, the question answering data set is divided into a first data set, a second data set, and a third data set. The predicted probability of a user answering correctly in the first data set is higher than the preset upper limit, the predicted probability of a user answering correctly in the third data set is lower than the preset lower limit, and the predicted probability of a user answering correctly in the second data set is within the proximal development interval of the question. The percentage of correct answers in the first data set, the probability of incorrect predictions in the second data set, and the percentage of incorrect answers in the third data set are calculated respectively and used as the first ratio, the second ratio, and the third ratio. Obtain a set of preset lower and upper limits that maximize the result obtained after multiplying the first, second, and third ratios, and use them as the lower and upper limits of the proximal development interval of the problem. The candidate questions are input into a pre-trained ranking model. The ranking model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and selects the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended questions, and recommends the recommended questions to the current user. The ranking model is trained by learning the optimal recommendation strategy from the answer logs of each user.

2. The question recommendation method as described in claim 1, characterized in that, The question recommendation method further includes: updating the current user's mastery level based on the current user's answer results for the recommended questions.

3. The question recommendation method as described in claim 1, characterized in that, Obtain a set of preset lower and upper limits that maximize the value of the following formula, and use them as the lower and upper limits of the proximal development interval: ； Among them, L i U represents the preset lower limit of the proximal development zone. i Indicates the preset upper limit value of the proximal development zone, Cr Dup This represents the first ratio, (1-Ar) Dzpd ) represents the second ratio, Wr Dlow Indicates the third ratio, Ar Dzpd Represents the probability of an accurate prediction in the second dataset, 1-Ar Dzpd D represents the probability of a prediction error in the second dataset, and D represents the total number of data pairs in the answer dataset. up D represents the total number of data pairs in the first data set. zpd D represents the total number of data pairs in the second data set. low This represents the total number of data pairs in the third data set.

4. The question recommendation method as described in claim 1 or 3, characterized in that, Calculating the prediction error rate of the second dataset includes: determining a baseline value, which is located within the proximal development interval; In the second dataset, the number of data pairs in which the predicted probability of a user answering correctly is greater than the baseline value, and the number of data pairs in which the actual answer is correct, is taken as the first quantity; the number of data pairs in which the predicted probability of a user answering correctly is less than the baseline value, and the number of data pairs in which the actual answer is incorrect, is taken as the second quantity. The sum of the first quantity and the first quantity is divided by the total number of data pairs in the second data set to obtain the probability of accurate prediction in the second data set as the prediction accuracy. Subtracting the prediction accuracy from 1 yields the probability of a prediction error in the second dataset.

5. The question recommendation method as described in claim 4, characterized in that, The benchmark value is half the sum of the preset lower limit and the preset upper limit of the proximal development zone.

6. The method according to claim 1, characterized in that, The ranking model is a reinforcement learning model, which includes an encoding layer, a recurrent network, a concatenation layer, an activation layer, an action network, and a target network. The reinforcement learning model is trained in the following manner: Each user's answer log is input into the encoding layer, and the question number is converted into question embedding vector features; The title embedding vector features are input into a recurrent network, and the user state features at the current time are obtained through the recurrent network. A splicing layer is used to splice the user state features output by the recurrent network and the tag context obtained through the question-answering logs. An activation layer is used to perform nonlinear transformations on the concatenated user state features and tag context; The information output from the activation layer is input into the behavior network and the target network respectively. The behavior network is trained to fit the recommendation strategy of the question and serve as the behavior strategy of the behavior network. The target network is trained and the parameters of the target strategy of the target network are updated by the policy gradient ascent iterative method to obtain the optimal target strategy that maximizes the expected cumulative benefit. In the process of updating the parameters of the target strategy of the target network, the behavior strategy is used for correction. The steps of inputting the candidate questions into a pre-trained ranking model, wherein the ranking learning model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and selects the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as the recommended questions, include: obtaining the recommendation probability of each candidate question through the target network, and selecting the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability.

7. The method according to claim 6, characterized in that, In the step of updating the parameters of the target policy of the target network through policy gradient ascent iteration, the parameters of the target policy are updated using formulas (1) to (3): （1）； （2） （3）； in, This indicates the updated parameters. This indicates the parameters before the update. Indicates accumulated expectations Calculate revenue, Indicates the learning rate. Represents the gradient. This indicates the cumulative benefit from the discount. Represents the trajectory formed by actions at all moments. Indicates the use of discount factors In the context of [the previous context], the benefit generated by the action performed at the current time t', where t is the historical time. Importance sampling is indicated by calculating the ratio of the recommendation probabilities of the same item obtained using the target strategy and the behavioral strategy, respectively. Indicate the target strategy, This represents the probability of recommending the next question a given user state s at time t. Represents the state at time t Below, specific questions are recommended. The immediate benefits gained.

8. The method according to claim 7, characterized in that, Obtaining the importance sampling includes: calculating the ratio of the recommendation probabilities of obtaining the same question using the target strategy and the strategy respectively; Compare the ratio with the suppression interval; Specifically, when the ratio is within the suppression interval, the ratio is used as an importance sample; when the ratio is less than or equal to the lower limit of the suppression interval, the lower limit is used as an importance sample; and when the ratio is greater than or equal to the upper limit of the suppression interval, the upper limit is used as an importance sample.

9. A question recommendation device, characterized in that, include: The information acquisition module is used to acquire the current user's answer log and the current user's mastery level based on the answer log; The first filtering module is used to filter candidate questions from the question bank based on the current user's mastery level. Each question in the question bank has a corresponding proximal development interval. The proximal development interval is used to represent the range of question mastery levels that can improve the user's mastery level. The candidate questions are questions that correspond to the proximal development interval that matches the user's mastery level. Before selecting candidate questions from the question bank based on the current user's mastery level, the method further includes: constructing a question bank, such that each question in the question bank obtains a corresponding proximal development interval. Constructing the question bank includes: performing data processing on each question to obtain the proximal development interval corresponding to each question; wherein, the data processing includes: acquiring the answer logs of each user; using the answer logs of each user, acquiring the answer data set corresponding to the question, wherein the answer data set includes multiple sets of data pairs obtained after each user answers the question at different times, and each set of data pairs includes the predicted probability of the user answering correctly and the user's actual answer to the question; Based on the predicted probability of a user answering correctly, and combined with the preset lower limit and preset upper limit of the proximal development interval, the question answering data set is divided into a first data set, a second data set, and a third data set. The predicted probability of a user answering correctly in the first data set is higher than the preset upper limit, the predicted probability of a user answering correctly in the third data set is lower than the preset lower limit, and the predicted probability of a user answering correctly in the second data set is within the proximal development interval of the question. The percentage of correct answers in the first data set, the probability of incorrect predictions in the second data set, and the percentage of incorrect answers in the third data set are calculated respectively and used as the first ratio, the second ratio, and the third ratio. Obtain a set of preset lower and upper limits that maximize the result obtained after multiplying the first, second, and third ratios, and use them as the lower and upper limits of the proximal development interval of the problem. The second filtering module is used to input the candidate questions into a pre-trained ranking model. The ranking model obtains the recommendation probability of each candidate question based on the optimal recommendation strategy, and filters out the candidate question with the highest recommendation probability or the top n candidate questions with the highest recommendation probability as recommended questions, and recommends the recommended questions to the current user. The ranking model is trained by learning the optimal recommendation strategy from the answer logs of each user.

10. A device, characterized in that, It includes at least one memory and at least one processor, the memory storing one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the question recommendation method as described in any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores one or more computer instructions, which are used to implement the question recommendation method as described in any one of claims 1 to 8.

Images