A knowledge point prediction method and system based on reinforcement learning

Abstract

The application discloses a knowledge point prediction method and system based on reinforcement learning. The application constructs a three-section prediction framework of "self-example generation-candidate recall-candidate rearrangement". Firstly, a teacher large language model is used to generate structured analysis and pseudo knowledge point description from the test question itself, forming a self-example enhanced representation independent of external example library, effectively alleviating the difficulty in distinguishing similar knowledge points and the sparsity of stem information. On this basis, a two-stage prediction strategy is adopted: through semantic retrieval, a high-relevance candidate set is quickly recalled from a large-scale knowledge point set, significantly reducing the subsequent reasoning cost and input length pressure; then, a rearrangement module jointly trained by supervised fine-tuning and reinforcement learning based on group relative strategy optimization is used to sort and screen the candidate set, so that the model directly faces the list-level quality index optimization, improves the ability to distinguish confused knowledge points, and ensures the stability of strategy updating.

Description

Technical Field

[0001] This invention relates to the field of intelligent education technology, specifically to a knowledge point prediction method and system based on reinforcement learning. Background Technology

[0002] Knowledge point prediction is crucial in modern intelligent education applications, serving as a foundational step in online question bank construction, student learning analysis, and test question recommendation. The accuracy of the prediction results directly impacts the effectiveness and usability of subsequent functions. Knowledge point prediction refers to automatically identifying and outputting one or more knowledge point tags related to a test question based on its textual content, including the question stem, options, and explanations. Traditional question bank construction typically relies on subject matter experts manually annotating each question. This process demands high professional skills from the annotators and is costly and time-consuming. As question banks continue to expand and question types become increasingly diverse, manual annotation is prone to subjective bias and inconsistencies, affecting the stable maintenance and continuous iteration of the question bank. Therefore, researching and developing accurate automated knowledge point annotation technology is both practically necessary and valuable.

[0003] Early methods often employed feature engineering or shallow classifiers to represent the test text and independently determine the relevance of each knowledge point. With the development of pre-trained language models, more and more research has incorporated deep models, such as BERT, to semantically encode the test text and output knowledge point predictions through multi-label learning frameworks. To further improve performance, some works have begun to attempt to integrate question parsing information and model co-occurrence relationships or hierarchical structures between knowledge points, thereby alleviating the prediction difficulties caused by insufficient question stem information and semantically similar labels.

[0004] While the aforementioned multi-label knowledge point prediction methods have improved the accuracy of automatic labeling to some extent, most of them still rely on discriminative modeling of the surface text of the questions, making it difficult to fully explore the hidden problem-solving logic and deep semantic information of the questions. In scenarios where the question stem information is limited, the knowledge points are semantically similar, or the long-tail distribution is significant, the prediction effect is still limited.

[0005] In recent years, researchers have begun to explore knowledge point prediction methods based on large language models. These methods typically model knowledge point annotation as a semantic judgment or reasoning task directly performed by the large language model. Through carefully designed prompt templates, the model is guided to combine the question stem content, knowledge point definition, and a few examples to determine whether a question involves a specific knowledge point. To further improve annotation performance, some works have introduced example retrieval, chained reasoning, or multi-agent collaboration mechanisms, enabling the model to explicitly deduce the problem-solving process, verify knowledge point constraints, and mitigate the impact of insufficient question stem information to some extent. Overall, these methods focus on the general reasoning capabilities of large language models, reducing reliance on traditional supervised models and providing a more flexible technical path for knowledge point prediction.

[0006] While knowledge point annotation methods based on large language models can improve the understanding and judgment of question semantics through zero-shot inference, example retrieval enhancement, or multi-agent collaboration, they still have certain limitations. Zero-shot methods mainly rely on the model's inherent knowledge and prompt constraints, and their output is prone to fluctuation when question information is insufficient or knowledge points are highly similar. Example enhancement methods require external example libraries, but example coverage and matching are difficult to guarantee stably, and inappropriate examples may introduce noise and reduce prediction consistency. While multi-agent solutions can improve reliability through task decomposition, they typically involve multiple rounds of calls and complex process control, leading to high inference latency and deployment costs. Furthermore, existing works simplify task settings to knowledge point-by-knowledge point judgment or small-scale label verification, and their efficiency and accuracy in large-scale question banks and multi-label set prediction scenarios remain unknown. Summary of the Invention

[0007] To address the above problems, this invention provides a knowledge point prediction method based on reinforcement learning, comprising the following steps: S1. Obtain the test questions for the knowledge points to be predicted; S2. Construct a knowledge point prediction model, which includes a self-example generation module, a knowledge point recall module, and a candidate set rearrangement module. S3. Input the test questions for the knowledge points to be predicted into the knowledge point prediction model, and obtain the prediction results of the knowledge points corresponding to the test questions.

[0008] Preferably, the self-example generation module is used to automatically construct enhanced self-example representations of test questions from the test questions themselves, which can be used in subsequent modules, including: The structured analysis results and simulated knowledge point descriptions of the test questions are obtained through the teacher's large language model; Based on the structured parsing results and the simulated knowledge point descriptions, an enhanced representation from the examples is obtained.

[0009] Preferably, the knowledge point recall module is used to recall a set of candidate knowledge points from the entire set of knowledge points based on the self-example enhanced representation, including: Based on the semantic description of each knowledge point in the complete knowledge point set, the knowledge point vector corresponding to each knowledge point is obtained through a text encoder. Based on the self-example augmented representation, the question vector is obtained through a text encoder; Based on the question vector and the vector of each knowledge point, the relevance between the question and each knowledge point is obtained; Based on relevance, a predetermined number of knowledge points are recalled from the complete set of knowledge points to form a candidate knowledge point set.

[0010] Preferably, the candidate set reordering module is used to generate a sorted knowledge point set based on the self-example augmented representation and the candidate knowledge point set. The candidate set reordering module is optimized through supervised fine-tuning and reinforcement learning, and includes: Based on the candidate knowledge point set and the gold standard knowledge point set, training samples are obtained. The training samples include positive samples, strong negative examples and weak negative examples. The candidate set rearrangement module is fine-tuned under supervision based on the training samples to obtain the initially trained candidate set rearrangement module.

[0011] Preferably, the strong negative examples include: selecting knowledge points from the candidate knowledge point set that have a high similarity to the self-example enhanced representation and do not belong to the gold standard knowledge point set.

[0012] Preferably, the candidate set rearrangement module is obtained through reinforcement learning optimization and includes: Based on the candidate set rearrangement module that has been initially trained, obtain a reference strategy; For the same pre-test question, the candidate knowledge point set is sampled multiple times according to the current strategy to obtain the ranking result within the group; Based on the group ranking results and the gold standard knowledge point set, obtain the relative advantages within the group; Update the current strategy based on the group's relative strengths and reference strategies.

[0013] Preferably, the Plackett-Luce distribution is used to sample the candidate knowledge point set without replacement to obtain an ordered list, which is used as the ranking result within the group.

[0014] This invention also provides a knowledge point prediction system based on reinforcement learning, the system being used to implement the above method, comprising: The acquisition unit is used to obtain the self-example-enhanced representation of the questions to be tested; The building unit is used to build a knowledge point prediction model, which includes a knowledge point recall module and a candidate set rearrangement module. The prediction unit is used to input the self-example augmented representation into the knowledge point prediction model to obtain the knowledge point prediction result corresponding to the question to be tested.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention constructs a three-stage prediction framework of "self-example generation - candidate recall - candidate reordering." First, it utilizes a teacher's large language model to generate structured parsing and realistic knowledge point descriptions from the test questions themselves, forming a self-example-enhanced representation that does not rely on an external example library. This effectively alleviates the difficulty of distinguishing semantically similar knowledge points due to the sparsity of question stem information. Based on this, a two-stage prediction strategy is adopted: semantic retrieval quickly recalls a set of highly relevant candidates from a large-scale knowledge point database, significantly reducing subsequent inference costs and input length pressure; then, a reordering module jointly trained with supervised fine-tuning and reinforcement learning based on group relative policy optimization performs fine-grained sorting and filtering of the candidate set, enabling the model to directly optimize for list-level quality indicators. This improves the ability to distinguish confusing knowledge points while ensuring the stability of policy updates. This method, while ensuring prediction accuracy, also considers scalability and deployment controllability in large-scale question bank scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the model structure according to an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Example 1 This embodiment provides a knowledge point prediction method based on reinforcement learning, the steps of which include: S1. Obtain the test questions for the knowledge points to be predicted.

[0021] S2. Construct a knowledge point prediction model, which includes a self-example generation module, a knowledge point recall module, and a candidate set rearrangement module. like Figure 1 As shown in the figure, this embodiment constructs a knowledge point prediction model based on reinforcement learning. This model can automatically identify the knowledge points involved in test questions and output multi-label sets under large-scale question banks and complex knowledge point systems, while ensuring prediction quality and scalability and deployment controllability. The overall architecture of the model is shown in the figure. This framework is built around three core stages: constructing test question examples, recalling candidate knowledge points, and reordering and optimizing the candidate set. These stages are used to make the implicit problem-solving clues in the test questions explicit, compress the large-scale knowledge point space into a highly relevant candidate set, and further output stable and accurate knowledge point prediction results.

[0022] This embodiment first performs self-parsing processing on the input question, using a large language model to generate structured parsing information highly consistent with the question, and based on this, generates a realistic knowledge point description, thus forming a self-example representation of the question. Compared with existing solutions that rely on external example libraries constructed by experts for prompt enhancement, this invention uses the information generated by the question itself as the example source, which can reduce noise and fluctuations caused by insufficient example coverage and unstable matching, and improve prediction consistency when the question stem information is sparse or the knowledge points are semantically similar. Subsequently, based on the above enhanced representation, this method performs coarse-grained recall in the knowledge point database through text encoding and vector retrieval to obtain a set of candidate knowledge points with high coverage, thereby alleviating the recall difficulties caused by the large scale and long-tail distribution of knowledge points.

[0023] Building upon this foundation, this embodiment introduces a candidate set re-ranking module to refine and filter candidate knowledge points, and employs supervised warm-up and reinforcement learning optimization training strategies to improve ranking quality. Specifically, the reinforcement learning phase utilizes Group Relative Policy Optimization (GRPO), generating multiple candidate ranking results for the same question. Based on set quality metrics, the relative advantages between different rankings are calculated, and stable optimization signals are formed through intra-group comparisons. This iteratively improves the model's ranking strategy, making it directly responsible for the quality of the final multi-label set. Based on this design, this embodiment can achieve efficient recall of knowledge point candidates within a unified process, and aligns the model training objective with the engineering evaluation objective through advantage-based policy optimization, thereby supporting high-quality and efficient knowledge point prediction.

[0024] S3. Input the test questions for the knowledge points to be predicted into the knowledge point prediction model, and obtain the prediction results of the knowledge points corresponding to the test questions.

[0025] (1) Problem Description Let the set of knowledge points be Given a dataset of test questions ,in This refers to the number of questions in the test. Each question... It consists of natural language text and mathematical expressions. The corresponding set of real-world knowledge points is as follows: And it is usually in multi-label form. The goal of knowledge point prediction is to learn a prediction model. Treating the pre-test questions from The most relevant output One knowledge point.

[0026] (2) Self-example generation module To reduce the reliance on external example libraries manually constructed by experts for knowledge point prediction and to enhance the semantic expressive ability of test questions under information sparsity conditions, this embodiment sets up a self-example generation module to automatically construct test question self-example enhanced representations from the test questions themselves that can be used in subsequent modules.

[0027] Self-example information originates from the current test question itself, automatically generated from the question stem and its implicit solution clues, rather than relying on demonstration examples manually selected or written by experts from an external question bank. In this framework, self-examples serve a similar role to external demonstration examples, providing subsequent modules with alignable, comparable, and searchable semantic references. This allows the system to recall candidate knowledge points from the knowledge point base and perform rearrangement and discrimination within the candidate set. Specifically, the subsequent candidate recall module will use self-example representations to match with knowledge point semantic representations to obtain the candidate set, thereby improving the stability and consistency of predictions.

[0028] Let the question to be tested be qi, which contains the question stem text, options, and optional structured fields, and can be represented as: in, This is natural language text (question stem, options, question type, etc.). Translate mathematical expressions or graphs into text. This is information for existing answers. This is an existing parsing. When or Even if missing, this module can still be based solely on Generate enhanced information.

[0029] The Teacher Large Language Model (Teacher LLM) is imported from the example generation module, denoted as... This model is used to generate high-quality intermediate supervision signals to guide subsequent, relatively lightweight or more controllable candidate recall and rearrangement modules to complete the final prediction; in some implementations, It can be used to generate self-example information for larger-scale or more powerful models, either offline or online, thereby "teaching" subsequent modules how to more stably align and distinguish knowledge points.

[0030] Specifically, via preset prompt template The test questions are analyzed and summarized, and structured analysis results are output. Description of simulated knowledge points The process can be formalized as follows: Among them, the structured analysis results Used to explicitly reveal implicit problem-solving clues and key constraints in test questions, it includes at least one of the following fields: ① Given conditions and constraints; ② Solve the objective; ③ Key problem-solving strategies or intermediate steps; ④ Key concepts, theorems, variations, or operational points related to the knowledge points.

[0031] Simulation knowledge points The description is based on the model and The generated semantic summarization text, oriented towards the knowledge point system, is used to match the semantic representations of knowledge points in the knowledge point base during the subsequent recall phase. It can be a collection of phrases or sentences: Each of them Describe the knowledge concepts, skills, or methodological elements embodied in the test questions, and allow the inclusion of operational descriptions related to the question type, thereby improving the differentiation of semantically similar knowledge points.

[0032] In obtaining and Then, this module constructs self-example-enhanced representations of the test questions. The original question content is then combined with the generated information: .

[0033] Through the aforementioned self-example generation mechanism, this invention can make explicit the problem-solving logic and examination points implicit in the test questions, and form self-owned example information that is highly matched with the current test questions, thereby improving the performance of subsequent recall and reordering modules without relying on experts to manually construct example libraries.

[0034] (3) Knowledge Point Recall Module In scenarios with a relatively small knowledge point system, it's possible to directly input the test questions, knowledge point set, and examples into a large language model for end-to-end prediction. However, in practical engineering applications, the knowledge point system is usually large, and different knowledge points exhibit semantic similarity, hierarchical overlap, and long-tail distribution. Directly inputting the entire knowledge point set into a large language model for judgment would not only significantly increase the input length and inference cost but also easily lead to prediction results being overly concentrated on a few high-frequency or prominently expressed knowledge points.

[0035] For the reasons mentioned above, this invention employs a two-stage prediction strategy combining candidate knowledge point recall and candidate set rearrangement. The recall module first filters out a subset of candidate knowledge points that are highly semantically relevant to the test questions from the full set of knowledge points. The subsequent rearrangement module performs fine-grained judgment and sorting only within the candidate set, thereby improving the ability to distinguish between similar knowledge points.

[0036] Let the complete set of knowledge points be... Each knowledge point A semantic description of a corresponding knowledge point (e.g., knowledge point names and definitions), denoted as This module uses a text encoder. Mapping knowledge point descriptions to a vector space: in, It can be implemented using online or offline embedding models, such as Sentence-BERT-like models based on a dual-tower structure, domain-adaptive text embedding models, or general text embedding models called via API, such as OpenAI's text-embedding-like models.

[0037] Similarly, for test questions Self-example enhanced representation The question vector is obtained by encoding: Subsequently, the cosine similarity of this module Calculate the correlation between the test questions and each knowledge point: And from the full set of knowledge points The top N knowledge points with the highest scores are selected to form a candidate set: .

[0038] (4) Candidate set reshoot module To further improve the accuracy, stability, and ability to distinguish similar knowledge points within the candidate set, this embodiment sets up a candidate set reordering module. While self-example augmentation can significantly improve the coverage of candidate recall, it may also introduce some semantic noise, making it easier to recall semantically similar but irrelevant knowledge point candidates in the coarse retrieval stage. Therefore, it is necessary to introduce a reordering model with stronger discriminative power within the candidate set to perform fine-grained relevance discrimination and ranking of candidate knowledge points, thereby suppressing noisy candidates and improving the quality of the final multi-label set.

[0039] Let the candidate set of the i-th question be . The rearrangement model is denoted as Its enhancement of test questions semantic description of candidate knowledge points Perform joint modeling and output scores: in, The larger the value, the more likely it is to be a candidate knowledge point. and The more relevant the questions are.

[0040] To obtain stable initial discrimination ability, this embodiment first performs supervised fine-tuning (SFT) on the rearrangement model. The SFT training data is constructed by the recall module: for each question... The recall module obtains the candidate set. And its coarse search similarity ranking; based on the gold standard knowledge point set The knowledge points in the candidate set are divided into positive and negative samples. Furthermore, to improve the model's ability to distinguish between "similar but unrelated" confusing knowledge points, this embodiment constructs two types of negative samples from the candidate set: Strong negative example: Selecting from the candidate set... It has a high similarity score, but it does not belong to the gold standard set. Knowledge points; Weak negative examples: Select knowledge points from the candidate set whose similarity ranking is in the middle or lower (which also do not belong to the category of knowledge points with similarity ranking). ), used to provide a wider coverage of negative example distributions.

[0041] Based on this, construct the SFT binary classification training sample set. ,in: During training, the scores were rearranged. Apply Obtain the predicted probability And it is optimized using binary cross-entropy loss: .

[0042] After completing the SFT, to enable the reordering model to directly optimize for the quality of the ranked list, this invention further employs GRPO-Style reinforcement learning to update the reordering strategy. This process uses the candidate set of the same question as the environment state and calculates relative advantage through intra-group comparisons to update the strategy that generates high-reward rankings.

[0043] ① Action definition and Plackett-Luce sampling without replacement For each question The reordering model outputs a candidate set. Scoring vector: A will be generated The ordered candidate list is regarded as a strategy The action. This embodiment uses the Plackett-Luce (PL) distribution for sampling without replacement to obtain the ranking results: in the first... t wheel( ), for the current set of unselected candidates Perform softmax normalization to obtain the sampling probability: according to Sampling yielded the first t The candidates are selected and removed from the pool of unselected candidates to advance to the next round. The final result is... Ordered list: .

[0044] To perform policy gradient updates, it is necessary to calculate the log probability of the sorting action under the current policy. For PL sampling without replacement, the above logarithmic probability can be expressed as the sum of the logarithmic probabilities of each step sample: in, Indicates the first t The score vector after masking the selected elements in the round.

[0045] The reward function in this embodiment is designed as follows: Let the set of key knowledge points for the i-th question be . The ordered list obtained from sampling. Calculate rewards The ranking criteria used are: .

[0046] The intra-group sampling and advantage update in this embodiment are as follows: For the same test question qi Under the current strategy G independent samples The sorted results form a group. : .

[0047] Calculate the reward and log probability for each sample within the group: in, This indicates that the first... Test Questions In the current strategy Next, proceed to the next step An action sample obtained from each independent sampling. Specifically, this action sample is an ordered list of Top-K knowledge points, used to represent the model's ranking and selection results for candidate knowledge points.

[0048] The within-group mean was used to calculate relative advantage. in, Indicates the first i The first question corresponds to the Each sampling result The reward points.

[0049] Based on this, the policy gradient loss of GRPO-Style is constructed as follows: Meanwhile, to constrain the update magnitude of the policy during the reinforcement learning phase, improve training stability, and prevent policy collapse, this embodiment introduces a policy relative to the reference policy. KL divergence regularization term: The optimization objective for the final reinforcement learning phase can be written as: in, The weights are KL regularized. By using intra-group advantage-driven updates combined with KL constraints, the re-ranking model can make relative comparisons among multiple possible rankings of the same question, gradually increasing the probability of generating a high-reward ranking, thereby directly optimizing the ranking quality index NDCG; at the same time, it ensures stable strategy updates, which is conducive to engineering implementation.

[0050] Example 2 In one specific embodiment, the knowledge point prediction method based on reinforcement learning described in this invention is implemented through a three-stage process: self-example generation, candidate recall, and candidate re-ranking, using models of different scales and capabilities for each stage. Specifically, the self-example generation module uses DeepSeekv3.2 as the teacher's large language model. The system generates structured parsing and realistic knowledge point descriptions for input questions, and then constructs a self-example-enhanced representation of the questions. Candidate knowledge point recall module selection As a text encoder ,Will Vectorization and semantic retrieval and recall in a knowledge point vector database; the candidate set reordering module selects... As a rearrangement model Within the recalled candidate set, fine-grained relevance sorting and filtering are performed to output the final set of knowledge points.

[0051] In this implementation, the training and evaluation of the reordering model are based on the publicly available dataset DA-20K. This dataset contains 22,498 high school math questions, covering 427 related knowledge points, and is divided into training, validation, and test sets in an 8:1:1 ratio. Based on a candidate recall strategy, this embodiment performs a coarse search of the knowledge point database for each question, selecting the top 30 knowledge points with the highest similarity to form a candidate set. This ensures candidate coverage and significantly reduces the computational cost of subsequent reordering.

[0052] The reordering model training consists of two phases: supervised warm-up and reinforcement learning optimization. In the supervised training phase, an early stopping strategy is employed, using the minimum validation set loss as the model selection criterion, and saving the model parameters with the optimal validation set loss. In the reinforcement learning phase, the optimal validation set metric is used as the model selection criterion, and saving the model parameters with the optimal validation set metric. The learning rate for both the SFT and RL phases is set to... .

[0053] In practical applications, when a question to be annotated is input, the system first calls DeepSeekv3.2 to generate a structured parsing and simulated knowledge point description for the question. This forms a self-example-enhanced representation; subsequently, it utilizes... right Vectorization is performed, and a Top-30 recall candidate set is retrieved from the knowledge point vector base; finally, by The candidate set is rearranged to output the final set of knowledge points. Through the above specific implementation methods, this invention can effectively improve the overall performance of knowledge point prediction in large-scale knowledge point systems.

[0054] By employing a joint training strategy that combines candidate set supervised fine-tuning warm-up with GRPO-Style reinforcement learning optimization, the candidate reordering module of this invention not only possesses stable local relevance discrimination capabilities but also continuously improves the sorting and set output quality at the list level, further supporting high-precision knowledge point prediction under a large-scale knowledge point system.

[0055] Example 3 This embodiment also provides a knowledge point prediction system based on reinforcement learning, including: a collection unit for acquiring self-example augmented representations of the questions to be tested; a construction unit for constructing a knowledge point prediction model, wherein the knowledge point prediction model includes a knowledge point recall module and a candidate set rearrangement module; and a prediction unit for inputting the self-example augmented representations into the knowledge point prediction model to obtain the knowledge point prediction results corresponding to the questions to be tested.

[0056] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A knowledge point prediction method based on reinforcement learning, characterized in that, Includes the following steps: S1. Obtain the test questions for the knowledge points to be predicted; S2. Construct a knowledge point prediction model, which includes a self-example generation module, a knowledge point recall module, and a candidate set rearrangement module. S3. Input the test questions for the knowledge points to be predicted into the knowledge point prediction model, and obtain the prediction results of the knowledge points corresponding to the test questions.

2. The knowledge point prediction method based on reinforcement learning according to claim 1, characterized in that, The self-example generation module is used to automatically construct enhanced self-example representations of test questions from the test questions themselves, which can be used in subsequent modules, including: The structured analysis results and simulated knowledge point descriptions of the test questions are obtained through the teacher's large language model; Based on the structured parsing results and the simulated knowledge point descriptions, an enhanced representation from the examples is obtained.

3. The knowledge point prediction method based on reinforcement learning according to claim 2, characterized in that, The knowledge point recall module is used to recall a set of candidate knowledge points from the entire knowledge point set based on the self-example enhanced representation, including: Based on the semantic description of each knowledge point in the complete knowledge point set, the knowledge point vector corresponding to each knowledge point is obtained through a text encoder. Based on the self-example augmented representation, the question vector is obtained through a text encoder; Based on the question vector and the vector of each knowledge point, the relevance between the question and each knowledge point is obtained; Based on relevance, a predetermined number of knowledge points are recalled from the complete set of knowledge points to form a candidate knowledge point set.

4. The knowledge point prediction method based on reinforcement learning according to claim 1, characterized in that, The candidate set reordering module is used to generate a sorted set of knowledge points based on the self-example augmented representation and the candidate knowledge point set. The candidate set reordering module is optimized through supervised fine-tuning and reinforcement learning, and includes: Based on the candidate knowledge point set and the gold standard knowledge point set, training samples are obtained. The training samples include positive samples, strong negative examples and weak negative examples. The candidate set rearrangement module is fine-tuned under supervision based on the training samples to obtain the initially trained candidate set rearrangement module.

5. The knowledge point prediction method based on reinforcement learning according to claim 4, characterized in that, The strong negative examples include: knowledge points selected from the candidate knowledge point set that have a high similarity to the self-example augmented representation and do not belong to the gold standard knowledge point set.

6. The knowledge point prediction method based on reinforcement learning according to claim 4, characterized in that, The candidate set rearrangement module is obtained through reinforcement learning optimization and includes: Based on the candidate set rearrangement module that has been initially trained, obtain a reference strategy; For the same pre-test question, the candidate knowledge point set is sampled multiple times according to the current strategy to obtain the ranking result within the group; Based on the group ranking results and the gold standard knowledge point set, obtain the relative advantages within the group; Update the current strategy based on the group's relative strengths and reference strategies.

7. The knowledge point prediction method based on reinforcement learning according to claim 6, characterized in that, The Plackett-Luce distribution is used to sample the candidate knowledge point set without replacement to obtain an ordered list, which serves as the ranking result within the group.

8. A knowledge point prediction system based on reinforcement learning, said system being used to implement the method according to any one of claims 1-7, characterized in that, include: The acquisition unit is used to obtain the self-example-enhanced representation of the questions to be tested; The building unit is used to build a knowledge point prediction model, which includes a self-example generation module, a knowledge point recall module, and a candidate set rearrangement module. The prediction unit is used to input the self-example augmented representation into the knowledge point prediction model to obtain the knowledge point prediction result corresponding to the question to be tested.

Images