Training method of course learning model, course learning method and related device

Abstract

The present application provides a course learning model training method, a course learning method and related devices, the course learning model training method comprises: using the problem data set corresponding to the target course to perform reinforcement learning training on the large language model, in each training process, the current problem data set is filtered using a question classification threshold, and the filtered problem data set is used as the training sample of the next round of iteration training of the large language model, until the course learning model is obtained under the condition that the large language model meets the target iteration condition; the question classification threshold is determined based on at least one of the correctness of the reply data corresponding to different questions and the course category. The method and device of the present application can provide an adaptive course learning scheme with single sample granularity, based on real-time capability feedback and without additional strategy network training, thereby improving the reliability of the course learning result.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and in particular to a training method, a course learning method, and related apparatus for a course learning model. Background Technology

[0002] In recent years, Reinforcement Learning with Verifiable Rewards (RLVR) has become a core paradigm for improving the long-range reasoning capabilities of Large Language Models (LLMs). Pioneering work, exemplified by the AI-assisted tool DeepSeek-R1, demonstrates that rewarding the correctness of the final result can effectively incentivize models to generate deeper and more refined reasoning chains, thereby achieving breakthrough performance in tasks requiring complex logical reasoning, such as mathematics and programming. To further optimize training efficiency and model performance, a curriculum learning strategy has been widely adopted, which organizes training data in order of increasing complexity.

[0003] In related technologies, the FastCuRL method is used for course learning training. This method constructs courses based on the prior assumption that "length equals difficulty," binning training data by length and gradually introducing long text data as training progresses. This leads to a large number of short but logically complex high-value samples being incorrectly ignored in the early stages of training, while lengthy but logically simple samples occupy valuable computational resources, resulting in a mismatch between course arrangement and the model's actual cognitive development. Existing technologies also use the SEC (Self-Evolving Curriculum) method to design course learning frameworks. Training data is pre-divided into several categories (such as different difficulty levels or question types). During training, the algorithm dynamically adjusts the probability of selecting data from each category based on the real-time learning gain (such as advantage value) estimated by the policy gradient, attempting to find the optimal sampling strategy among different categories. This method requires setting up an outward policy network, which is complex. Furthermore, as the model's response length increases and its ability improves, the accuracy of the same question increases, leading to low reliability of course learning results. Summary of the Invention

[0004] This invention provides a training method for a course learning model, a course learning method, and related devices to address the shortcomings of existing technologies that, when training course learning based on the assumption of "length equals difficulty," easily lead to a mismatch between the course arrangement and the model's actual cognitive development. Furthermore, when designing a course learning frame using the SEC method, it is necessary to set up an outward policy network, which is complex. Moreover, as the model's response length and ability improve, the accuracy of the same question will increase, resulting in low reliability of the course learning results.

[0005] This invention provides a method for training a course learning model, comprising: The large language model is trained by reinforcement learning using the question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training. The course learning model is obtained when the large language model meets the target iteration conditions. The question dataset includes multiple questions and corresponding answer data for each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rates of the answer data for different questions and the course category.

[0006] According to a training method for a course learning model provided by the present invention, before iteratively training a large language model using the question dataset corresponding to the target course, the method further includes: The question dataset is obtained by filtering the response length of all questions in the target course using a response length threshold.

[0007] According to a training method for a course learning model provided by the present invention, before filtering the response lengths of all questions in the target course using a response length threshold, the method further includes: Collect the full dataset of the target course, which includes course information in multiple different formats; The course information in the full dataset is structured and transformed to obtain the transformed question data; The converted question data is then subjected to format filtering and data cleaning to obtain all the questions for the target course.

[0008] According to the training method of the course learning model provided by the present invention, the target iteration condition includes any one of the following: Preset the number of training iterations; The average accuracy of the responses to each question output by the current large language model; The target number of iterations is determined based on the preset number of training iterations and corresponding weights, and the average accuracy and corresponding weights.

[0009] This invention also provides a course learning method including: Questions about obtaining courses to be learned; The question to be learned is input into the course learning model to obtain question answer data; wherein, the course learning model is trained by the course learning model training method.

[0010] The present invention also provides a training device for a course learning model, comprising: The training module is used to perform reinforcement learning training on the large language model using the question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training, until the large language model meets the target iteration conditions, and the course learning model is obtained. The question dataset includes multiple questions and corresponding answer data for each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rates of the answer data for different questions and the course category.

[0011] The present invention also provides a course learning device, comprising: The question retrieval module is used to retrieve questions for the courses to be learned. The course learning module is used to input the questions of the course to be learned into the course learning model and obtain question answer data; wherein, the course learning model is trained by the course learning model training method.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a training method or a course learning method as described above for any of the course learning models.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method or a course learning method for any of the course learning models described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a training method or a course learning method as described above for any of the course learning models.

[0015] The training method, learning method and related apparatus of the course learning model provided by this invention train a large language model through reinforcement learning using a question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training. The course learning model is obtained when the large language model meets the target iteration conditions. It can provide an adaptive course learning scheme that is accurate to the granularity of a single sample, based on real-time capability feedback and without the need for additional policy network training, thereby improving the reliability of the course learning results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the flowcharts illustrating the training method of the course learning model provided by this invention.

[0018] Figure 2 This is the second flowchart illustrating the training method of the course learning model provided by this invention.

[0019] Figure 3 This is a flowchart illustrating the course learning method provided by the present invention.

[0020] Figure 4 This is a schematic diagram of the training device for the course learning model provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the learning device provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] The following is combined Figures 1-5 The present invention describes the training method, the course learning method, and the related apparatus of the course learning model.

[0025] Figure 1 This is one of the flowcharts illustrating the training method of the course learning model provided by this invention, such as... Figure 1 As shown, the method includes the following steps: Step 110: Use the question dataset corresponding to the target course to train the large language model through reinforcement learning. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training, until the large language model meets the target iteration conditions, and the course learning model is obtained. The question dataset includes multiple questions and the corresponding answer data for each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rate of the answer data corresponding to different questions and the course category.

[0026] In this step, the target course can be a specific subject or knowledge area that the large language model is currently focusing on learning, such as mathematics (including geometry and algebra), natural sciences (physics and chemistry), or computer programming.

[0027] For example, the target course could be mathematics, and the corresponding problem dataset would include geometry, non-geometry, and graph problems (accounting for about 80%), with the remaining problems coming from general reasoning tasks, including scientific, social, and spatial understanding tasks (accounting for about 20%).

[0028] In this step, the training process of the large language model adopts the same multi-stage context length expansion strategy (8K, 16K, 24K) and GRPO reinforcement learning algorithm as DeepScaleR to ensure direct comparability with baseline methods.

[0029] It should be noted that, for Group Relative Policy Optimization (GRPO), given a problem... Strategy Model Sampling One answer .question The current accuracy rate is defined as , It is a discriminant function for determining the correctness of the answer; assuming the group size is... The reward distribution follows a Bernoulli distribution with a mean of Standard deviation According to the standardization mechanism of GRPO, the advantage function is defined as follows: The effective gradient contribution strength of policy updates can be calculated by the sum of the absolute values ​​of the within-group dominance values. To measure this, the contribution of the positive sample portion is... The contribution of the negative sample portion is Combining and simplifying the two, we can obtain the total gradient contribution as: This indicates that the variance of the reward is greatest when the within-group accuracy is 50%, and the contrastive gradient signal obtained by the large language model for distinguishing positive and negative samples is strongest. When the accuracy is 0% or 100%, the samples do not contribute to the gradient. Therefore, it is expected that the accuracy of each sample is 50% during training. However, considering that there are some answers truncated by the maximum answer length, and that the ability of the large oracle model increases with the training phase, even for the same question, It also increases monotonically during the training phase, and samples with 0% accuracy in this phase may contribute to the gradient in the next phase.

[0030] In this step, the question classification threshold is based on the model's mastery of a specific question (response accuracy) and the difficulty characteristics of the course class. It is used to classify each question sample in the question dataset during each iteration of training into easy, medium, and difficult types. In the GRPO reinforcement learning algorithm, the gradient contribution is greatest when the sample accuracy is close to 50%.

[0031] In this embodiment, the current problem dataset is filtered using a problem classification threshold. Low-value samples that the model has fully mastered (p≈100%) for the current subject are dynamically removed, while medium and difficult samples that are challenging and can generate effective gradients are retained. These samples are used to construct training samples for the next round of iteration training. This filtering process is adjusted in real time as the model's ability in a specific course improves (i.e., the accuracy of the response data changes).

[0032] In this embodiment, a rule-based validator is used to compare the responses generated by the large language model with preset standard answers to obtain the accuracy of each response.

[0033] In this embodiment, a reward function can be designed to supervise the reasoning format and final correctness of the large language model; specifically, for each input question, the large language model first generates corresponding response data, and for the question... q and response o Rewards r The following formula is used to express: ; In this way, the reward function above prompts the language model to perform long-range thought chain reasoning (only by determining how many correct and how many incorrect answers are obtained through the reward function, and calculating p, can the question be judged as "easy", "medium" or "difficult"). Only when the format and answer are both correct can the reward be obtained.

[0034] In a feasible implementation, assuming the current target course is mathematics, and the dataset used is a selection of mathematical problems from ViRL39K (i.e., the current problem dataset contains geometric proofs, algebraic operations, etc.), the large language model, in the current training phase (16K context length), sets a classification threshold for the mathematics course based on the accuracy cache established in the previous round (8K warm-up phase): accuracy p>0.8 is "easy", 0.1≤p≤0.8 is "medium", and p<0.1 is "difficult". For problem A (a basic algebra problem) in the current problem dataset, the model's accuracy p=0.95 in historical sampling is classified as "easy". This indicates that the model has mastered the mathematical knowledge point, and it is removed from the current training set. For problem B (a complex solid geometry problem), the accuracy rate p = 0.45, and it is judged as "medium"; for problem C (a high-difficulty number theory problem), the accuracy rate p = 0.05, and it is judged as "difficult". At this point, problems B and C are retained and combined into a new training sample set, i.e., the filtered problem dataset. The large language model performs GRPO training on the filtered geometry and number theory problems. During the training process, the model attempts to generate longer reasoning chains. After one round of training, the accuracy rate of problem B increases to 0.6 and it is still retained in the training set; problem A, having been removed, no longer consumes computing power. As the number of training steps accumulates (such as reaching the number of steps set for this stage), or the model's accuracy metric on validation no longer increases significantly (meeting the target iteration condition), the large language model ends the training for this stage, resulting in a course learning model with high-order mathematical reasoning capabilities.

[0035] The training method for the course learning model provided in this invention uses a problem dataset corresponding to the target course to perform reinforcement learning training on a large language model. In each training process, the current problem dataset is filtered using a problem classification threshold, and the filtered problem dataset is used as the training sample for the large language model in the next round of iteration training. This process continues until the large language model meets the target iteration conditions, thus obtaining the course learning model. This method can provide an adaptive course learning scheme that is accurate to the granularity of a single sample, based on real-time capability feedback, and does not require additional policy network training, thereby improving the reliability of the course learning results.

[0036] In some embodiments, before iteratively training the large language model using the question dataset corresponding to the target course, the training method of the course learning model further includes: filtering the response lengths of all questions in the target course using a response length threshold to obtain a question dataset.

[0037] In this embodiment, a short context window limit (e.g., 8k tokens) can be set as a constraint for generating responses during the initial stage of training.

[0038] Specifically, we first use a large language model to attempt to answer all questions within a limited length to generate an initial question dataset. This solves the problem of traversing the entire dataset through short text tests with low computational cost when there is no historical data to judge the difficulty of the questions, thus providing initial data support for the subsequent establishment of an accuracy cache.

[0039] For example, suppose the target course is "Mathematics," containing approximately 30,000 geometry and algebra problems from the ViRL39K dataset. A "response length threshold" of 8192 tokens (8K) is set as the first stage of training to quickly evaluate the accuracy of each problem in the dataset. All 3000 problems are then used as input, requiring a large language model (such as Qwen2.5-R1-distill-1.5B) to generate the reasoning process and answer within the 8K token length limit. For a simple "Pythagorean theorem calculation problem," the large language model might only need 500 tokens to solve it correctly within the threshold, while for a complex "multidimensional space proof problem," it might need 15,000 tokens. Only tokens can be used for reasoning, but due to the 8K threshold, the responses will be truncated or forcibly shortened, resulting in incomplete reasoning and incorrect judgment. After the first stage of screening, 30,000 sets of response data based on the 8K length can be obtained. Then, the reward function is used to score these responses and calculate the initial accuracy of each question in the "short thinking mode" (e.g., easy questions p=0.9). This dataset with the initial accuracy label is the question dataset used in the subsequent formal training cycle.

[0040] The training method for the course learning model provided in this embodiment of the invention effectively achieves low-cost difficulty initialization and saves a lot of computing resources by using a response length threshold to filter the response length of all questions in the target course.

[0041] In some embodiments, before filtering the response lengths of all questions in the target course using a response length threshold, the training method for the course learning model further includes: collecting the full dataset of the target course, the full dataset including course information in multiple different formats; performing a structured transformation on the course information in the full dataset to obtain transformed question data; and performing format filtering and data cleaning on the transformed question data to obtain all questions of the target course.

[0042] In this embodiment, the full dataset refers to the unfiltered original set of questions, which may include various modalities such as text, images, and tables.

[0043] In this embodiment, in order to adapt to the input requirements of large language models, unstructured or multimodal information (such as geometric figures and scientific charts) is transformed into a unified, high-quality text description format.

[0044] In this embodiment, preset template rules can be used to remove dirty data that fails to transform, loses semantics, or whose answer format cannot be automatically verified, ensuring that subsequent reinforcement learning training will not generate noise or invalid gradients due to data quality issues (such as unclear question descriptions or lack of standard answers).

[0045] In this embodiment, in order to obtain descriptive text, the multimodal large model Qwen2.5-VL-72B can be used to generate high-quality descriptions for all visual inputs. The prompt words input to this model are determined by structured templates, ensuring that the generated descriptions maintain the semantic integrity of the original question and are relevant to the solution.

[0046] In this embodiment, after filtering out samples with inconsistent formats or failed descriptions, a cleaned subset containing multiple valid questions is retained, which constitutes the final dataset used in the experiment, i.e., the question dataset used for reinforcement learning training of the model. The cleaned question dataset provides a reliable foundation for reinforcement learning training with aligned visual and text inputs.

[0047] The training method for the course learning model provided in this embodiment of the invention collects the full dataset of the target course, which includes course information in multiple different formats; the course information in the full dataset is structurally transformed to obtain transformed problem data; the transformed problem data is subjected to format filtering and data cleaning to ensure the high quality and alignment of the training data. At the same time, invalid data is removed through data cleaning, providing reliable data support for the subsequent establishment of an accurate accuracy cache and preventing the model from wasting computing resources on bad data that it cannot solve.

[0048] In some embodiments, the target iteration condition includes any one of the following: a preset number of training iterations; the average accuracy of the response data corresponding to each question output by the current large language model; and the target number of iterations, which is determined based on the preset number of training iterations and corresponding weights, the average accuracy and corresponding weights.

[0049] In this embodiment, the target iteration condition is the termination criterion for determining whether the current training phase (such as the 16K context length phase) has ended and the final model has been output (or the next phase has begun).

[0050] In this embodiment, a fixed number of training steps can be preset according to the computational budget. For example, the "preset number of training iterations" can be set to 440 steps. That is, no matter how the large language model performs, once the number of training steps reaches 440, training will be forcibly stopped, which can ensure that the total computational resources consumed are within a controllable budget.

[0051] In this embodiment, the average accuracy refers to the performance metric (such as Pass@1) of the large language model on the validation set or the current training batch. For example, by monitoring the average accuracy of the model on the validation set (such as MATH500) in real time, a target threshold of Pass@1=55% is set. If the model's accuracy has reached 55.2% by the 300th step, which satisfies the target iteration condition, the model will terminate training early. This method avoids the model from continuing to do ineffective work after it has already learned the material.

[0052] In this embodiment, the optimal stopping point can be calculated by weighting the speed of model learning (accuracy rise slope), that is, the timing of model termination can be dynamically adjusted to balance training cost and model performance. The target number of iterations set by this method ensures that the model fully converges and masters the knowledge of the current course, while avoiding overfitting or wasting expensive computing resources in the later stages of diminishing marginal returns.

[0053] For example, assuming the initial preset number of iterations is 500, if the model's accuracy improves very quickly in the first 100 steps (high weight), it indicates that the course at this stage is too simple for the model or that the model has a strong learning ability. In this embodiment, the target number of iterations can be reduced to 300 steps by adjusting the weight factor to save computing power. If the accuracy increases slowly (low weight), it indicates that the model is struggling to overcome "medium" and "difficult" samples. In this embodiment, the target number of iterations can be maintained or appropriately extended to 500 steps to give the model more time to digest the data.

[0054] The training method for the course learning model provided in this embodiment of the invention sets target iteration conditions including any one of the following: a preset number of training iterations; the average accuracy of the response data corresponding to each question output by the current large language model; and a target number of iterations determined based on the preset number of training iterations and corresponding weights, the average accuracy and corresponding weights. By flexibly setting iteration conditions, the method avoids the model from continuing to idle after its performance is saturated, and can provide the optimal "cost-benefit" solution for the model during the training process.

[0055] Figure 2 This is the second flowchart illustrating the training method of the course learning model provided by this invention. Figure 2In the illustrated embodiment, during the warm-up training process (Phase 1), each question in the full dataset is sampled using a shorter response length, and the sampled data is input into the base model for reinforcement learning training to obtain the Phase 1 model. An initial accuracy rate for each question is established in the accuracy cache. Then, based on the corresponding question classification threshold and the initial accuracy rates, each question in the question dataset is classified into "easy questions," "medium-difficult questions," and "hard questions." During the course training process (Phase 2), the "medium-difficult questions" and "hard questions" cached in Phase 1 are input into the Phase 1 model for reinforcement learning training to obtain the Phase 2 model. The accuracy rate of each question in Phase 2 is obtained, and then the aforementioned question classification is used... Based on the class threshold and the accuracy of each question in Stage 2, the data in the filtered problem dataset are further divided into "easy problems," "medium problems," and "hard problems." During course training (Stage 3), the "medium problems" and "hard problems" cached in Stage 2 are input into the Stage 2 model for reinforcement learning training, resulting in the Stage 3 model. The accuracy of each question in Stage 3 is then obtained. Using the aforementioned problem classification threshold and the accuracy of each question in Stage 3, the data in the filtered problem dataset are again divided into "easy problems," "medium problems," and "hard problems," until the maximum number of iterations is met or the accuracy of each question reaches a certain threshold, resulting in the trained course learning model.

[0056] The course learning method provided by this invention will be described below. The course learning method described below can be referred to in correspondence with the training method of the course learning model described above.

[0057] Figure 3 This is a flowchart illustrating the course learning method provided by the present invention, such as... Figure 3 As shown, the learning method for this course includes the following steps: Step 310: Obtain questions about the courses to be learned.

[0058] In this step, the questions for the course to be studied can be tailored to a specific subject or knowledge area.

[0059] The courses to be studied include mathematics (including geometry and algebra), natural sciences (physics and chemistry), or computer programming.

[0060] For example, the course to be learned could be mathematics, and the corresponding course problems could include geometry, non-geometry, graph problems, science, social or spatial understanding tasks.

[0061] Step 320: Input the questions for the course to be learned into the course learning model to obtain the question answer data; wherein, the course learning model is trained using the course learning model training method.

[0062] In this step, the course learning model can be obtained through the following steps: (1) Use the question dataset corresponding to the target course to train the large language model for reinforcement learning. In each training process, the current question dataset is filtered using the question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training until the large language model meets the target iteration conditions, and the course learning model is obtained. The question dataset includes multiple questions and the answer data corresponding to each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rate of the answer data corresponding to different questions and the course category.

[0063] It should be noted that the implementation method of step (1) above corresponds one-to-one with the specific implementation method of step 110 above, and will not be repeated in this embodiment.

[0064] In this embodiment, before iteratively training the large language model using the question dataset corresponding to the target course, the training method of the course learning model further includes: filtering the response lengths of all questions in the target course using a response length threshold to obtain the question dataset.

[0065] In this embodiment, before filtering the response lengths of all questions in the target course using a response length threshold, the training method of the course learning model further includes: collecting the full dataset of the target course, which includes course information in multiple different formats; performing a structured transformation on the course information in the full dataset to obtain transformed question data; and performing format filtering and data cleaning on the transformed question data to obtain all questions of the target course.

[0066] In this embodiment, the target iteration condition includes any one of the following: a preset number of training iterations; the average accuracy of the response data corresponding to each question output by the current large language model; and the target number of iterations, which is determined based on the preset number of training iterations and corresponding weights, the average accuracy and corresponding weights.

[0067] In this embodiment, after the questions for the course to be learned are input into the trained course learning model, the course learning model outputs more accurate question answer data.

[0068] The course learning method provided in this embodiment of the invention processes the questions in the course to be learned by training the course learning model through a training method, and obtains question answer data, thereby improving the accuracy of question answering and thus improving the efficiency of course learning.

[0069] The training apparatus for the course learning model provided by the present invention will be described below. The training apparatus for the course learning model described below and the training method for the course learning model described above can be referred to in correspondence with each other.

[0070] Figure 4 This is a schematic diagram of the training device for the course learning model provided by the present invention, as shown below. Figure 4 As shown, the training device for the course learning model includes: training module 410.

[0071] Training module 410 is used to perform reinforcement learning training on the large language model using the question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training, until the large language model meets the target iteration conditions, and the course learning model is obtained. The question dataset includes multiple questions and the corresponding answer data for each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rate of the answer data corresponding to different questions and the course category.

[0072] The training device for the course learning model provided in this embodiment of the invention performs reinforcement learning training on a large language model using a question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training. The course learning model is obtained when the large language model meets the target iteration conditions. It can provide an adaptive course learning scheme that is accurate to the granularity of a single sample, based on real-time capability feedback, and does not require additional policy network training, thereby improving the reliability of the course learning results.

[0073] The course learning device provided by the present invention is described below. The course learning device described below and the course learning method described above can be referred to in correspondence.

[0074] Figure 5 This is a schematic diagram of the structure of the course learning device provided by the present invention, as shown below. Figure 5 As shown, the course learning device includes: a problem acquisition module 510 and a course learning module 520.

[0075] The question acquisition module 510 is used to acquire questions for the courses to be learned; The course learning module 520 is used to input questions about the courses to be learned into the course learning model and obtain question answer data; the course learning model is trained using the course learning model training method.

[0076] The course learning device provided in this embodiment of the invention processes questions about the course to be learned by training a course learning model through a training method, and obtains question answer data, thereby improving the accuracy of question answers and thus improving the efficiency of course learning.

[0077] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a training method for a course learning model. This method includes: performing reinforcement learning training on a large language model using a question dataset corresponding to the target course; in each training process, filtering the current question dataset using a question classification threshold, and using the filtered question dataset as training samples for the large language model in the next iteration, until the large language model meets the target iteration conditions, thus obtaining the course learning model; wherein the question dataset includes multiple questions and corresponding answer data for each question, different answer data correspond to different accuracy rates, and the question classification threshold is determined based on at least one of the accuracy rates of the answer data corresponding to different questions and the course category.

[0078] Alternatively, a course learning method may be implemented, including: obtaining the course questions to be learned; inputting the course questions to be learned into the course learning model to obtain question answer data; wherein, the course learning model is trained using the course learning model training method.

[0079] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a training method for the course learning model provided by the above methods. The method includes: performing reinforcement learning training on a large language model using a question dataset corresponding to a target course; in each training process, filtering the current question dataset using a question classification threshold, and using the filtered question dataset as the training sample for the large language model in the next round of iteration training, until the large language model meets the target iteration condition, thereby obtaining the course learning model; wherein, the question dataset includes multiple questions and corresponding answer data for each question, different answer data correspond to different accuracy rates, and the question classification threshold is determined based on at least one of the accuracy rates of the answer data corresponding to different questions and the course category.

[0081] Alternatively, a course learning method may be implemented, including: obtaining the course questions to be learned; inputting the course questions to be learned into the course learning model to obtain question answer data; wherein, the course learning model is trained using the course learning model training method.

[0082] In another aspect, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method of the course learning model provided by the above methods. The method includes: performing reinforcement learning training on a large language model using a question dataset corresponding to the target course; in each training process, filtering the current question dataset using a question classification threshold, and using the filtered question dataset as the training sample for the large language model in the next round of iteration training, until the large language model meets the target iteration conditions, thereby obtaining the course learning model; wherein, the question dataset includes multiple questions and corresponding answer data for each question, different answer data correspond to different accuracy rates, and the question classification threshold is determined based on at least one of the accuracy rates of the answer data corresponding to different questions and the course category.

[0083] Alternatively, a course learning method may be implemented, including: obtaining the course questions to be learned; inputting the course questions to be learned into the course learning model to obtain question answer data; wherein, the course learning model is trained using the course learning model training method.

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a course learning model, characterized in that, include: The large language model is trained by reinforcement learning using the question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training. The course learning model is obtained when the large language model meets the target iteration conditions. The question dataset includes multiple questions and corresponding answer data for each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rates of the answer data for different questions and the course category.

2. The training method for the course learning model according to claim 1, characterized in that, Before iteratively training the large language model using the question dataset corresponding to the target course, the method further includes: The question dataset is obtained by filtering the response length of all questions in the target course using a response length threshold.

3. The training method for the course learning model according to claim 2, characterized in that, Before filtering the response lengths of all questions in the target course using a response length threshold, the method further includes: Collect the full dataset of the target course, which includes course information in multiple different formats; The course information in the full dataset is structured and transformed to obtain the transformed question data; The converted question data is then subjected to format filtering and data cleaning to obtain all the questions for the target course.

4. The training method for the course learning model according to any one of claims 1-3, characterized in that, The target iteration condition includes any one of the following: Preset the number of training iterations; The average accuracy of the responses to each question output by the current large language model; The target number of iterations is determined based on the preset number of training iterations and corresponding weights, and the average accuracy and corresponding weights.

5. A course learning method, characterized in that, include: Questions about obtaining courses to be learned; The question to be learned is input into the course learning model to obtain question answer data; wherein the course learning model is trained by the training method of the course learning model as described in any one of claims 1-4.

6. A training device for a course learning model, characterized in that, include: The training module is used to perform reinforcement learning training on the large language model using the question dataset corresponding to the target course. In each training process, the current question dataset is filtered using a question classification threshold, and the filtered question dataset is used as the training sample for the large language model in the next round of iteration training, until the large language model meets the target iteration conditions, and the course learning model is obtained. The question dataset includes multiple questions and corresponding answer data for each question. Different answer data correspond to different accuracy rates. The question classification threshold is determined based on at least one of the accuracy rates of the answer data for different questions and the course category.

7. A course learning device, characterized in that, include: The question retrieval module is used to retrieve questions for the courses to be learned. The course learning module is used to input the questions of the course to be learned into the course learning model to obtain question answer data; wherein, the course learning model is trained by the training method of the course learning model as described in any one of claims 1-4.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the course learning model as described in any one of claims 1 to 4 or the course learning method as described in claim 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the course learning model as described in any one of claims 1 to 4 or the course learning method as described in claim 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the course learning model as described in any one of claims 1 to 4 or the course learning method as described in claim 5.

Images