A method, system, device and medium for constructing a large model for primary and secondary school education and tutoring

Abstract

The present invention relates to the field of artificial intelligence technology, and more specifically, to a method, system, device, and medium for constructing a large model for primary and secondary school education and tutoring. The method first uses Qwen2.5-3B-Instruct as a basic model and constructs a training data set based on the acquired teaching data set. Then, based on the training data set, a reward function is constructed to obtain a reward value, and a reinforcement learning method is called to adjust the model parameters to obtain a reinforced basic model. Finally, based on the constructed teacher style data set and the set personalized prompt words, the reinforced basic model is fine-tuned in a supervised instruction fine-tuning manner to obtain a large model for primary and secondary school education and tutoring. Through two-stage fine-tuning, the model's reasoning ability is improved. While maintaining low computing resource requirements, the elderly model's ability to handle complex problems is improved and a clear teacher style is given.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system, device and medium for constructing a large model for primary and secondary school education and tutoring. Background Art

[0002] Although current large language models perform well on general tasks, they suffer from the following shortcomings in specific teaching scenarios, especially in primary and secondary education:

[0003] Lack of proprietary corpus in the primary and secondary education vertical field, resulting in unsatisfactory Q&A results.

[0004] Existing high-performance large models have large parameters and extremely high requirements for computing resources, making them difficult to deploy.

[0005] The general large model lacks the teacher's unique explanation style when answering questions, and it is difficult to meet the requirements of a "virtual teacher" (speaking style), and is not suitable for serving groups such as primary and secondary school students.

[0006] Therefore, a new model building method is needed that can improve the model's ability to handle complex problems and give it a clear teacher style while maintaining low computing resource requirements. Summary of the Invention

[0007] In response to the problems of existing large language models in educational applications, such as insufficient ability to deal with complex problems, lack of teacher explanation style, and excessive demand for computing resources, the present invention proposes a method, system, equipment and medium for constructing a large model for primary and secondary school education tutoring. The method first uses Qwen2.5-3B-Instruct as the basic model and constructs a training data set based on the acquired teaching data set; then, based on the training data set, a reward function is constructed to obtain a reward value, and a reinforcement learning method is called to adjust the model parameters to obtain a reinforced basic model; finally, based on the constructed teacher style data set and the set personalized prompt words, the reinforced basic model is fine-tuned in a supervised instruction fine-tuning manner to obtain a large model for primary and secondary school education tutoring. Through two-stage fine-tuning, the model's reasoning ability is improved. While maintaining low computing resource requirements, the elderly model's ability to handle complex problems is improved and it is given a clear teacher style.

[0008] The specific implementation contents of the present invention are as follows:

[0009] A method for constructing a large model for primary and secondary school education tutoring, specifically comprising the following steps:

[0010] Step S1: Use Qwen2.5-3B-Instruct as the basic model and construct a training dataset based on the obtained teaching dataset;

[0011] Step S2: Based on the training data set, a reward function is constructed to obtain a reward value, and the reinforcement learning method is called to adjust the model parameters to obtain a reinforced basic model;

[0012] Step S3: Based on the constructed teacher style dataset and the set personalized prompt words, the strengthened basic model is fine-tuned by means of supervised instruction fine-tuning to obtain a large model for primary and secondary education tutoring.

[0013] In order to better implement the present invention, further, step S2 specifically includes the following steps:

[0014] Step S21: Construct a reward function based on the answer generated by the basic model, the standard answer, the similarity function, and the set reward parameters, and calculate the reward value;

[0015] Step S22: Calculate the probability distribution according to the set regularization coefficient, the set clipping threshold, and the model parameters;

[0016] Step S23: Calculate the KL divergence based on the reward value and probability distribution;

[0017] Step S24: adjusting the regularization coefficient according to the upper and lower limits of the KL divergence to obtain the enhanced basic model.

[0018] In order to better implement the present invention, further, step S22 specifically includes the following steps:

[0019] Step S221: Calculate the strategy ratio based on the questions obtained from the training data set, the model sampled answers, and the large model strategy;

[0020] Step S222: Calculate advantage estimate based on the reward value and the average reward of all questions in the current buffer;

[0021] Step S223: According to the set regularization coefficient, the set clipping threshold, the advantage estimate, and the strategy ratio, the probability of the basic model generating the answer with the model parameter θ is calculated to obtain the probability distribution.

[0022] In order to better implement the present invention, the specific operation of step S24 is as follows: according to the upper limit and lower limit of KL divergence, the regularization coefficient λ is adaptively adjusted. If the KL divergence is greater than the upper limit of KL divergence, the regularization coefficient λ is adjusted. KL Adjust to If the KL divergence is greater than the upper limit of the KL divergence, the regularization coefficient λ KL Adjust to .

[0023] In order to better implement the present invention, further, step S3 specifically includes the following steps:

[0024] Step S31: constructing a teacher style enhanced dataset based on the obtained typical test questions and the corresponding question-answer pair datasets explained in the teacher style;

[0025] Step S32: Convert the teacher style enhancement dataset into data pairs in the form of Alpaca instructions, and call Role-Play to construct personalized prompt words;

[0026] Step S33: Fine-tune the strengthened basic model by means of supervised instruction fine-tuning to obtain a large model for primary and secondary education tutoring.

[0027] In order to better implement the present invention, further, step S33 specifically includes the following steps:

[0028] Step S331: inserting LoRA adapters into several linear transformation matrices of the basic model;

[0029] Step S332: Keep the original linear transformation matrix frozen and train the newly added LoRA adapter matrix;

[0030] Step S333: Based on the data pairs in the form of Alpaca instructions, the Adam optimizer is called to perform gradient updates on the LoRA adapter matrix to obtain the basic model after supervised instruction fine-tuning, i.e., the primary and secondary school education tutoring model.

[0031] In order to better implement the present invention, further, the step S1 is specifically as follows: using Qwen2.5-3B-Instruct as a basic model, and constructing a composite mathematical reasoning training dataset based on the acquired high-quality mathematical reasoning dataset and the local private primary and secondary school mathematics dataset.

[0032] Based on the above-mentioned method for constructing a large model for primary and secondary education tutoring, in order to better implement the present invention, a system for constructing a large model for primary and secondary education tutoring is further proposed, which is used to execute the above-mentioned method for constructing a large model for primary and secondary education tutoring; the system includes an initial unit, a reinforcement unit, and a supervision and fine-tuning unit;

[0033] The initial unit is used to use Qwen2.5-3B-Instruct as a basic model and construct a training data set based on the obtained teaching data set;

[0034] The reinforcement unit is used to construct a reward function based on the training data set to obtain a reward value, and call a reinforcement learning method to adjust the model parameters to obtain a reinforced basic model;

[0035] The supervised fine-tuning unit is used to fine-tune the strengthened basic model in a supervised instruction fine-tuning manner according to the constructed teacher style data set and the set personalized prompt words, so as to obtain a large model for primary and secondary school education tutoring.

[0036] Based on the above-mentioned method for constructing a large model for primary and secondary school education tutoring, in order to better realize the present invention, an electronic device is further proposed, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the above-mentioned method for constructing a large model for primary and secondary school education tutoring is implemented.

[0037] Based on the above-mentioned method for constructing a large model for primary and secondary school education tutoring, in order to better realize the present invention, a computer-readable storage medium is further proposed, on which computer instructions are stored; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned method for constructing a large model for primary and secondary school education tutoring is implemented.

[0038] The present invention has the following beneficial effects:

[0039] (1) Through two-stage fine-tuning, the present invention improves the model's ability to handle complex problems and gives it a clear teacher style while maintaining low computing resource requirements.

[0040] (2) The present invention constructs a large model for primary and secondary education tutoring based on a two-stage fine-tuning framework, which effectively solves the problems of existing large language models in specific educational applications, such as insufficient ability to deal with complex problems, lack of teacher explanation style, and excessive demand for computing resources. Compared with models at the same level, the present invention has stronger capabilities and a distinctive teacher style.

[0041] (3) The present invention continuously adjusts model parameters through reinforcement learning methods, gradually improving the model's performance in complex mathematical reasoning tasks and significantly enhancing the model's mathematical reasoning ability.

[0042] (4) The model fine-tuned by supervised instructions in this invention can not only solve specific subject problems, but also significantly demonstrate a professional teaching style similar to that of primary and secondary school teachers. The problem-solving process generated by the model is detailed and clear, which is easy for students to understand. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic block diagram of the specific process provided by the present invention.

[0044] Figure 2 This is a schematic diagram of the thought chain provided by the present invention. DETAILED DESCRIPTION

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.

[0046] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections, electrical connections; direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0047] Example 1:

[0048] This embodiment proposes a method for constructing a large model for primary and secondary education tutoring, which specifically includes the following steps:

[0049] Step S1: Use Qwen2.5-3B-Instruct as the base model and construct a training dataset based on the obtained teaching dataset.

[0050] The specific steps of step S1 are: using Qwen2.5-3B-Instruct as a basic model, and constructing a composite mathematical reasoning training dataset based on the acquired high-quality mathematical reasoning dataset and the local private primary and secondary school mathematics dataset.

[0051] Step S2: Based on the training data set, a reward function is constructed to obtain a reward value, and the reinforcement learning method is called to adjust the model parameters to obtain a reinforced basic model.

[0052] The step S2 specifically includes the following steps:

[0053] Step S21: Construct a reward function based on the answer generated by the basic model, the standard answer, the similarity function, and the set reward parameters, and calculate the reward value;

[0054] Step S22: Calculate the probability distribution according to the set regularization coefficient, the set clipping threshold, and the model parameters;

[0055] Furthermore, the step S22 specifically includes the following steps:

[0056] Step S221: Calculate the strategy ratio based on the questions obtained from the training data set, the model sampled answers, and the large model strategy;

[0057] Step S222: Calculate advantage estimate based on the reward value and the average reward of all questions in the current buffer;

[0058] Step S223: According to the set regularization coefficient, the set clipping threshold, the advantage estimate, and the strategy ratio, the probability of the basic model generating the answer with the model parameter θ is calculated to obtain the probability distribution.

[0059] Step S23: Calculate the KL divergence based on the reward value and probability distribution;

[0060] Step S24: adjusting the regularization coefficient according to the upper and lower limits of the KL divergence to obtain the enhanced basic model.

[0061] Furthermore, the specific operation of step S24 is: adaptively adjust the regularization coefficient according to the upper and lower limits of the KL divergence. If the KL divergence is greater than the upper limit of the KL divergence, the regularization coefficient λ is adjusted. KL Adjust to If the KL divergence is greater than the upper limit of the KL divergence, the regularization coefficient λ KL Adjust to .

[0062] Step S3: Based on the constructed teacher style dataset and the set personalized prompt words, the strengthened basic model is fine-tuned by means of supervised instruction fine-tuning to obtain a large model for primary and secondary education tutoring.

[0063] The step S3 specifically includes the following steps:

[0064] Step S31: constructing a teacher style enhanced dataset based on the obtained typical test questions and the corresponding question-answer pair datasets explained in the teacher style;

[0065] Step S32: Convert the teacher style enhancement dataset into data pairs in the form of Alpaca instructions, and call Role-Play to construct personalized prompt words;

[0066] Step S33: Fine-tune the strengthened basic model by means of supervised instruction fine-tuning to obtain a large model for primary and secondary education tutoring.

[0067] The step S33 specifically includes the following steps:

[0068] Step S331: inserting LoRA adapters into several linear transformation matrices of the basic model;

[0069] Step S332: Keep the original linear transformation matrix frozen and train the newly added LoRA adapter matrix;

[0070] Step S333: Based on the data pairs in the form of Alpaca instructions, the Adam optimizer is called to perform gradient updates on the LoRA adapter matrix to obtain the basic model after supervised instruction fine-tuning, i.e., the primary and secondary school education tutoring model.

[0071] Working Principle: This embodiment uses two-stage fine-tuning to improve the model's ability to handle complex problems and give it a clear teacher style while maintaining low computing resource requirements. Based on the two-stage fine-tuning framework, a large model for primary and secondary school education tutoring is constructed, which effectively solves the problems of existing large language models in specific educational applications, such as insufficient ability to handle complex problems, lack of teacher explanation style, and excessive computing resource requirements. Compared with models at the same level, it has stronger capabilities and a distinct teacher style.

[0072] Example 2:

[0073] This embodiment is based on the above embodiment 1. Figure 1 、 Figure 2 As shown, a specific embodiment is described in detail.

[0074] Step S1: base model selection.

[0075] In this embodiment, Qwen2.5-3B-Instruct is selected as the basic model, i.e., the base model. The model has a moderate 3B scale parameter quantity, good language understanding and generation capabilities, and is suitable for application in primary and secondary education.

[0076] Selecting a large model with a small number of parameters such as Qwen2.5-3B-Instruct solves the problem of "excessive computing resource requirements" mentioned earlier, and then conducts two-stage fine-tuning on this basis, so that this small model has stronger capabilities and a distinctive teacher style compared to models at the same level.

[0077] Step S2: The first stage: reinforcement learning fine-tuning and reasoning ability enhancement stage.

[0078] Step S21: Dataset construction.

[0079] This example combines public, high-quality mathematical reasoning datasets, such as S1K, with private, locally available elementary and middle school mathematics datasets to create a composite mathematical reasoning training dataset, EdMath. This dataset covers mathematical reasoning problems of varying difficulty, offering high coverage and representativeness. The EdMath dataset contains approximately 3,000 entries and includes three fields: question, thought chain, and answer.

[0080] Step S22: reinforcement learning implementation method.

[0081] This example uses a reward-based reinforcement learning approach, using the accuracy of the model's answers as a reward signal. Specifically, after the model generates an answer, it compares it to a standard answer. Correct answers receive positive rewards, while incorrect answers receive negative rewards or zero rewards. This example also takes into account the Chain-of-Thought (CoT) process, continuously adjusting model parameters through reinforcement learning to gradually improve the model's performance in complex mathematical reasoning tasks.

[0082] This is the reasoning process, also known as the thinking chain. Before the launch of DeepSeek-R1, only ChatGPT-o1 had the ability to reason (this type of model is also called a reasoning model). This thinking process can significantly improve the model's capabilities, such as Figure 2 shown.

[0083] Step S221: Reward function setting.

[0084]

[0085] Here, A is the standard answer, and  is the answer given by the large model. This means that the answer given by the large model is closer and closer to the expected value. The default values ​​are α=0.7 and β=0.3. α and β are weight coefficients, and α>β indicates that the importance of the "answer" is higher than the "reasoning process."

[0086] EM: Exact Match, which determines whether the answer is correct. Only correct answers will be given high scores. It is a hard indicator.

[0087] Sim : Text similarity and similarity of reasoning processes are also taken into consideration, encouraging "correct thinking" even if the calculation is wrong; this can quickly increase the accuracy rate without abandoning the value of the thinking chain.

[0088] Step S222: Given a question Q~D, the model samples the answer  and gets a reward D contains questions and answers, and Q here is a question randomly selected from the answer column of dataset D. The big model will then solve this question Q and get an answer. It will then go back to dataset D and compare it with the corresponding standard answer (calculating a reward) to see if the big model got it right.

[0089] Update the parameter θ by maximizing the following formula:

[0090] (1)

[0091] Given a question Q, the probability that the model generates the answer  with parameters θ is the probability distribution; It is the policy optimization objective function used in reinforcement learning, which represents the expected total benefit that the model can obtain when optimizing the policy by adjusting the model parameters θ.

[0092] is the strategy ratio; It represents the probability that the model will generate the same predicted answer  for the same question Q when the parameters in the previous round or batch are the old policy. It serves as a benchmark during the policy update process to measure the degree of model improvement.

[0093] A t For advantage estimation, GAE can be used for calculation;

[0094] To trim the threshold and suppress excessive strategy step size;

[0095] λ KL is the KL regularization coefficient, which is dynamically adjusted to prevent strategy divergence;

[0096] Use "tailoring" to protect training stability and use KL to lock language fluency;

[0097] Step S23: training loop.

[0098] Step S231: Sampling.

[0099] Randomly extract BS questions from D and use the current strategy π θ Generate reasoning chain + answer; calculate reward r immediately;

[0100] "Thinking chain" and "reasoning chain" are the same thing. This thinking chain is constrained by the prompt word, which is the training completed in step S235. <reasoning> Reasoning process...< / reasoning> Through this large model of prompt words, you can understand that you should think first and then give an answer.

[0101] When using DeepSeek, Doubao, and Qianwen, they will think about a long series of questions before giving the answer. This is the "reasoning process", "thinking chain", and "reasoning chain".

[0102] Step S232: Advantage estimation.

[0103] After accumulating K batches of samples, use the GAE formula to obtain the advantage A of each trajectory t ; Sampling multiple times continuously, accumulating them and doing gradient update at one time.

[0104] General formula:

[0105]

[0106]

[0107] : Instant advantage;

[0108] : time discount;

[0109] λ GAE : smoothing coefficient;

[0110] : state value function, where t is the moment;

[0111] L: the number of remaining steps from the current time t to the end of the trajectory;

[0112] l : count, value 0, 1, 2…L-1;

[0113] However, in this embodiment, the formula can be simplified as follows: Since this is a single-step question answering, there is no question in step t, so the subscript t can be omitted. At the same time, L=1, and the formula is simplified to the following.

[0114]

[0115] r: The immediate reward the model gets for this problem;

[0116] : The average reward of all questions in the current buffer;

[0117] In the reinforcement learning training process of this embodiment, each batch contains a certain number of question samples. Assume that each batch contains N samples. Each sample can obtain a reward value r after the model answers it. i (Automatically calculated by the reward function) Then the average reward of the batch is The calculation formula is:

[0118]

[0119] r i Represents the reward value obtained by the i-th sample in the current batch;

[0120] N represents the total number of samples in the current batch;

[0121] The significance of this average reward value is to establish a baseline for all reward values ​​in the current batch to determine whether the reward value of a single sample is higher or lower than the overall average, thereby obtaining a more stable and effective training signal.

[0122] Step S233: Perform gradient ascent on equation (1) on these K batches of samples; repeat E epochs to fully utilize the samples.

[0123] Step S234: Adaptive KL.

[0124] Monitor the average KL during training, if KL>KL max , change λ KL ×1.5; if KL <KL min , change λ KL ÷1.5, Kullback-Leibler Divergence (KL divergence) is used to measure how different two sets of probability distributions are, and the difference between the new and old models.

[0125] Some preferred values ​​for hyperparameters:

[0126] Batch size BS: 32;

[0127] Number of sampling batches per round K: 8;

[0128] Epoch: 6;

[0129] Clip threshold ε: 0.2;

[0130] KL min / max : 0.01 / 0.05;

[0131] Initial λ KL :1.0;

[0132] Parameter explanation:

[0133] , a total of M questions - standard answer pairs;

[0134] π θ , the basic large model strategy to be optimized;

[0135] π0, the original parameter snapshot, used for KL constraint;

[0136] R(Q, Â), the reward function, scores the model’s answer Â;

[0137] EM, Exact-Match, 1 if the answer value matches exactly, otherwise 0;

[0138] Sim , semantic similarity, range [0,1];

[0139] Step S235: Training is completed.

[0140] Output the final strategy π θ ;

[0141] Output format:

[0142] <reasoning> Reasoning process...< / reasoning>

[0143] <answer> Answer< / answer>

[0144] Training effect:

[0145] After the reinforcement learning fine-tuning stage, the model's mathematical reasoning ability was significantly enhanced, especially in complex problems, where its performance was significantly better than that of the base model that had not undergone reinforcement learning fine-tuning.

[0146] During the model evaluation phase, this embodiment prepared 1,000 primary and secondary school math problems as an evaluation data set (not involved in training). The accuracy of the model before reinforcement learning was approximately 0.63, and the accuracy of the model after reinforcement learning was approximately 0.77.

[0147] Step S3: Phase II: Fine-tuning of supervised instruction (teacher style formation phase).

[0148] The purpose of this step is to make the model transition from "knowledge" to "personalization".

[0149] Step S31: Teacher style enhancement dataset.

[0150] A question-and-answer dataset containing approximately 10,000 typical test questions from various subjects in primary and secondary schools and their corresponding teacher-style explanations was constructed. This dataset, called the "Teacher Style Enhanced Dataset," reflects the expression methods and explanation logic commonly used by teachers in classroom lectures.

[0151] Step S32: Supervisory instruction fine-tuning.

[0152] Based on the first stage of strengthening reasoning ability, fine-tuning is performed using supervised fine-tuning (SFT). First, the teacher-style dataset is converted into data pairs in the form of Alpaca instructions, with the input being the test questions and the output being the test question analysis.

[0153] On this basis, this embodiment designs a personalized prompt word based on Role-Play. This can better enable the large model to learn the expected role characteristics and behavior patterns, while maintaining reasoning capabilities and achieving a personalized interactive experience.

[0154] #Role

[0155] You are a seasoned K-12 teacher. Drawing on years of experience in education, you possess a deep understanding of the diverse knowledge base across all disciplines. Not only can you accurately answer questions across a wide range of subjects, but you are also adept at analyzing the human mindset when approaching complex problems, clearly presenting this progressive reasoning process in a vivid, natural, and cognitively accurate manner.

[0156] #Require

[0157] 1. After receiving a subject question from a student, use progressive thinking, starting from the basic concepts and gradually deducing the final answer, clearly explaining the logical basis of each step.

[0158] 2. Develop the ability to think from multiple perspectives, explore a variety of different problem-solving strategies, and comprehensively demonstrate different problem-solving paths and applicable scenarios.

[0159] 3. Incorporate intuitive elements to share sudden inspirations that may occur during the problem-solving process and how these inspirations guide the solution. This foundation creates a new analytical process that is authentic, natural, and consistent with human cognitive characteristics.

[0160] In actual fine-tuning, LoRA efficient parameter fine-tuning is used to reduce the occupation of computing resources. In this embodiment, several linear transformation matrices of the base model are first Insert the LoRA adapter, and its update can be expressed as

[0161]

[0162] Among them, the rank , the original W remains frozen, and only the newly added A and B are trained. A and B are two matrices. For example, a 3*3 original matrix is ​​split into AB, which is 3*1 and 1*3. Then the training amount is reduced from 9 to 6.

[0163] The training process is to feed the "question-analysis" instruction pair into the model to generate predicted text , use cross entropy to measure the gap with the "standard answer" (the "standard answer" here refers to the text sent in), and use the Adam optimizer to perform gradient updates on A and B.

[0164] Step S33: teacher style formation;

[0165] The model, fine-tuned through supervised instructions, is not only able to solve specific subject problems, but also remarkably demonstrates a professional teaching style similar to that of primary and secondary school teachers. The problem-solving process generated by the model is detailed, clear, and easy for students to understand.

[0166] Working Principle: This embodiment addresses the three major shortcomings of existing general-purpose large models and fine-tuning solutions: 1. High computing / deployment costs; 2. Limited accuracy for complex problems; 3. Lack of personalized style. It proposes a two-stage fine-tuning framework of "small parameter size + reinforcement learning + LoRA-SFT", which fundamentally improves the model's usability and teaching value in primary and secondary education scenarios.

[0167] The rest of this embodiment is the same as that of the above-mentioned embodiment 1, and therefore will not be described in detail.

[0168] Example 3:

[0169] This embodiment, based on any one of the above-mentioned embodiments 1 and 2, proposes a system for constructing a large model for primary and secondary education tutoring, which is used to execute the above-mentioned method for constructing a large model for primary and secondary education tutoring; it includes an initial unit, a reinforcement unit, and a supervision and fine-tuning unit;

[0170] The initial unit is used to use Qwen2.5-3B-Instruct as a basic model and construct a training data set based on the obtained teaching data set;

[0171] The reinforcement unit is used to construct a reward function based on the training data set to obtain a reward value, and call a reinforcement learning method to adjust the model parameters to obtain a reinforced basic model;

[0172] The supervised fine-tuning unit is used to fine-tune the strengthened basic model in a supervised instruction fine-tuning manner according to the constructed teacher style data set and the set personalized prompt words, so as to obtain a large model for primary and secondary school education tutoring.

[0173] This embodiment proposes an electronic device, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the above-mentioned method of constructing a large model for primary and secondary school education tutoring is implemented.

[0174] This embodiment also proposes a computer-readable storage medium, on which computer instructions are stored; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned method for constructing the large model of primary and secondary school education tutoring is implemented.

[0175] The rest of this embodiment is the same as any of the above-mentioned embodiments 1 and 2, and thus will not be described in detail.

[0176] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A method for constructing a large model for primary and secondary school education and guidance, characterized in that: The specific steps include: Step S1: Use Qwen2.5-3B-Instruct as the basic model and construct a training dataset based on the obtained teaching dataset; Step S2: Based on the training data set, a reward function is constructed to obtain a reward value, and the reinforcement learning method is called to adjust the model parameters to obtain a reinforced basic model; Step S3: Based on the constructed teacher style dataset and the personalized prompt words set, the enhanced basic model is fine-tuned by means of supervised instruction fine-tuning to obtain a large model for primary and secondary education tutoring; The step S2 specifically includes the following steps: Step S21: Construct a reward function based on the answer generated by the basic model, the standard answer, the similarity function, and the set reward parameters, and calculate the reward value; Step S22: Calculate the probability distribution according to the set regularization coefficient, the set clipping threshold, and the model parameters; Step S23: Calculate the KL divergence based on the reward value and probability distribution; Step S24: adjusting the regularization coefficient according to the upper and lower limits of the KL divergence to obtain a strengthened basic model; The step S22 specifically includes the following steps: Step S221: Calculate the strategy ratio based on the questions obtained from the training data set, the model sampled answers, and the large model strategy; Step S222: Calculate advantage estimate based on the reward value and the average reward of all questions in the current buffer; Step S223: Calculate the probability of the basic model generating the answer with the model parameter θ according to the set regularization coefficient, the set clipping threshold, the advantage estimate, and the strategy ratio to obtain a probability distribution; The step S3 specifically includes the following steps: Step S31: constructing a teacher style enhanced dataset based on the obtained typical test questions and the corresponding question-answer pair datasets explained in the teacher style; Step S32: Convert the teacher style enhancement dataset into data pairs in the form of Alpaca instructions, and call Role-Play to construct personalized prompt words; Step S33: fine-tune the enhanced basic model by means of supervised instruction fine-tuning to obtain a large model for primary and secondary education tutoring; The step S33 specifically includes the following steps: Step S331: inserting LoRA adapters into several linear transformation matrices of the basic model; Step S332: Keep the original linear transformation matrix frozen and train the newly added LoRA adapter matrix; Step S333: Based on the data pairs in the form of Alpaca instructions, the Adam optimizer is called to perform gradient updates on the LoRA adapter matrix to obtain the basic model after supervised instruction fine-tuning, i.e., the primary and secondary school education tutoring model.

2. The method for constructing a large model for primary and secondary education tutoring according to claim 1, characterized in that: The specific operation of step S24 is: adaptively adjust the regularization coefficient according to the upper and lower limits of the KL divergence. If the KL divergence is greater than the upper limit of the KL divergence, the regularization coefficient λ is adjusted. KL Adjust to If the KL divergence is greater than the upper limit of the KL divergence, the regularization coefficient λ KL Adjust to .

3. The method for constructing a large model for primary and secondary school education and guidance according to claim 1 is characterized in that: The specific steps of step S1 are: using Qwen2.5-3B-Instruct as a basic model, and constructing a composite mathematical reasoning training dataset based on the acquired high-quality mathematical reasoning dataset and the local private primary and secondary school mathematics dataset.

4. A system for constructing a large model for primary and secondary school education and guidance, used to execute the method for constructing a large model for primary and secondary school education and guidance as claimed in claim 1; characterized in that: Includes initial unit, reinforcement unit, and supervision fine-tuning unit; The initial unit is used to use Qwen2.5-3B-Instruct as a basic model and construct a training data set based on the obtained teaching data set; The reinforcement unit is used to construct a reward function based on the training data set to obtain a reward value, and call a reinforcement learning method to adjust the model parameters to obtain a reinforced basic model; The supervised fine-tuning unit is used to fine-tune the strengthened basic model in a supervised instruction fine-tuning manner according to the constructed teacher style data set and the set personalized prompt words, so as to obtain a large model for primary and secondary school education tutoring.

5. An electronic device, characterized in that: It includes a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the method for constructing a large model for primary and secondary school education tutoring as described in any one of claims 1-3 is implemented.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions; when the computer instructions are executed on the electronic device as described in claim 5, the method for constructing a large model for primary and secondary school education tutoring as described in any one of claims 1-3 is implemented.

Images