A tower-like construction method for educational large models based on multi-level experiential learning

Abstract

The present application relates to the technical field of large language models. In order to solve the limitation that the traditional large model fine-tuning framework relies only on abstract experience to perform educational field migration, which makes it difficult to respond to teaching applications other than basic knowledge question-and-answer tasks, a method for constructing a large educational model tower based on multi-level experience learning, a computer device, a computer-readable storage medium, and a computer program product are disclosed. The method includes obtaining abstract experience based on multi-source heterogeneous knowledge carriers, injecting a general large language model to construct a first educational model; processing teaching behavior flow data to obtain observation experience, injecting it into the first educational model to construct a second educational model; obtaining practical experience based on the preference data feedback from the application of the second educational model, and performing preference alignment of the second educational model to construct a third educational model. The use of this method can realize the transfer training of the general large language model in the field of education, and improve the educational application performance of the educational model such as knowledge question-and-answer, question setting, and problem solving.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a method for constructing a large educational model tower based on multi-level experiential learning, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] The advent of large language models (hereinafter referred to as large models) represented by ChatGPT has pushed generative artificial intelligence to a new level, demonstrating powerful language comprehension, content generation, problem planning capabilities, etc., and has shined in various natural language processing tasks (translation, dialogue, information extraction, sentiment recognition, etc.), and has also shown strong application prospects in general fields.

[0003] Although the current mainstream large models perform amazingly on general tasks, they still have the following obvious shortcomings when solving segmented and vertical field problems: 1. They are not very professional and lack most of the domain expertise and conceptual understanding; 2. They are not reliable and their applications are limited by outdated information and illusions; 3. The mainstream fine-tuning paradigm is simple and crude, mindlessly instilling abstract knowledge (text information) without considering deeper domain experience, and the data construction method is simple, the data structure is single, and the training objectives have little difference, so it is difficult to fully adapt to the application needs of the target field.

[0004] Unlike general fields, the characteristic of the education field is that it focuses on inspiration rather than indoctrination. It requires not only that the big model can accurately "answer" but also that it can "teach". Just as excellent teachers not only need to learn a lot of theoretical knowledge, but also need to observe demonstration classes through internships and accumulate long-term teaching experience, that is, they need long-term observation and practical accumulation before they can truly "teach" and be called "understanding education".

[0005] At present, large language models for the education field (hereinafter referred to as educational large models) often focus on theoretical knowledge learning, ignore the understanding of specific scenarios, lack the accumulation of practical experience, and are difficult to truly meet the unique needs of the education field. In addition, the traditional method of building educational large models only performs domain transfer training around knowledge question-answering tasks, but cannot provide more teaching application services. Summary of the invention

[0006] In order to solve the limitation that the traditional fine-tuning framework that only relies on abstract experience to construct the educational big model is difficult to respond to the domain transfer training of teaching applications other than basic knowledge question and answer tasks, and to break through the application bottleneck of the current educational big model that "can only answer but not teach", the present invention provides a tower construction method of an educational big model based on multi-level experience learning, a computer device, a computer-readable storage medium and a computer program product, which realizes the transfer training of a large language model for general fields (referred to as the general large language model) applied to the field of education, and improves the educational application performance of the educational big model such as knowledge question and answer, personalized teaching, question setting and problem solving.

[0007] To achieve the above object, according to a first aspect of the present invention, a method for constructing a large educational model tower based on multi-level experiential learning is provided, the method comprising:

[0008] Obtain abstract experience based on text data recorded in multi-source heterogeneous knowledge carriers;

[0009] Based on abstract experience, we train a large language model for general fields and build the first large education model;

[0010] Acquiring observation experience, the observation experience includes behavior flow data generated by the first subject performing a specific teaching task based on the cognition of abstract experience;

[0011] The first educational model is trained based on observation experience to construct the second educational model;

[0012] Acquire practical experience, the practical experience includes preference data with feedback information generated by the second educational model performing specific teaching tasks, the feedback information is generated by the second object, and the second object has higher-order values ​​than the first object;

[0013] The second big education model is trained based on practical experience and the third big education model is constructed.

[0014] Furthermore, abstract experience is obtained based on the text data recorded in the multi-source heterogeneous knowledge carriers, including preprocessing the text data recorded in the multi-source heterogeneous knowledge carriers to obtain abstract experience. The abstract experience includes text knowledge, graph knowledge, arithmetic knowledge and general knowledge. The abstract experience is represented in the form of an unstructured sequence text data set.

[0015] Furthermore, the large language model for general fields is trained according to the abstract experience to construct the first educational large model, including inputting the abstract experience into the large language model for general fields, performing unsupervised learning, and constructing the first educational large model.

[0016] Furthermore, the observation experience includes multiple sample questions and corresponding sample answers. The first educational model is trained based on the observation experience to construct the second educational model, including obtaining the observation experience; inputting each sample question into the first educational model and outputting the actual answer; and conducting supervised training on the first educational model in the direction of minimizing the difference between the actual answer and the sample answer until the training stopping condition is reached, thereby obtaining the second educational model.

[0017] Furthermore, obtaining observation experience includes extracting key information from abstract experience to obtain multiple sample questions; obtaining the result corresponding to each sample question answered by the first subject; obtaining the idea of ​​obtaining the result answered by the first subject; and obtaining observation experience based on multiple sample questions, ideas and results.

[0018] Furthermore, the practical experience includes multiple sample instructions and corresponding preference data, the preference data includes positive preference solutions and negative preference solutions, the positive preference solutions are more in line with the preferences indicated by the feedback information than the negative preference solutions, the second education big model is trained according to the practical experience, and the third education big model is constructed, including inputting each sample instruction into the second education big model and outputting the actual solution; in the direction of minimizing the difference between the actual solution and the positive preference solution, the second education big model is subjected to preference alignment training until the training stop condition is reached, and the third education big model is obtained.

[0019] Furthermore, obtaining practical experience includes obtaining multiple sample instructions and corresponding feedback information; inputting each sample instruction into the second education model to obtain multiple different replies; ranking the multiple different replies by preference based on the feedback information to obtain positive preference solutions and negative preference solutions; and obtaining practical experience based on the multiple sample instructions, positive preference solutions and negative preference solutions.

[0020] According to a second aspect of the present invention, there is also provided a computer device, which includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of any one of the above methods.

[0021] According to a third aspect of the present invention, there is also provided a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0022] According to a fourth aspect of the present invention, there is also provided a computer program product, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0023] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0024] The present invention provides a tower-like construction method of a large educational model based on multi-level experience learning, which utilizes abstract experience, observational experience and practical experience to train a large language model for a general field, so that the large language model fully integrates abstract experience, observational experience and practical experience, realizes the transfer training of the large language model for a general field in the field of education, and improves the educational application performance of the large educational model in knowledge question and answer, personalized teaching, question setting and problem solving, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1A A schematic diagram of the tower of experience theory provided for the embodiments of the present application;

[0027] Figure 1B A schematic diagram of a tower-like construction method of a large educational model based on multi-level experiential learning provided in an embodiment of the present application;

[0028] Figure 2 A flowchart of a method for constructing a large educational model tower based on multi-level experiential learning provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the steps of using a controllable generation mode given by GPT4 provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of a training sample of the second education model in the step of acquiring question setting experience provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of the steps of the fine-grained solution mode provided by GPT4 provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of a training sample of the second educational model in the step of acquiring problem-solving experience provided in an embodiment of the present application;

[0033] Figure 7 A schematic diagram of a training sample of the second large education model in the step of acquiring task execution failure experience of the model provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of the steps for obtaining practical experience provided by the embodiments of the present application;

[0035] Fig. 9A schematic diagram of the form of training samples of the third educational model provided in an embodiment of the present application;

[0036] Fig.10 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0039] The "Tower of Experience" theory was proposed by the famous American educational psychologist Edgar Dale. Figure 1A As shown, the theory divides human learning experience into three levels: abstract experience, observational experience and practical experience, and believes that each level of experience is interconnected and mutually penetrating, and only by fully integrating the three levels of experience can high-quality learning be achieved.

[0040] Therefore, based on the "Tower of Experience" theory, this embodiment provides a method for constructing a tower-like educational model based on multi-level experience learning. This method integrates abstract experience learning, observation experience learning, and practical experience learning, improves the knowledge question-answering ability of the educational model, and makes the educational model based on three levels of experience learning more intelligent, not only "answering well" but also "teaching well". Figure 1B As shown, the method includes four stages and is implemented in a tower-like or step-by-step manner, including: building a large educational model, learning abstract experience (L1 stage), learning observational experience (L2 stage), and learning practical experience (L3 stage).

[0041] refer to Figure 2 In one embodiment, a method for constructing a large educational model tower based on multi-level experiential learning is provided, comprising the following steps:

[0042] Step 201, acquiring abstract experience based on text data recorded in multi-source heterogeneous knowledge carriers.

[0043] Among them, multi-source heterogeneous knowledge carriers can be knowledge carriers such as Internet corpus, textbooks, handouts, education books, psychology books, etc. Abstract experience refers to all knowledge and concepts related to the field of education, or the prerequisite knowledge required to perform specific teaching tasks. Abstract experience can be textual knowledge, graph knowledge, or arithmetic knowledge related to the field of education, or general knowledge unrelated to the field of education.

[0044] Exemplarily, the terminal preprocesses the text data recorded in the multi-source heterogeneous knowledge carriers to obtain abstract experience, which is represented in the form of an unstructured sequence text data set, that is, x={w_1, w_2, …, w_x}.

[0045] Step 202, training a large language model for a general domain based on abstract experience to construct a first large education model.

[0046] Among them, the large language model for general fields is a general large multimodal language model (Large Multimodal Models, LMM for short), which can be ChatGPT. The large language model can be built by the terminal itself, or it can be built by other terminals and obtained by the terminal.

[0047] Exemplarily, the terminal inputs abstract experience into a large language model for a general field, performs unsupervised learning, and constructs a first education large model (i.e., the education large model of the L1 stage, hereinafter referred to as the L1 education large model).

[0048] Step 203, obtaining observation experience, the observation experience includes the behavior flow data generated by the first object performing a specific teaching task based on the cognition of the abstract experience. The first object can be a human or machine subject, such as a professional field personnel such as a teacher or a student, or a large language model such as an existing large language model for a general field or a large language model for the education field. Specific teaching tasks can be visiting exhibitions, watching demonstration classes, watching other teachers' classroom videos, preparing lessons, setting questions, solving problems, answering questions, etc. Observational experience is represented in the form of a text data set. For example, when the specific teaching task is to solve a problem, observational experience can be represented as x={q, a}={w_1, w_2, …, w_q , w_(q+1), w_(q+2), …, w_(q+a)}, where q={w_1, w_2, …,w_q}, where q represents a sample question and a represents a sample answer, q={w_1, w_2, …,w_q}, a={w_(q+1), w_(q+2), …, w_(q+a)}. Observational experience is used to allow the L1 education model to learn the educational thinking of professionals such as teachers or the educational application performance of existing large language models such as knowledge question answering, and to learn from the erroneous inference data generated by the existing large language model.

[0049] Step 204, training the first educational model based on observation experience and constructing a second educational model.

[0050] Exemplarily, the terminal performs supervised training on the first educational big model based on observation experience to construct a second educational big model (i.e., the educational big model of the L2 stage, hereinafter referred to as the L2 educational big model).

[0051] Step 205, obtaining practical experience, the practical experience includes preference data with feedback information generated by the second educational model performing a specific teaching task, the feedback information is generated by the second object, and the second object has a higher-order value than the first object.

[0052] Among them, compared with the first object, the second object has higher-order domain values, that is, the second object can be a professional field personnel or a trained large language model. For example, the feedback information can be provided by professional field personnel such as teachers or trained large language models such as GPT4. Practical experience is represented in the form of a text data set. For example, when the specific teaching task is to solve a problem, practical experience can be represented as x={q, a1, a2}={w_1, w_2, …, w_q, w_(q+1), w_(q+2),…, w_(q+a), w_(q+a+1), w_(q+a+2),…, w_(q+a+a)}, where q={w_1, w_2, …, w_q}, where q represents task instructions, a1 represents high preference responses, and a2 represents low preference responses, q={w_1, w_2, …, w_q}, a1={w_(q+1), w_(q+2), …, w_(q+a)}, a2={ w_(q+a+1), w_(q+a+2),…, w_(q+a+a)}. Practical experience is used to correct the reasoning preferences (generative habits) of the L2 education model with higher-order domain values, allowing it to gradually evolve in the process of practice.

[0053] Step 206, training the second educational model based on practical experience to construct a third educational model.

[0054] Exemplarily, the terminal performs preference alignment training on the second education big model based on practical experience, and constructs the third education big model (i.e., the education big model of the L3 stage, hereinafter referred to as the L3 education big model).

[0055] This embodiment provides a tower-like construction method for a large educational model based on multi-level experience learning, which utilizes abstract experience, observational experience, and practical experience to train a large language model for a general field, thereby realizing the transfer training of the large language model for a general field in the field of education, and improving the educational application performance of the large educational model, such as knowledge question-answering, personalized teaching, question-setting, and problem-solving capabilities.

[0056] In one embodiment, text knowledge is represented as T={s_1, s_2, …, s_t}, where s_i represents a text sequence and t represents the total number of collected text sequences. The steps of preprocessing the text data recorded in the multi-source heterogeneous knowledge carrier to obtain text knowledge include: first, splicing multiple text data recorded in the multi-source heterogeneous knowledge carrier to obtain a super-long text sequence (S_T); then, dividing the super-long text sequence (S_T) into units of 4096 to obtain a number of text sequences s_i, then the maximum length of the text sequence s_i is 4096, which can be represented as s_i={w_1,w_2,…,w_4096}, where w_i represents a token (character), that is, the minimum input unit in the large language model training stage; finally, combining a number of text sequences s_i to obtain text knowledge T={s_1, s_2, …, s_t}.

[0057] For example, the text knowledge is {“response”: “The first place to the right of the decimal point is called the tenth place, and the counting unit is one tenth (0.1); the second place is called the hundredth place, and the counting unit is one hundredth (0.01)... The largest counting unit of the decimal part is one tenth, and there is no smallest counting unit. The number of digits in the decimal part is called the number of decimal places...”}.

[0058] In one embodiment, graph knowledge is represented as G=<{N_1, N_2,…, N_g}, E>, where N_i represents a node set of a certain node type, such as a node set of knowledge points, concept descriptions, examples or other descriptions, which can be represented as N_i={n_1, n_2,…, n_n}. E represents a set of edge information between nodes, which can be represented as E={e_1, e_2,…,e_e}. If there is an edge between two nodes, it means that the information contained in the two nodes is related (for example, there is an edge between a knowledge point and an example, which means that the example examines the mastery of the knowledge point). The steps of preprocessing the text data recorded in the multi-source heterogeneous knowledge carriers to obtain graph knowledge include: first, generating a graph G by random sampling based on the multiple text data recorded in the knowledge carriers, randomly selecting a node in the graph G, and then using the edge information as the path for a set number of random walk steps (for example, 4 steps), and then obtaining a summary information containing 5 nodes (such as 2 knowledge points + 2 concept descriptions + 1 example question), and the summary information is represented in the form of a text data set.

[0059] For example, the graph knowledge about knowledge point-example question is {“response”: “The perimeter of rectangle and square” is a knowledge point for grade 3. The specific meaning of this knowledge point is: The perimeter of rectangle and square is determined by their side lengths. For a rectangle, its perimeter is equal to twice “length + width”. Specifically,…

[0060] Here are some examples of questions that cover the knowledge point of "perimeter of rectangles and squares":

[0061] Question: A rectangle made of wire is 60 cm long and 20 cm wide. Straighten the wire and make a square. What is the area of ​​the square? How many square decimeters?

[0062] Solution: First find the side length of the square. The perimeter of a rectangle is equal to (length plus width) multiplied by 2, and the perimeter of a square is equal to the perimeter of a rectangle...}.

[0063] In one embodiment, arithmetic knowledge is represented as A={a_1, a_2,…, a_a}, where a_i represents a mathematical equation (such as 1+2=3, 4*4=16), and a represents the total number of mathematical equations collected. The steps of preprocessing the text data recorded in the multi-source heterogeneous knowledge carrier to obtain arithmetic knowledge include: first, based on the text data of multiple arithmetic types recorded in the knowledge carrier, randomly select a number of test questions (for example, 10 questions) to form a training sample, which is represented as: "The following are 10 arithmetic equations with correct calculations. Please remember their form, understand their pattern, and master their internal rules: a_1, a_2,…, a_10".

[0064] For example, arithmetic knowledge is {“response”: Here are 5 basic math calculation questions:

[0065] Question 1: Find -8074*0 Analysis: -8074*0 = 0

[0066] Question 2: What is -3563+4241*8407%+2270.1463327996-7735 / 8885=? Analysis: ...

[0067] Question 3: ...

[0068] Question 4: ...

[0069] Question 5: ...}

[0070] In one embodiment, the representation form of general knowledge is similar to that of textual knowledge, and the acquisition steps are the same as those of textual knowledge. The format of the obtained training samples is also exactly the same, except that the content is common sense knowledge irrelevant to the field of education, and the source can be: Wikipedia, SkyPile-150B, MOSS, BELLE and other Internet open source corpora. The purpose of training the large language model based on general knowledge is to alleviate the phenomenon of forgetting common sense knowledge irrelevant to the field of education while the large language model learns knowledge in the field of education.

[0071] In one embodiment, the above step 202 trains a large language model for a general field based on abstract experience to construct an L1 education large model, that is, performs the L1 stage: learning of abstract experience. The input training sample of the L1 stage is x={w_1,w_2,…w_x}, and the input training sample is input into the LMM model for learning. For example, w_1 is input into the LMM model to predict w_2; w_1 and w_2 are input into the LMM model to predict w_3; and so on. The back propagation based on the LMM model training starts from the first input token to calculate the loss function loss. The loss function of the LMM model is:

[0072] ,

[0073] In the above formula, Represents the parameters of the loss function of the LMM model, and the superscript 0 represents the LMM model.

[0074] In one embodiment, the observation experience includes a plurality of sample questions and corresponding sample answers. The above step 204 trains the first educational model based on the observation experience to construct the second educational model, including inputting each sample question into the first educational model and outputting the actual answer; and performing supervised training on the first educational model in the direction of minimizing the difference between the actual answer and the sample answer until the training stop condition is reached, thereby obtaining the second educational model. The training stop condition may be that the loss function obtains the minimum value, or that all the observation experiences have been trained once.

[0075] Among them, the first educational model is trained according to the observation experience, that is, the L2 stage is executed: learning of observation experience. The input training sample form of the L2 stage is a single round or multi-round question-answer pair. Let q represent the sample question, a represent the sample answer, q={w_1, w_2, …, w_q}, a={w_(q+1), w_(q+2), …, w_(q+a)}, then the input training sample is x={q, a}={w_1, w_2, …, w_q , w_(q+1), w_(q+2), …, w_(q+a)}. Although the input training samples are all text sequences as in the L1 stage, the difference between the L2 stage and the L1 stage is that the sample questions (i.e., q) in the text sequence are masked, that is, the mask vector corresponding to the input training sample x is mask_x={0, 0 …, 0 , 1,1, …, 1}, where the first q 0s represent that the loss of this part is not calculated in the L1 education model training stage, and the last a 1s represent that the loss of this part of the text is calculated. In other words, the back propagation of the L1 education model training starts from the input q+1th token (the first token of the sample answer a) to calculate the loss function loss, that is, the L1 education model only learns to generate answers to given questions. The loss function of the L1 education model is:

[0076] ,

[0077] In the above formula, Represents the parameters of the loss function of the L1 education model, and the upper right subscript 1 represents the L1 education model.

[0078] The observation experience is the behavior flow data generated by the first subject performing a specific teaching task based on its cognition of abstract experience, including the first subject's successful task execution experience and failed task execution experience. The first subject's successful task execution experience may be automatic question setting or automatic problem solving by professional field personnel / large language model, and the first subject's failed task execution experience may be failed task execution experience by large language model.

[0079] In one embodiment, the above step 203, obtaining observation experience includes: extracting key information from abstract experience to obtain multiple sample questions; obtaining the result corresponding to each sample question answered by the first object; obtaining the idea of ​​obtaining the result answered by the first object; and obtaining observation experience based on multiple sample questions, ideas and results.

[0080] In one embodiment, the steps for obtaining observation experience are described with automatic question setting in the teaching process as the target task. In the application scenario where automatic question setting is the target task, the observation experience can be the question setting experience of professionals / large language models, and the question setting experience can be a triple consisting of question setting requirements, question setting ideas, and original test questions, where the question setting requirements correspond to sample questions, and the question setting ideas and original test questions correspond to sample answers.

[0081] The steps to obtain the question-setting experience of professionals / large language models include:

[0082] First, collect exercises and test papers covering all stages from elementary school to junior high school, and extract math questions from them. Secondly, based on manual annotation, extract the key information of math questions from dimensions such as stage, question type, knowledge points and difficulty, and reversely deduce the question requirements. Professionals or large language models determine original questions based on the question requirements. Finally, construct "requirement-question" pairs, manually write examples of question ideas, and use GPT4 (the language model released by OpenAI for the chatbot ChatGPT) in a few-shot manner to batch supplement question ideas. Taking into account the differences in cognitive levels and writing styles of people in different professional fields, it will make it difficult to align the granularity of question ideas. Therefore, in order to generate fine-grained question ideas, use GPT4 to generate question ideas with consistent writing style, referring to Figure 3 , using the mode given by GPT4: controllable generation. In this controllable generation mode, the learning stage, knowledge points, question types and difficulty level in the question setting requirements are used as input, and "Please construct an original math problem that meets the requirements based on the learning stage range, knowledge points, question types and difficulty level given in the input. It is required to record the question setting ideas and original test questions step by step in a rigorous and specific manner. The question setting ideas must include the design ideas of knowledge points, difficulty, question stems, data, etc." is used as the task, so that the question setting ideas with consistent writing style can be generated. Finally, reference Figure 4 ,The training samples of the L1 education model are in the form of {question requirements-question ideas-original test questions}.

[0083] In one embodiment, the steps for obtaining observation experience are described with automatic problem solving in the teaching process as the target task. In the application scenario where automatic problem solving is the target task, the observation experience can be the problem solving experience of professionals / large language models, and the problem solving experience can be a triple consisting of problem solving requirements (also called test questions / questions), problem solving ideas (also called analysis), and answers, where the problem solving requirements correspond to sample questions, and the problem solving ideas and answers correspond to sample answers.

[0084] The steps to obtain problem-solving experience of professionals / large language models include:

[0085] First, we collected exercises and test papers covering all grades from elementary school to junior high school, and extracted math questions from them. Then, we hired professional personnel such as normal school students to write solutions and answers for the collected math questions. Considering that there are differences in the cognitive level and writing style of personnel in different professional fields, it will make it difficult to align the granularity of the analysis. Therefore, in order to generate fine-grained analysis, GPT4 is used to generate high-quality analysis with consistent writing style based on the manual analysis data. Reference Figure 5 , using the mode given by GPT4: fine-grained solution, in this mode, the question (image format and / or text format) is used as input, and "please give a rigorous and specific answer to the math problem given in the input, and record the solution ideas and process step by step in detail" is used as the task, so that high-quality analysis with consistent writing style can be generated. Finally, reference Figure 6 ,The training samples of the L1 education model are in the form of {problem-solving requirements-problem-solving ideas-answers}.

[0086] For example, there are three ways to generate problem-solving experience:

[0087] 1) Input the question, GPT4 solves the question directly, that is: {question}↣{answer}, and the training sample of the L1 education model is in the form of {question-answer};

[0088] 2) Input the question and answer, and GPT4 will provide additional analysis, i.e. {question-answer}↣{analysis}, so the training sample of the L1 education model is in the form of {question-analysis-answer};

[0089] 3) Input the complete question, analysis and answer. GPT4 rewrites the analysis so that the format of all analyses is consistent, that is, {question-analysis-answer}↣{modified analysis}. The training sample of the L1 education model is in the form of {question-modified analysis-answer}.

[0090] Among them, method 1) is applicable to the situation where no answer and analysis are provided, method 2) is applicable to the situation where no analysis is provided, and method 3) is applicable to the situation where questions, analysis and answers are provided.

[0091] In one embodiment, the steps for obtaining the model's task execution failure experience during the teaching process are described. The model's task execution failure experience mainly comes from a large language model for general domains that has not been trained for domain transfer, such as GPT4 that is in the training stage and has not been completed. In the application scenario where the model's task execution failure experience is used as observation experience, the failure experience can be a reference composed of task instructions, collaboration information, and guidance feedback. Figure 7 The triples in , where the task instructions correspond to sample questions, and the collaborative information and guided feedback correspond to sample answers.

[0092] The steps for obtaining the model's task execution failure experience include:

[0093] First, we build a collaborative environment consisting only of large language models, where GPT4 plays the role of discriminator and multiple large language models for general fields (such as Qwen-VL-Chat Yi-VL-6 / 34B, etc.) play the role of generator. Figure 7 The content displayed in the blue background box in the figure), randomly assigns two or more generators to complete the task in a collaborative or independent manner, and obtains collaborative information (reference Figure 7 The discriminator then guides and evaluates the task completion of these two or more generators to obtain guidance feedback. Finally, the training samples of the L1 education model are in the form of {task instructions-collaboration information-guidance feedback}.

[0094] In one embodiment, the practical experience includes multiple sample instructions, positive preference solutions and negative preference solutions, and the positive preference solutions are more consistent with the preferences indicated by the feedback information than the negative preference solutions. In the above step 206, the second educational model is trained according to the practical experience, and the third educational model is constructed by inputting each sample instruction into the second educational model and outputting the actual solution; the second educational model is trained for preference alignment in the direction of minimizing the difference between the actual solution and the positive preference solution until the training stop condition is reached, thereby obtaining the third educational model. The training stop condition can be that the loss function obtains the minimum value, or that all practical experiences have been trained once.

[0095] Among them, the second education model is trained according to practical experience, that is, the L3 stage: learning of practical experience is executed. The loss function of the L2 education model in the L3 stage is different from the loss function of the L1 stage and the L2 stage. Its focus is not on learning the generation ability, but on correcting its generation preference on the basis of the generation ability established in the L1 stage and the L2 stage, so that the generated answers are more in line with the expectations of actual applications, that is, to increase the probability of the L3 education model generating preferred solutions (also called positive preference solutions) and reduce the probability of non-preferred solutions (also called negative preference solutions). Preferred solutions and non-preferred solutions can be defined according to specific needs. For example, if the correctness of the generated answer exceeds 80%, it is defined as a preferred solution, otherwise it is defined as a non-preferred solution.

[0096] Specifically, although the input forms of the loss functions in the L1 stage and the L2 stage are different, they are both cross entropy loss functions. The loss function of the L2 education model in the L3 stage is the Direct Preference Optimization (DPO) algorithm, which is expressed as:

[0097]

[0098] In the above formula, represents the parameters of the loss function of the L2 education model, represents the parameters of the L3 education model, and is represented by As its initial solution, for the same input , and denote preferred solutions and non-preferred solutions respectively.

[0099] In one embodiment, the above step 205, obtaining practical experience includes obtaining multiple sample instructions and corresponding feedback information; inputting each sample instruction into the second education model to obtain multiple different replies; according to the feedback information, performing preference sorting on the multiple different replies to obtain positive preference solutions and negative preference solutions; based on the multiple sample instructions, positive preference solutions and negative preference solutions, obtaining practical experience.

[0100] In one embodiment, reference Figure 8 , the steps for acquiring practical experience are explained. First, obtain the task instruction (prompt) (i.e. sample instruction), for example: "Please act as a teaching assistant and assist the teacher to complete all possible questions and answers for the students", or "Please answer the input math test questions", etc. Secondly, input the instruction (prompt) into the L2 education large model. The L2 education large model can generate multiple different responses for one instruction based on the randomness of its own reasoning. Finally, obtain feedback information from professional personnel or trained large language models, and based on the feedback information, perform preference sorting on multiple different responses to obtain positive preference solutions and negative preference solutions. Finally, as Fig. 9 As shown, the training samples of the L2 education model are in the form of {task instruction-positive preference solution-negative preference solution}. Among them, the positive preference solution and the negative preference solution are any two replies from multiple different replies, among which the positive preference solution is more consistent with the preference indicated by the feedback information than the negative preference solution. For example, for the same instruction, the L2 education model generated a total of 9 replies and sorted them by preference, among which the reply with the first serial number was the positive preference solution, and the reply with the ninth serial number was the negative preference solution.

[0101] By adopting the tower-like construction method of a large educational model based on multi-level experience learning provided in this embodiment, a large educational model can be built from scratch, or a large language model for a general field can be completed for transfer training in the field of education. On the basis of supplementing the professional knowledge of the large language model, the injection of observation experience and practical experience in teaching is realized, so that it can not only "answer well" (the answers generated by the model are highly correct and more in line with the expectations of actual applications), but also "teach well" (in addition to generating answers to questions, the model also generates analysis), and truly understands education better. Specifically, it includes but is not limited to the following aspects:

[0102] In terms of answering questions, the knowledge reserves are richer and more professional, the answers are more accurate, and it can trigger Socratic teaching, while taking into account the emotional support (empathy) of learners while making full use of teaching strategies.

[0103] In terms of personalized teaching, teaching strategies can be adjusted for different students, teaching can be tailored to the students’ aptitude, and the cognitive and emotional states of questioners can be controlled more accurately.

[0104] In terms of problem solving and question setting, it is superior to other existing open source large-scale models of the same scale in terms of language fluency, answer accuracy, question solvability, instruction responsiveness, and analysis detail.

[0105] The present application also provides a computer device, whose internal structure diagram can be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a tower-like construction method of a large educational model based on multi-level experiential learning is implemented.

[0106] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0107] like Fig.10 As shown, the present application also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps in the above-mentioned method embodiments.

[0108] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0109] The present application also provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0110] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0111] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0112] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

[0113] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a large educational model tower based on multi-level experiential learning, characterized in that: include: According to the text data recorded in the multi-source heterogeneous knowledge carriers, abstract experience is obtained, wherein the abstract experience includes text knowledge, graph knowledge, arithmetic knowledge and general knowledge, and the abstract experience is represented in the form of an unstructured sequence text data set; According to the abstract experience, a large language model for general fields is trained to construct a first large education model; Acquire observation experience, the observation experience includes a plurality of sample questions and corresponding sample answers, the observation experience is represented in the form of a text data set; the observation experience includes behavior flow data generated by a first subject performing a specific teaching task based on cognition of abstract experience, the first subject is a human and / or a machine subject, the behavior flow data includes successful task execution experience and failed task execution experience of the first subject, the failed task execution experience is used for the first education macro model to learn from the erroneous reasoning data; The successful task execution experience includes observation experience in application scenarios of automatic question setting and automatic problem solving; in application scenarios where the specific teaching task is automatic question setting, the observation experience includes a triple consisting of question setting requirements, question setting ideas and original test questions, wherein the question setting requirements correspond to sample questions, and the question setting ideas and original test questions correspond to sample answers; in application scenarios where the specific teaching task is automatic problem solving, the observation experience includes a triple consisting of problem solving requirements, problem solving ideas and answers, wherein the problem solving requirements correspond to sample questions, and the problem solving ideas and answers correspond to sample answers; the failed task execution experience includes a triple consisting of task instructions, collaboration information and guidance feedback, wherein the task instructions correspond to sample questions, and the collaboration information and guidance feedback correspond to sample answers; the first education model is trained according to the observation experience to construct the second education model; Acquire practical experience, the practical experience comprising a plurality of sample instructions and a positive preference solution and a negative preference solution with feedback information generated by the second education macro model when executing a specific teaching task according to the sample instructions, the feedback information being generated by the second object, the second object having a higher-order value than the first object; the positive preference solution being more consistent with the preference indicated by the feedback information than the negative preference solution; The obtaining of practical experience includes obtaining a plurality of sample instructions and corresponding feedback information; inputting each sample instruction into the second large education model to obtain a plurality of different responses; According to the feedback information, a plurality of different replies are ranked by preference to obtain positive preference solutions and negative preference solutions; based on the plurality of sample instructions, the positive preference solutions and the negative preference solutions, practical experience is obtained; The second educational model is trained according to the practical experience to construct the third educational model. During the training process, the loss function of the second educational model is a direct preference optimization algorithm, and its expression is: In the above formula, represents the parameters of the loss function of the second education model, Represents the parameters of the third education model, and is represented by As its initial solution, for the same input , and represent positive preference solutions and negative preference solutions respectively.

2. The method according to claim 1, characterized in that The step of obtaining abstract experience based on text data recorded in multi-source heterogeneous knowledge carriers includes: Preprocess the text data recorded in multi-source heterogeneous knowledge carriers to obtain abstract experience.

3. The method according to claim 1 or 2, characterized in that The step of training a large language model for a general field based on the abstract experience to construct a first large education model includes: Input abstract experience into a large language model for general fields, perform unsupervised learning, and build the first large education model.

4. The method according to claim 1, characterized in that The step of training the first educational model based on the observation experience and constructing the second educational model includes: Input each sample question into the First Education Big Model and output the actual answer; The first educational model is supervisedly trained in the direction of minimizing the difference between the actual answer and the sample answer until the training stop condition is reached, thereby obtaining a second educational model.

5. The method according to claim 1, characterized in that The training of the second education model according to the practical experience to construct the third education model includes: Input each sample instruction into the second educational model and output the actual solution; The second educational model is subjected to preference alignment training in the direction of minimizing the difference between the actual solution and the positive preference solution until the training stop condition is reached, thereby obtaining a third educational model.

6. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images