A model processing method, apparatus, device, and storage medium

Abstract

This invention discloses a model processing method, apparatus, device, and storage medium. The method includes: acquiring a teaching sample set containing N teaching sample pairs, where N is a positive integer; calling a first language model to generate guiding feedback information for the answering context of each teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; generating N feedback sample pairs based on the generated guiding feedback information and the N teaching sample pairs, and performing quality assessment processing on each feedback sample pair to obtain the quality assessment results of the N feedback sample pairs; selecting M feedback sample pairs that meet the quality requirements from the N feedback sample pairs according to the quality assessment results of each feedback sample pair, where M ∈ [1, N]; and training a second language model using the selected M feedback sample pairs; which can effectively improve the training effectiveness of the second language model in teaching scenarios.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model processing method, apparatus, device and storage medium. Background Technology

[0002] With the rapid development of language models (LM) in the field of education, intelligent teaching and interactive Q&A systems have been widely applied in basic education, online tutoring, and adaptive learning scenarios. Currently, the main approach is to inject educational knowledge and task data into the model during the post-training stage, enabling it to understand textbook content and answer questions related to subject knowledge. For example, supervised fine-tuning (SFT) can be used for model training during the post-training stage. This method primarily uses manually labeled teaching dialogue samples to supervise the model's training, allowing it to learn the direct mapping relationship between input and ideal output, thereby generating feedback answers that conform to the teacher's demonstration style on specific tasks.

[0003] Practice has shown that language models trained using current methods typically focus more on the student's input question itself and struggle to reliably incorporate the student's knowledge comprehension into their guidance and feedback. Therefore, how to effectively train language models in teaching scenarios so that the output of the trained language model is consistent with the student's understanding of the knowledge has become a current research hotspot. Summary of the Invention

[0004] This application provides a model processing method, apparatus, device, and storage medium that can effectively train a language model in a teaching scenario and enable the trained language model to output feedback answers consistent with the student's understanding of the knowledge, thereby effectively improving the training effectiveness of the language model in a teaching scenario.

[0005] On one hand, embodiments of the present invention provide a model processing method, including: Obtain a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for the question-and-answer context. The understanding state information is used to indicate the student's knowledge understanding of the question-and-answer context, where N is a positive integer. The first language model is invoked to generate guiding feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; Based on the generated guidance feedback information and N teaching sample pairs, N feedback sample pairs are generated, and each feedback sample pair is subjected to quality assessment processing to obtain the quality assessment results of the N feedback sample pairs. Based on the quality assessment results of each feedback sample pair, select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs; M∈[1,N]; The second language model is trained using M selected feedback samples; the trained second language model is used to output guiding feedback information that matches the understanding state information of any question-answering context.

[0006] On the other hand, embodiments of the present invention provide a model processing apparatus, including: The acquisition unit is used to acquire a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for the question-and-answer context. The understanding state information is used to indicate the student's knowledge understanding of the question-and-answer context, where N is a positive integer. The processing unit is used to call the first language model to generate guidance feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; The processing unit is also used to generate N feedback sample pairs based on the generated guidance feedback information and N teaching sample pairs, and to perform quality evaluation processing on each feedback sample pair to obtain the quality evaluation results of the N feedback sample pairs. The processing unit is further configured to select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs based on the quality assessment results of each feedback sample pair; M∈[1,N]; The processing unit is further configured to train the second language model using the selected M feedback samples; the trained second language model is configured to output guiding feedback information that matches the understanding state information of any question-answering context.

[0007] On the other hand, embodiments of the present invention provide a computer device including a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are interconnected. The memory is used to store a computer program that supports the computer device in executing the above-described method. The computer program includes program instructions. The processor is configured to call the program instructions to execute the above-described model processing method.

[0008] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the above-described model processing method.

[0009] On the other hand, embodiments of the present invention provide a computer program product, which includes a computer program and program instructions. When the program instructions are invoked by a processor, the processor executes the above-described model processing method.

[0010] This application embodiment can obtain a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for that context. A first language model can be invoked to generate guiding feedback information for the question-and-answer context of each teaching sample pair based on the understanding state information of that teaching sample pair. Based on the generated guiding feedback information and the N teaching sample pairs, N feedback sample pairs are generated. Therefore, this application embodiment can automatically construct feedback sample pairs based on real teaching samples, achieving automatic generation of feedback sample pairs, effectively reducing the generation cost of feedback sample pairs. Furthermore, the feedback sample pairs generated based on real teaching samples and the guiding feedback from the first language model can improve the quality of the generated feedback sample pairs.

[0011] Furthermore, this embodiment of the application can also perform quality assessment processing on each feedback sample pair to obtain the quality assessment results of N feedback sample pairs. Based on the quality assessment results of each feedback sample pair, M feedback sample pairs that meet the quality requirements are selected from the N feedback sample pairs. The selected feedback sample pairs are then used to train the second language model, so that the trained second language model can output guiding feedback information that matches the understanding state of any question-answering context. It can be seen that this embodiment of the application can use high-quality feedback sample pairs to train the second language model, which can improve the training effect of the second language model, enhance the training effectiveness of the second language model, and improve the guiding feedback capability of the second language model, so that the output guiding feedback information matches the understanding state. When the second language model is used in an educational context, it can automatically generate teaching dialogue samples (i.e., feedback sample pairs) with teaching guidance features without manual annotation. Through quantitative evaluation and screening of quality assessment, high-quality paired teaching samples (i.e., feedback sample pairs that meet quality requirements) can be obtained. This provides interpretable, controllable, and highly reliable data support for the training of reinforcement learning and educational language models (i.e., second language models), thereby improving the guidance and explanation capabilities of the second language model in teaching scenarios. Attached Figure Description

[0012] Figure 1 This is a schematic flowchart of a model processing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a model processing flow provided in an embodiment of the present invention; Figure 3 This is a schematic block diagram of a model processing device provided in an embodiment of the present invention; Figure 4 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical terms used in the embodiments of this application are described below.

[0014] I. Teaching sample set and teaching sample pair.

[0015] The teaching sample set is a set of samples used for language model training. This set contains multiple teaching sample pairs, each containing two teaching samples. Each teaching sample is a dialogue sample collected in a teaching scenario. In this scenario, the dialogue sample is generated by a student posing a question (e.g., a math problem), followed by a guiding question from the language model (e.g., the language model in the teaching scenario) or a teacher. This guides the student to think about the question. The student can then provide an answer to the guiding question. The language model then provides further guidance based on the student's response, and this process is iterated to obtain the dialogue sample in the teaching scenario.

[0016] For example, a dialogue sample in a teaching scenario could be: "Student: What is the value of 1+2+3? Model output: Think about what the value of 1+2 is. Student: It's 3. Model output: Do you know the value of 1+2+3 now?"

[0017] The dialogue samples in this teaching scenario can also be called Q&A contexts. In this embodiment, two different understanding state information are concatenated to form a Q&A context, thus generating a teaching sample pair. Therefore, the teaching sample pair in this embodiment differs only in the understanding state information. This understanding state information is: understood or not understood, correct answer or incorrect answer.

[0018] This comprehension status information can be determined from the student's responses in a teaching scenario. For example, in a teaching scenario, when a student understands the question, they might reply with "I understand" or "OK, I get it," indicating that their comprehension status is "understanding." Conversely, if the student doesn't understand, they might reply with "I'm not quite sure" or "I don't understand," indicating that their comprehension status is "not understanding." Additionally, the comprehension status information indicating whether an answer is correct or incorrect serves as an indication of whether the student's answer to the guiding question is right or wrong.

[0019] For example, a teaching sample pair could be: "Q&A context, understand" + "Q&A context, do not understand"; or, a teaching sample pair could also be: "Q&A context, answer correctly" + "Q&A context, answer incorrectly".

[0020] II. Language Model.

[0021] The language model in this application primarily refers to a Large Language Model (LLM). A Large Language Model is a model with a very large scale (e.g., a large number of parameters) that is capable of recognizing, understanding, and generating complex text content. LLMs are trained using deep learning and large amounts of training data to capture the complex structure and semantics of natural language, and are mainly applied in machine translation, text summarization, question answering systems, and dialogue generation. In this application, the Large Language Model is primarily used for question answering in an educational setting.

[0022] This application involves two different language models in an educational setting. For ease of explanation, these two different language models are referred to as the first language model and the second language model, respectively. The first language model is a large language model used to generate guided feedback information based on the question-and-answer context, while the second language model is a large language model trained using high-quality feedback sample pairs. It is evident that this application, based on the guided feedback information generation capability of the first language model, automatically constructs feedback sample pairs, selects high-quality feedback sample pairs through quality assessment, and uses these constructed high-quality feedback sample pairs to train the second language model. This improves the training efficiency of the second language model and effectively reduces training costs.

[0023] III. Model Training.

[0024] The training of the second language model in this application embodiment can adopt a feedback-based training method, including reinforcement learning from human feedback (RLHF) and related preference optimization methods, such as direct preference optimization (DPO) and Kahneman-Tversky optimization (KTO). Through preference optimization or reward modeling, the second language model can gradually learn strategic behaviors that are more in line with the teaching logic during the interaction process. This allows the second language model to shift from "giving answers" to "guiding students to think" when responding to questions, thereby achieving optimization at the teaching strategy level.

[0025] The feedback sample pairs used to train the second language model are also known as positive-negative pairs. During reinforcement learning training, feedback sample pairs that meet the quality requirements can be used to construct a preference signal, and the second language model can be trained based on this preference signal.

[0026] This application proposes a model processing scheme applicable to intelligent teaching and interactive Q&A systems in basic education scenarios. It constructs teaching sample pairs based on collected Q&A contexts and calls upon a first language model to generate guiding feedback information for that context, generating corresponding feedback sample pairs. Furthermore, by evaluating the quality of these feedback sample pairs, it selects those meeting quality requirements for training the second language model. This achieves automatic generation of feedback sample pairs in teaching scenarios, reducing the cost of constructing sample pairs for the second language model and improving its training efficiency. It addresses the scarcity and annotation difficulties of paired sample data in real teaching scenarios, automatically generating uniformly formatted training sample pairs for the second language model, thereby enhancing its training effectiveness. Through an integrated process of teaching sample pair generation, quality evaluation, and optimization, this scheme provides high-quality, interpretable, and pedagogically effective training samples for reinforcement learning of educational second language models without relying on manual annotation, significantly improving the model's strategy adaptability and interaction quality in teaching and Q&A tasks.

[0027] It should be specifically noted that the relevant data involved in the model processing of this application (such as question-answering context, understanding status information, guidance feedback information, etc.) requires the permission or consent of the target user when the above embodiments of this application are applied to specific products or technologies. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the region, adhering to the principles of legality, legitimacy, and necessity, and not involving the acquisition of data types prohibited or restricted by laws and regulations. In some optional embodiments, the relevant data involved in the embodiments of this application is obtained after separate authorization from the target user. Additionally, when obtaining separate authorization from the target user, the purpose of the relevant data is explained to the target user. Figure 1This is a schematic flowchart illustrating a model processing method provided in an embodiment of the present invention. The method can be executed by a computer device. For example, the computer device can be a terminal device or a server, or a system composed of a terminal device and a server. For example, the terminal device can include, but is not limited to: smartphones (such as smartphones running the Android system, or smartphones running an Internet operating system), tablets, portable personal computers, mobile Internet devices (MIDs), in-vehicle devices, head-mounted devices, smart home devices, and smart robots, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Figure 1 As shown, the method may include S101-S104: S101, Obtain the teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same answer context and carries different understanding state information for the answer context. The understanding state information is used to indicate the student's knowledge understanding of the answer context. N is a positive integer.

[0028] The understanding state information concatenated for the same question-answering context is any one of the paired state information. For example, the paired state information can be "understand-do not understand" and "correct answer-incorrect answer", etc. If the understanding state information contained in one teaching sample in a teaching sample pair is "understand", then the understanding state information contained in the other teaching sample in the teaching sample pair is "do not understand".

[0029] When generating the question-answering context, the large language model can be invoked to answer the student's question information based on the teaching behavior prompt, and the dialogue context (or dialogue sample) between the student and the large language model can be used as the question-answering context. The large language model can be a first language model, a second language model, or other language models. This application embodiment does not limit the specific language model.

[0030] The teaching behavior prompt is the system prompt of the large language model. It can constrain the basic answering criteria of the large language model for the student's questions during the dialogue process, such as what principles should be followed in what situations, and the output format of the response text, etc.

[0031] During the conversation between the student and the large language model, the student's understanding of the model's output response text (or question information) can be determined based on the student's responses. This understanding state information is then concatenated with the question-and-answer context to obtain a teaching sample. For example, if the student's response includes phrases like "understood" or "ok, I understand," the determined understanding state is "understood"; if the response includes phrases like "I'm not sure" or "I don't understand," the determined understanding state is "not understood."

[0032] In one embodiment, the understanding state information of "correct or incorrect answer" is used to indicate the correctness of the response text generated by the large language model, or to indicate the correctness of the student object's response based on the response text of the large language model.

[0033] For example, a teaching sample pair may include sample 1 and sample 2. For example, sample 1 may be “teaching behavior prompt + answer context + understanding status information (such as “I understand”)”, and sample 2 may be “teaching behavior prompt + answer context + understanding status information (such as “I am not sure”)”.

[0034] S102, the first language model is invoked to generate guiding feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set.

[0035] The generated guidance feedback information includes: the output text from which the first language model, acting as a teacher, guides or explains the student's question-answering context under the guidance of teaching behavior prompts.

[0036] For example, any one of the N teaching sample pairs is a target teaching sample pair. The target teaching sample pair includes a first teaching sample and a second teaching sample. The first teaching sample carries positive comprehension state information (such as understanding or answering correctly), and the second teaching sample carries negative comprehension state information (such as not understanding or answering incorrectly). The guidance feedback information is generated by the first language model based on the teaching behavior cue words. The guidance feedback information generated for the target teaching sample pair includes first guidance feedback information and second guidance feedback information. The first guidance feedback information is generated based on the positive comprehension state information, and the second guidance feedback information is generated based on the negative comprehension state information. The generated N feedback sample pairs include the target feedback sample pairs corresponding to the target teaching sample pairs. The target feedback sample pairs include: a first feedback sample obtained by splicing together the teaching behavior cue words, the answer context, the positive comprehension state information, and the first guidance feedback information, and a second feedback sample obtained by splicing together the teaching behavior cue words, the answer context, the negative comprehension state information, and the second guidance feedback information.

[0037] For example, the first feedback sample may be composed of the first teaching sample and the first guidance feedback information, and the second feedback sample may be composed of the second teaching sample and the second guidance feedback information.

[0038] S103. Based on the generated guidance feedback information and N teaching sample pairs, generate N feedback sample pairs, and perform quality assessment processing on each feedback sample pair to obtain the quality assessment results of N feedback sample pairs.

[0039] For example, a feedback sample pair may include feedback sample 1 and feedback sample 2. Feedback sample 1 may be: "teaching behavior prompt + answer context + understanding status information (such as "I understand") + model response 1 (i.e., the first guiding feedback information)" and feedback sample 2 may be: "teaching behavior prompt + answer context + understanding status information (such as "I am not quite clear") + model response 2 (i.e., the second guiding feedback information)".

[0040] In this embodiment, during the feedback sample pair generation stage, teaching behavior prompts are embedded in the first language model, and the comprehension state information of different students is concatenated after the input question-and-answer context. This guides the language model to generate feedback content (i.e., guiding feedback information) that matches the comprehension state, thereby achieving automatic generation of feedback sample pairs. The feedback sample pairs generated during the generation stage can be input into a teaching behavior evaluation framework, where each feedback sample is quantitatively analyzed and scored based on dimensions such as the accuracy of perceived feedback, the coherence of explanation progression, and the hierarchy of thought-provoking questions. Based on the multi-dimensional scoring results, the system further calculates the behavioral score differences and strategy consistency indicators among the feedback sample pairs, and selects samples that perform well in terms of teaching logic, behavioral patterns, and strategy stability according to set thresholds or ranking rules. The selected high-quality feedback sample pairs are used for the optimization training of the second language model in the reinforcement learning stage to continuously improve the second language model's behavior generation ability and strategy consistency in teaching scenarios, thus forming a "generation-evaluation-selection-optimization" model training mechanism.

[0041] For example, when performing quality assessment processing on each feedback sample pair to obtain quality assessment results for N feedback sample pairs, a quality assessment rule can be obtained. The quality assessment rule is used to instruct the quality assessment processing of each feedback sample pair from multiple dimensions. The multiple dimensions include at least a first dimension and a second dimension. The first dimension is used to generate a basic quality score and includes at least an organization dimension. If the feedback sample pair belongs to a specific type (i.e., the answer is correct / incorrect), a perception dimension is further included. Furthermore, the quality assessment processing can be performed on each feedback sample pair under the multiple dimensions indicated by the quality assessment rule to obtain the quality assessment results for N feedback sample pairs.

[0042] The second dimension can be an elicitation dimension. In this application embodiment, the quality assessment processing of each feedback sample is performed under multiple dimensions, including the following two cases: 1) performing quality assessment processing on feedback sample pairs under the organizational dimension + elicitation dimension; 2) performing quality assessment processing on feedback sample pairs under the perception dimension + organizational dimension + elicitation dimension. In this application embodiment, the process of performing quality assessment processing on feedback sample pairs under the three dimensions of perception dimension + organizational dimension + elicitation dimension is mainly illustrated as an example. The requirements for quality assessment processing under these three dimensions are shown in Table 1. Perception PA: Affirm When a student is in a positive state, should the teacher provide clear affirmation and positive feedback? P-Redirect (Corrective Feedback) When a student is in a negative state, do you promptly identify their mistakes or confusion and provide targeted corrections or re-guidance? Organization O-Advance (Advanced Explanation) When students are in a positive state, should we proceed with the explanation, introduce new concepts or more challenging questions to deepen their learning? O-Reconfigure (Refactoring Explanation) When students are in a negative state, should the teaching structure be adjusted, and methods such as simplification, analogy, or scaffolding be used to help them understand? Elicitation E-Strategic (Advanced Questioning) When students are in a positive state, should we ask them higher-order, open-ended questions to promote their reasoning, transfer of knowledge, and critical thinking? E-Heuristic (Intuitive Inspiration) When students are in a negative state, should we use heuristic or intuitive questions to reduce their cognitive load and guide them to form intuitive understanding? By evaluating the quality of each feedback sample pair across multiple dimensions, the effectiveness of the quality evaluation can be improved, and the credibility of the evaluation results can be increased. This ensures that feedback sample pairs that meet the quality requirements are selected from N feedback sample pairs based on the evaluation results, thereby guaranteeing the reinforcement learning effect of the second language model.

[0043] In one embodiment, the scores for the perception and organization dimensions are either 0 or 1, where 0 indicates non-compliance with the dimension's requirements and 1 indicates compliance. The heuristic dimension categorizes the questions presented hierarchically, and the quality assessment result can include scores of 1, 2, and 3. A higher heuristic dimension score indicates higher feedback sample quality when the student's state is positive, and a lower score indicates higher feedback sample quality when the student's state is negative. For the same feedback sample pair, the difference between the heuristic dimension scores of the positive and negative feedback samples can be calculated, and the quality score of the feedback sample pair under the heuristic dimension can be determined based on this difference.

[0044] When performing quality assessment on N feedback sample pairs across multiple dimensions, if a feedback sample in a pair carries a positive understanding state (such as understanding or answering correctly), it is best if the two PAs in the corresponding positive dimension, Affirm and O-Advance, are both 1, and the higher the score for E-Strategic, the better. If the feedback sample carries a negative understanding state, it is best if the two Ps in the corresponding negative dimension, Redirect and O-Reconfigure, are both 1, and the lower the score for E-Heuristic, the better.

[0045] In one embodiment, N feedback sample pairs include target feedback sample pairs, and each target feedback sample pair includes a first feedback sample and a second feedback sample. Therefore, when performing quality assessment processing on the target feedback sample pairs under multiple dimensions indicated by the quality assessment rules to obtain the quality assessment results, quality assessment processing can be performed on the first feedback sample (denoted as y+) and the second feedback sample (denoted as y-) respectively under the first dimension indicated by the quality assessment rules. This yields the first-dimensional quality score of the first feedback sample (including a positive score S_pos(y+) and a negative score S_neg(y+)) and the first-dimensional quality score of the second feedback sample (including a positive score S_pos(y+)). The first feedback sample and the second feedback sample are evaluated under the second dimension indicated by the quality assessment rule, respectively, to obtain the second-dimensional quality score of the first feedback sample (denoted as E_Strategic(y+)) and the second-dimensional quality score of the second feedback sample (denoted as E_Heuristic(y-)). Based on the first-dimensional and second-dimensional quality scores of the first and second feedback samples, the target quality score of the target feedback sample pair is generated, and the target quality score is used as the quality assessment result of the target feedback sample pair.

[0046] The first feedback sample is obtained by splicing positive understanding state information, and the second feedback sample is obtained by splicing negative understanding state information. The first dimension quality score of the first feedback sample includes: the score of the first feedback sample under the positive scoring standard (denoted as S_pos(y+)) and the score under the negative scoring standard (denoted as S_neg(y+)). The first dimension quality score of the second feedback sample includes: the score of the second feedback sample under the positive scoring standard (denoted as S_pos(y-)) and the score under the negative scoring standard (denoted as S_neg(y-)).

[0047] When generating the target quality score for the target feedback sample pair based on the first and second dimension quality scores of the first feedback sample and the first and second dimension quality scores of the second feedback sample, an expected score (i.e., S_pos(y+) + S_neg(y-)) can be generated based on the score of the first feedback sample under the positive scoring criterion and the score of the second feedback sample under the negative scoring criterion. An error score (S_neg(y+) + S_pos(y-)) can also be generated based on the score of the first feedback sample under the negative scoring criterion and the score of the second feedback sample under the positive scoring criterion. The difference between the expected score and the error score is calculated to obtain the basic quality score. The difference between the second dimension quality score of the first feedback sample and the second dimension quality score of the second feedback sample is also calculated to determine the importance of the basic quality score. Based on the importance and the basic quality score, the target quality score for the target feedback sample pair is generated.

[0048] Specifically, the design principle of quality scoring is as follows: positive samples should be highly rated on the positive scoring dimension and low on the negative scoring dimension; negative samples are the opposite. Therefore, for positive samples constructed under the condition that the student's understanding state is positive (positive understanding state), the model's feedback guidance should be to affirm the student, advance the explanation, and ask higher-order questions (corresponding to the scoring criteria of the positive dimension), rather than to correct the student, reconstruct the explanation, or ask intuitively inspiring questions (corresponding to the scoring criteria of the negative dimension).

[0049] During the calculation, firstly, the baseline quality score (denoted as A_delta) of the target feedback sample pair can be calculated, which satisfies Equation 1: A_delta=[ S_pos(y+)+S_neg(y-)]-[S_neg(y+)+S_pos(y-)] Formula 1 Where S_pos represents the base score under the positive scoring criteria, which is the average score under the two dimensions of P-Affirm and O-Advance; S_neg represents the base score under the negative scoring criteria, which is the average score under the two dimensions of P-Redirect and O-Reconfigure.

[0050] The basic quality score of the target feedback sample pair is calculated according to Equation 1. S_pos(y+) + S_neg(y-) accurately describes the degree to which positive sample images are positive and negative sample images are negative in the target feedback sample pair, and S_neg(y+) + S_pos(y-) accurately describes the degree to which positive sample images are negative and negative sample images are positive. After subtracting the two, the higher the score, the clearer the state distinction is when constructing the target feedback sample pair.

[0051] Secondly, a heuristic questioning dimension can be introduced as a weighting term to calculate the difference in question levels (i.e., importance, denoted as E_delta), where this importance satisfies Equation 2: E_delta=E_Strategic(y+)-E_Heuristic(y-) Equation 2 Here, E_Strategic(y+) represents the level of higher-order questioning in positive samples, and E_Heuristic(y-) represents the level of heuristic questioning in negative samples. Ideally, the question level in positive samples should be higher, and the question level in negative samples should be more basic. Therefore, the larger E_delta is, the more the questioning strategy matches the differences in students' states.

[0052] The target quality score (denoted as C) of the target feedback sample pair generated based on the importance score in Equation 2 and the basic quality score in Equation 1 can then satisfy Equation 3: C=A_delta×[1+beta×max(E_delta,0)] Formula 3 Here, `beta` is a weighting coefficient used to control the influence of the heuristic question dimension on the final screening score; `max(E_delta,0)` means that `E_delta` is taken when it is greater than 0, and 0 when it is less than or equal to 0. Therefore, a high-quality pairwise sample should satisfy the following: positive samples score high on the positive dimension and low on the negative dimension; negative samples score high on the negative dimension and low on the positive dimension; at the same time, the heuristic question level of the positive sample should be higher than that of the negative sample.

[0053] The quality scoring method according to the embodiments of this application can improve the accuracy of the quality assessment results of feedback sample pairs.

[0054] S104. Based on the quality assessment results of each feedback sample pair, select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs; M∈[1,N].

[0055] In one embodiment, the higher the quality score C corresponding to the quality assessment result, the more valuable the feedback sample pair is in teaching and the greater its ability to differentiate states. For example, the feedback sample pairs can be screened based on a score threshold (which can be set to 1).

[0056] Specifically, based on the quality assessment results of each feedback sample pair, the N feedback sample pairs can be sorted, and based on the sorting order, M feedback sample pairs whose sorting order meets the order threshold can be selected from the N feedback sample pairs; or, the quality assessment results of each feedback sample pair can be compared with the quality assessment threshold (such as 1), and based on the comparison results, M feedback sample pairs greater than or equal to the quality assessment threshold can be selected from the N feedback sample pairs.

[0057] S105, the selected M feedback samples are used to train the second language model; the trained second language model is used to output guiding feedback information that matches the understanding state information of any question-answering context.

[0058] In one embodiment, the system can acquire the question text output by the student, call a trained second language model to provide guided feedback to the question text, and output the question feedback information. The question text and its feedback information constitute the target answering context. The system then acquires the student's understanding state information within the answering context; the trained second language model, based on this understanding state information, generates target guided feedback information that matches the understanding state information; and finally, the target guided feedback information is output to the student. This allows the trained second language model to output guided feedback information that matches the student's understanding state, thereby improving the effectiveness of teaching guidance.

[0059] The following is combined Figure 2 This paper provides an overall description of the execution flow of the model processing method in the embodiments of this application. The key technologies of this flow include: the implementation structure of the automated data generation / screening process, the setting of evaluation dimensions and scoring mechanism, and the combination of large language models and quality assessment.

[0060] (1) Implementation of automated data generation / filtering process.

[0061] This application can automatically generate and filter teaching feedback sample pairs based on the input question-and-answer context. The specific process includes question-and-answer context input, student status construction, teacher feedback generation, teaching behavior evaluation, screening score calculation, and high-value sample output.

[0062] The system constructs student responses based on the question-and-answer context, indicating whether the student answered correctly, incorrectly, understood, or still did not understand. It then uses a large language model to generate corresponding teacher feedback. Subsequently, a teaching behavior evaluation mechanism scores the generated content, and the sample quality is determined based on the screening score. A higher screening score indicates better teaching behavior quality and state differentiation, making the sample more valuable for training.

[0063] (2) The setting of evaluation dimensions and scoring mechanism.

[0064] This application sets up assessment dimensions based on student status and teaching behavior. Student status includes four categories: correct answer, incorrect answer, understood, and still do not understand. Correct answer and understood are considered positive statuses, while incorrect answer and still do not understand are considered negative statuses.

[0065] In terms of teaching practices, this invention summarizes them into three aspects: perception, organization, and inspiration. Perception is used to determine whether the teacher's feedback correctly identifies the student's state; organization is used to determine whether the teacher's feedback adopts appropriate teaching strategies; and inspiration is used to determine whether the level of questioning in the teacher's feedback is consistent with the student's current state.

[0066] Based on the above three aspects, this application further sets up six scoring dimensions: P-Affirm is used to evaluate whether the student is affirmed in a positive state; P-Redirect is used to evaluate whether feedback is corrected in a negative state; O-Advance is used to evaluate whether the explanation is advanced in a positive state; O-Reconfigure is used to evaluate whether the explanation is restructured in a negative state; E-Strategic and E-Heuristic are used to evaluate the problem level in different states.

[0067] Through the above scoring mechanism, the system can determine whether the generated teacher feedback matches the student's status, and further use it to calculate the screening score, thereby selecting teaching feedback samples with high teaching value and status differentiation.

[0068] (3) Processing mechanism that combines large language model with evaluation.

[0069] This application combines the generative capabilities of a large language model with a teaching behavior evaluation mechanism. First, the system calls upon the large language model to generate corresponding teacher feedback content based on the question-and-answer context and student status. Then, through an evaluation model or evaluation prompts, the system automatically scores the generated content in dimensions such as perception, organization, and inspiration.

[0070] The scoring results are not only used to evaluate model performance, but also further used for sample selection. The system calculates selection scores based on the scores of each dimension to determine whether the generated samples are suitable for students' conditions, whether they have reasonable teaching strategies, and whether they have training value. This forms a closed-loop processing mechanism of "feedback generation - behavior assessment - score calculation - sample selection".

[0071] like Figure 2As shown, N feedback sample pairs can be generated in the generation stage (1). For example, the question-and-answer dialogue context can be obtained from the upstream data source, which is a series of "student question - teacher answer" round combinations that maintain the time sequence, such as (student question, teacher answer) (student question, teacher answer), etc. After obtaining the context (i.e. the question-and-answer context), the student status reply can be concatenated at the end of the context to explicitly indicate the current learning situation according to the needs of the target task; the student status is limited to four atomic states: correct answer, incorrect answer, student understands, and student does not understand, and is organized in pairs as a contrast scenario (e.g., "correct answer - incorrect answer", "student understands - student does not understand"). When the original context already contains a clear state, the system can also directly retain the real state to generate the corresponding contrast state, thereby forming a structurally symmetrical state pair input in the same question context. After completing the state concatenation, the context input pair used for generation can be obtained. Each pair is only contrasted with each other in the end student state, and the rest of the context remains consistent to ensure that the subsequent feedback only produces comparable differences in the "teaching strategy selection" dimension.

[0072] (2) Teaching behavior assessment can be conducted.

[0073] After the candidate teaching feedback is generated, the system enters the teaching behavior evaluation stage. This stage uses a teaching behavior evaluation framework to identify behaviors and score the quality of the teaching feedback samples generated by the model, thereby determining whether the teacher's response reflects reasonable teaching strategies and guidance logic.

[0074] The students' comprehension status is categorized into four basic types: correct answer, incorrect answer, student understands, and student does not understand. "Correct – understand" is classified as a positive state, and "incorrect – do not understand" is classified as a negative state. When these student states intersect with three types of teaching behaviors (Perception, Orchestration, and Elicitation), six teaching behavior scoring dimensions are formed. By automatically detecting and scoring the behavioral patterns in each dimension, a teaching behavior score (i.e., quality score) is obtained for each feedback sample pair.

[0075] This application embodiment can obtain a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for that context. A first language model can be invoked to generate guiding feedback information for the question-and-answer context of each teaching sample pair based on the understanding state information of that teaching sample pair. Based on the generated guiding feedback information and the N teaching sample pairs, N feedback sample pairs are generated. Therefore, this application embodiment can automatically construct feedback sample pairs based on real teaching samples, achieving automatic generation of feedback sample pairs, effectively reducing the generation cost of feedback sample pairs. Furthermore, the feedback sample pairs generated based on real teaching samples and the guiding feedback from the first language model can improve the quality of the generated feedback sample pairs.

[0076] Furthermore, this embodiment of the application can also perform quality assessment processing on each feedback sample pair to obtain the quality assessment results of N feedback sample pairs. Based on the quality assessment results of each feedback sample pair, M feedback sample pairs that meet the quality requirements are selected from the N feedback sample pairs. The selected feedback sample pairs are then used to train the second language model, so that the trained second language model can output guiding feedback information that matches the understanding state of any question-answering context. It can be seen that this embodiment of the application can use high-quality feedback sample pairs to train the second language model, which can improve the training effect of the second language model, enhance the training effectiveness of the second language model, and improve the guiding feedback capability of the second language model, so that the output guiding feedback information matches the understanding state. When the second language model is used in an educational context, it can automatically generate teaching dialogue samples (i.e., feedback sample pairs) with teaching guidance features without manual annotation. Through quantitative evaluation and screening of quality assessment, high-quality paired teaching samples (i.e., feedback sample pairs that meet quality requirements) can be obtained. This provides interpretable, controllable, and highly reliable data support for the training of reinforcement learning and educational language models (i.e., second language models), thereby improving the guidance and explanation capabilities of the second language model in teaching scenarios.

[0077] By introducing a teaching behavior assessment and screening mechanism, efficient quality control of samples generated by large language models is achieved, resulting in the following significant technical effects: (1) The teaching behavior assessment framework realizes the quantification and interpretability of teaching behavior characteristics. The system can generate multi-dimensional behavior scores for each candidate teaching sample, thereby transforming abstract teaching characteristics such as "reasonable feedback", "coherent explanation" and "level of inspiration" into measurable indicators, providing an objective basis for subsequent screening and model optimization.

[0078] (2) By introducing teaching behavior prompts and student state modeling during the generation stage, the model is guided to perform targeted deep generation, making the initial candidate samples significantly better than those generated without constraints in terms of teaching logic, thus ensuring that the screening mechanism is based on high-quality data. Table 2 shows the improvement effect of the generated responses on the same assessment dataset before and after adding prompts.

[0079] PA (Acc.) 0.8546 → 0.9071 +0.0525 PR (Err.) 0.5483 → 0.9149 +0.3666 OA (Acc.) 0.9445 → 0.9554 +0.0109 OR (Err.) 0.7036 → 0.8857 +0.1821 ESA (Acc / Err) 0.2659 → 0.4362 +0.1703 (3) The automated screening process of this application significantly reduces the cost of manual annotation and improves screening efficiency. In the experiment, the system evaluated and screened approximately 50,000 candidate teaching samples in parallel (using the GPT-4o-mini model with a maximum concurrency of 16), taking approximately 1 hour in total. The results show that, while ensuring evaluation consistency and interpretability (experiments show that the evaluation consistency of dimensions is 89.0%–97.0%), this method can achieve a hundredfold increase in efficiency compared to manual annotation, greatly improving the economy and scalability of teaching behavior sample construction and model optimization.

[0080] (4) Furthermore, by filtering candidate teaching feedback samples through screening scores, this invention can improve the quality of subsequent model training data. Specifically, the filtered samples have clearer student state differentiation and more stable teaching behavior characteristics, which can reduce the entry of low-quality samples or policy conflict samples into the training set, thereby improving the stability and effectiveness of model training.

[0081] Figure 3 This is a schematic diagram of the structure of a model processing device provided in an embodiment of the present invention. Figure 3 As shown, the model processing device may include an acquisition unit 301 and a processing unit 302.

[0082] The acquisition unit 301 is used to acquire a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for the question-and-answer context. The understanding state information is used to indicate the student's knowledge understanding of the question-and-answer context, where N is a positive integer. Processing unit 302 is used to call the first language model to generate guidance feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; The processing unit 302 is further configured to generate N feedback sample pairs based on the generated guidance feedback information and N teaching sample pairs, and to perform quality assessment processing on each feedback sample pair to obtain the quality assessment results of the N feedback sample pairs. The processing unit 302 is further configured to select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs based on the quality assessment results of each feedback sample pair; M∈[1,N]; The processing unit 302 is further configured to train a second language model using the selected M feedback samples; the trained second language model is configured to output guiding feedback information that matches the understanding state information of any question-answering context.

[0083] In one embodiment, any one of the N teaching sample pairs is a target teaching sample pair, which includes a first teaching sample and a second teaching sample. The first teaching sample carries positive understanding state information, and the second teaching sample carries negative understanding state information. The guidance feedback information is generated by the first language model based on teaching behavior prompts. The guidance feedback information generated for the target teaching sample includes first guidance feedback information and second guidance feedback information. The first guidance feedback information is generated based on the positive understanding state information, and the second guidance feedback information is generated based on the negative understanding state information. The generated N feedback sample pairs include the target feedback sample pairs corresponding to the target teaching sample pairs. The target feedback sample pairs include: a first feedback sample obtained by splicing the teaching behavior prompt words, the question-answering context, the positive understanding state information and the first guidance feedback information, and a second feedback sample obtained by splicing the teaching behavior prompt words, the question-answering context, the negative understanding state information and the second guidance feedback information.

[0084] In one embodiment, the processing unit 302 is specifically used for: Obtain quality assessment rules; the quality assessment rules are used to instruct the quality assessment processing of each feedback sample pair from multiple dimensions, the multiple dimensions including at least a first dimension and a second dimension; the first dimension includes at least an organizational dimension, and further includes a perceptual dimension if the feedback sample pair belongs to a preset type, the second dimension includes a heuristic dimension; Under the multiple dimensions indicated by the quality assessment rules, each feedback sample pair is subjected to quality assessment processing to obtain the quality assessment results of N feedback sample pairs.

[0085] In one embodiment, the N feedback sample pairs include target feedback sample pairs, and the target feedback sample pairs include a first feedback sample and a second feedback sample; the processing unit 302 is specifically used for: Under the first dimension indicated by the quality assessment rule, the first feedback sample and the second feedback sample are respectively subjected to quality assessment processing to obtain the first dimension quality score of the first feedback sample and the first dimension quality score of the second feedback sample. Under the second dimension indicated by the quality assessment rule, the first feedback sample and the second feedback sample are respectively subjected to quality assessment processing to obtain the second-dimensional quality score of the first feedback sample and the second-dimensional quality score of the second feedback sample. Based on the first-dimensional quality score and the second-dimensional quality score of the first feedback sample, and the first-dimensional quality score and the second-dimensional quality score of the second feedback sample, a target quality score is generated for the target feedback sample pair, and the target quality score is used as the quality assessment result of the target feedback sample pair.

[0086] In one embodiment, the first feedback sample is obtained by concatenating positive understanding state information, and the second feedback sample is obtained by concatenating negative understanding state information. The first-dimensional quality score of the first feedback sample includes: the score of the first feedback sample under the positive scoring standard and the score under the negative scoring standard. The first-dimensional quality score of the second feedback sample includes: the score of the second feedback sample under the positive scoring standard and the score under the negative scoring standard. The processing unit 302 is specifically used for: Based on the scores of the first feedback sample under the positive scoring criteria and the scores of the second feedback sample under the negative scoring criteria, an expected score is generated, and based on the scores of the first feedback sample under the negative scoring criteria and the scores of the second feedback sample under the positive scoring criteria, an error score is generated. The difference between the expected score and the error score is calculated to obtain the basic quality score. The difference between the second-dimensional quality score of the first feedback sample and the second-dimensional quality score of the second feedback sample is calculated to determine the importance of the basic quality score. Based on the importance and the basic quality score, a target quality score is generated for the target feedback sample pair.

[0087] In one embodiment, the processing unit 302 is specifically used for: Based on the quality assessment results of each feedback sample pair, the N feedback sample pairs are sorted, and based on the sorting order, M feedback sample pairs whose sorting order satisfies the order threshold are selected from the N feedback sample pairs; or; The quality assessment result of each feedback sample pair is compared with the quality assessment threshold, and based on the comparison result, M feedback sample pairs that are greater than or equal to the quality assessment threshold are selected from the N feedback sample pairs.

[0088] In one embodiment, the acquisition unit 301 is further configured to acquire the question text output by the student object, call the trained second language model to perform guided feedback processing on the question text, and output the question feedback information of the question text; the question text and the question feedback information of the question text constitute the target Q&A context; The acquisition unit 301 is also used to acquire the student object's understanding state information in relation to the question-answering context; The processing unit 302 is further configured to call the trained second language model to generate target guidance feedback information that matches the understanding state information for the target question answering context based on the understanding state information; The processing unit 302 is also used to output the target guidance feedback information to the student object.

[0089] This application embodiment can obtain a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for that context. A first language model can be invoked to generate guiding feedback information for the question-and-answer context of each teaching sample pair based on the understanding state information of that teaching sample pair. Based on the generated guiding feedback information and the N teaching sample pairs, N feedback sample pairs are generated. Therefore, this application embodiment can automatically construct feedback sample pairs based on real teaching samples, achieving automatic generation of feedback sample pairs, effectively reducing the generation cost of feedback sample pairs. Furthermore, the feedback sample pairs generated based on real teaching samples and the guiding feedback from the first language model can improve the quality of the generated feedback sample pairs.

[0090] Furthermore, this embodiment of the application can also perform quality assessment processing on each feedback sample pair to obtain the quality assessment results of N feedback sample pairs. Based on the quality assessment results of each feedback sample pair, M feedback sample pairs that meet the quality requirements are selected from the N feedback sample pairs. The selected feedback sample pairs are then used to train the second language model, so that the trained second language model can output guiding feedback information that matches the understanding state of any question-and-answer context. It can be seen that this embodiment of the application can use high-quality feedback sample pairs to train the second language model, which can improve the training effect of the second language model, enhance the training effectiveness of the second language model, and improve the guiding feedback capability of the second language model, so that the output guiding feedback information matches the understanding state. When the second language model is used in an educational context, it can automatically generate teaching dialogue samples (i.e., feedback sample pairs) with teaching guidance features without manual annotation. Through quantitative evaluation and screening of quality assessment, high-quality paired teaching samples (i.e., feedback sample pairs that meet quality requirements) can be obtained. This provides interpretable, controllable, and highly reliable data support for the training of reinforcement learning and educational language models (i.e., second language models), thereby improving the guidance and explanation capabilities of the second language model in teaching scenarios.

[0091] According to the embodiments of this application, Figure 3 The modules in the model processing apparatus shown can be individually or entirely combined into one or more units, or one or more of these units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division; in practical applications, the function of one module can be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the model processing apparatus may also include other units; in practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0092] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0093] According to one embodiment of this application, a computer program capable of executing the steps involved in the corresponding methods shown in the various embodiments of this application can be run on a general-purpose computer device (which may include processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM)) to construct a system as described in the embodiments of this application. Figure 3 The model processing apparatus shown. The computer program described above can be recorded on a computer-readable recording medium, and can be loaded into the computer device described above through the computer-readable recording medium and run therein.

[0094] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 4 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. In some embodiments, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 4 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0095] exist Figure 4 In the computer device 1000 shown, the network interface 1004 provides network communication functions; while the user interface 1003 is mainly used to provide an input interface for the user. In one embodiment, the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve: Obtain a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same question-and-answer context and carries different understanding state information for the question-and-answer context. The understanding state information is used to indicate the student's knowledge understanding of the question-and-answer context, where N is a positive integer. The first language model is invoked to generate guiding feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; Based on the generated guidance feedback information and N teaching sample pairs, N feedback sample pairs are generated, and each feedback sample pair is subjected to quality assessment processing to obtain the quality assessment results of the N feedback sample pairs. Based on the quality assessment results of each feedback sample pair, select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs; M∈[1,N]; The second language model is trained using M selected feedback samples; the trained second language model is used to output guiding feedback information that matches the understanding state information of any question-answering context.

[0096] In one embodiment, any one of the N teaching sample pairs is a target teaching sample pair, which includes a first teaching sample and a second teaching sample. The first teaching sample carries positive understanding state information, and the second teaching sample carries negative understanding state information. The guidance feedback information is generated by the first language model based on teaching behavior prompts. The guidance feedback information generated for the target teaching sample includes first guidance feedback information and second guidance feedback information. The first guidance feedback information is generated based on the positive understanding state information, and the second guidance feedback information is generated based on the negative understanding state information. The generated N feedback sample pairs include the target feedback sample pairs corresponding to the target teaching sample pairs. The target feedback sample pairs include: a first feedback sample obtained by splicing the teaching behavior prompt words, the question-answering context, the positive understanding state information and the first guidance feedback information, and a second feedback sample obtained by splicing the teaching behavior prompt words, the question-answering context, the negative understanding state information and the second guidance feedback information.

[0097] In one embodiment, the processor 1001 can be used to invoke a device control application stored in the memory 1005, and also to implement: Obtain quality assessment rules; the quality assessment rules are used to instruct the quality assessment processing of each feedback sample pair from multiple dimensions, the multiple dimensions including at least a first dimension and a second dimension; the first dimension includes at least an organizational dimension, and further includes a perceptual dimension if the feedback sample pair belongs to a preset type, the second dimension includes a heuristic dimension; Under the multiple dimensions indicated by the quality assessment rules, each feedback sample pair is subjected to quality assessment processing to obtain the quality assessment results of N feedback sample pairs.

[0098] In one embodiment, the N feedback sample pairs include target feedback sample pairs, and the target feedback sample pairs include a first feedback sample and a second feedback sample; the processor 1001 can be used to call the device control application stored in the memory 1005, and is also used to implement: Under the first dimension indicated by the quality assessment rule, the first feedback sample and the second feedback sample are respectively subjected to quality assessment processing to obtain the first dimension quality score of the first feedback sample and the first dimension quality score of the second feedback sample. Under the second dimension indicated by the quality assessment rule, the first feedback sample and the second feedback sample are respectively subjected to quality assessment processing to obtain the second-dimensional quality score of the first feedback sample and the second-dimensional quality score of the second feedback sample. Based on the first-dimensional quality score and the second-dimensional quality score of the first feedback sample, and the first-dimensional quality score and the second-dimensional quality score of the second feedback sample, a target quality score is generated for the target feedback sample pair, and the target quality score is used as the quality assessment result of the target feedback sample pair.

[0099] In one embodiment, the first feedback sample is obtained by concatenating positive understanding state information, and the second feedback sample is obtained by concatenating negative understanding state information. The first-dimensional quality score of the first feedback sample includes: the score of the first feedback sample under the positive scoring standard and the score under the negative scoring standard. The first-dimensional quality score of the second feedback sample includes: the score of the second feedback sample under the positive scoring standard and the score under the negative scoring standard. The processor 1001 can be used to call the device control application stored in the memory 1005, and is also used to implement: Based on the scores of the first feedback sample under the positive scoring criteria and the scores of the second feedback sample under the negative scoring criteria, an expected score is generated, and based on the scores of the first feedback sample under the negative scoring criteria and the scores of the second feedback sample under the positive scoring criteria, an error score is generated. The difference between the expected score and the error score is calculated to obtain the basic quality score. The difference between the second-dimensional quality score of the first feedback sample and the second-dimensional quality score of the second feedback sample is calculated to determine the importance of the basic quality score. Based on the importance and the basic quality score, a target quality score is generated for the target feedback sample pair.

[0100] In one embodiment, the processor 1001 can be used to invoke a device control application stored in the memory 1005, and also to implement: Based on the quality assessment results of each feedback sample pair, the N feedback sample pairs are sorted, and based on the sorting order, M feedback sample pairs whose sorting order satisfies the order threshold are selected from the N feedback sample pairs; or; The quality assessment result of each feedback sample pair is compared with the quality assessment threshold, and based on the comparison result, M feedback sample pairs that are greater than or equal to the quality assessment threshold are selected from the N feedback sample pairs.

[0101] In one embodiment, the processor 1001 can be used to invoke a device control application stored in the memory 1005, and also to implement: The system obtains the question text output by the student object, calls the trained second language model to perform guided feedback processing on the question text, and outputs the question feedback information of the question text; the question text and the question feedback information of the question text constitute the target answering context; Obtain the student's understanding state information in relation to the question-answering context; The trained second language model is invoked to generate target guidance feedback information that matches the understanding state information for the target question-answering context; The target-guided feedback information is output to the student object.

[0102] It should be understood that the computer device 1000 described in the embodiments of this application can execute the model processing methods described in the embodiments of this application, and can also execute the methods described above. Figure 3 The description of the model processing apparatus in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0103] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program. When a processor executes this computer program, it can perform the model processing methods described in the various embodiments of this application; therefore, they will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0104] As an example, the aforementioned computer program can be deployed and executed on a single computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network can form a blockchain network.

[0105] The aforementioned computer-readable storage medium can be an internal storage unit of the computer device, such as a hard drive or memory. It can also be an external storage device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD) card, or flash card. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. This computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. It can also be used to temporarily store data that has been output or will be output.

[0106] This application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the model processing methods described in the embodiments of this application; therefore, these will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0107] The terms "target," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0108] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A model processing method, characterized in that, include: Obtain a teaching sample set, which contains N teaching sample pairs. Each teaching sample in each pair contains the same question-and-answer context and carries different understanding state information for the question-and-answer context. The understanding status information is used to indicate the student's understanding of the question-answering context, where N is a positive integer; The first language model is invoked to generate guiding feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; Based on the generated guidance feedback information and N teaching sample pairs, N feedback sample pairs are generated, and each feedback sample pair is subjected to quality assessment processing to obtain the quality assessment results of the N feedback sample pairs. Based on the quality assessment results of each feedback sample pair, select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs; M∈[1,N]; The second language model is trained using M selected feedback samples; the trained second language model is used to output guiding feedback information that matches the understanding state information of any question-answering context.

2. The method as described in claim 1, characterized in that, Any one of the N teaching sample pairs is a target teaching sample pair. The target teaching sample pair includes a first teaching sample and a second teaching sample. The first teaching sample carries positive understanding state information, and the second teaching sample carries negative understanding state information. The guidance feedback information is generated by the first language model based on teaching behavior prompts. The guidance feedback information generated for the target teaching sample includes first guidance feedback information and second guidance feedback information. The first guidance feedback information is generated based on the positive understanding state information, and the second guidance feedback information is generated based on the negative understanding state information. The generated N feedback sample pairs include the target feedback sample pairs corresponding to the target teaching sample pairs. The target feedback sample pairs include: a first feedback sample obtained by splicing the teaching behavior prompt words, the question-answering context, the positive understanding state information and the first guidance feedback information, and a second feedback sample obtained by splicing the teaching behavior prompt words, the question-answering context, the negative understanding state information and the second guidance feedback information.

3. The method as described in claim 1, characterized in that, The process of performing quality assessment on each feedback sample pair to obtain the quality assessment results of the N feedback sample pairs includes: Obtain quality assessment rules; the quality assessment rules are used to instruct the quality assessment processing of each feedback sample pair from multiple dimensions, the multiple dimensions including at least a first dimension and a second dimension, the first dimension including at least an organizational dimension and a perceptual dimension for a specific type, and the second dimension including a heuristic dimension; Under the multiple dimensions indicated by the quality assessment rules, each feedback sample pair is subjected to quality assessment processing to obtain the quality assessment results of N feedback sample pairs.

4. The method as described in claim 3, characterized in that, The N feedback sample pairs include target feedback sample pairs, and the target feedback sample pairs include a first feedback sample and a second feedback sample; Under the multiple dimensions indicated by the quality assessment rules, the target feedback sample pairs are subjected to quality assessment processing to obtain the quality assessment results of the target feedback sample pairs, including: Under the first dimension indicated by the quality assessment rule, the first feedback sample and the second feedback sample are respectively subjected to quality assessment processing to obtain the first dimension quality score of the first feedback sample and the first dimension quality score of the second feedback sample. Under the second dimension indicated by the quality assessment rule, the first feedback sample and the second feedback sample are respectively subjected to quality assessment processing to obtain the second-dimensional quality score of the first feedback sample and the second-dimensional quality score of the second feedback sample. Based on the first-dimensional quality score and the second-dimensional quality score of the first feedback sample, and the first-dimensional quality score and the second-dimensional quality score of the second feedback sample, a target quality score is generated for the target feedback sample pair, and the target quality score is used as the quality assessment result of the target feedback sample pair.

5. The method as described in claim 4, characterized in that, The first feedback sample is obtained by splicing together positive understanding state information, and the second feedback sample is obtained by splicing together negative understanding state information. The first dimension quality score of the first feedback sample includes: the score of the first feedback sample under the positive scoring standard and the score under the negative scoring standard. The first dimension quality score of the second feedback sample includes: the score of the second feedback sample under the positive scoring standard and the score under the negative scoring standard. The step of generating a target quality score for the target feedback sample pair based on the first-dimensional and second-dimensional quality scores of the first feedback sample, and the first-dimensional and second-dimensional quality scores of the second feedback sample, includes: Based on the scores of the first feedback sample under the positive scoring criteria and the scores of the second feedback sample under the negative scoring criteria, an expected score is generated, and based on the scores of the first feedback sample under the negative scoring criteria and the scores of the second feedback sample under the positive scoring criteria, an error score is generated. The difference between the expected score and the error score is calculated to obtain the basic quality score. The difference between the second-dimensional quality score of the first feedback sample and the second-dimensional quality score of the second feedback sample is calculated to determine the importance of the basic quality score. Based on the importance and the basic quality score, a target quality score is generated for the target feedback sample pair.

6. The method as described in claim 1, characterized in that, The step of selecting M feedback sample pairs that meet the quality requirements from the N feedback sample pairs based on the quality assessment results of each feedback sample pair includes: Based on the quality assessment results of each feedback sample pair, the N feedback sample pairs are sorted, and based on the sorting order, M feedback sample pairs whose sorting order satisfies the order threshold are selected from the N feedback sample pairs; or; The quality assessment result of each feedback sample pair is compared with the quality assessment threshold, and based on the comparison result, M feedback sample pairs that are greater than or equal to the quality assessment threshold are selected from the N feedback sample pairs.

7. The method as described in claim 1, characterized in that, The method further includes: The system obtains the question text output by the student object, calls the trained second language model to perform guided feedback processing on the question text, and outputs the question feedback information of the question text; the question text and the question feedback information of the question text constitute the target answering context; Obtain the student's understanding state information in relation to the question-answering context; The trained second language model is invoked to generate target guidance feedback information that matches the understanding state information for the target question-answering context; The target-guided feedback information is output to the student object.

8. A model processing device, characterized in that, include: The acquisition unit is used to acquire a teaching sample set, which contains N teaching sample pairs. Each teaching sample pair contains the same answer context and carries different understanding state information for the answer context. The understanding status information is used to indicate the student's understanding of the question-answering context, where N is a positive integer; The processing unit is used to call the first language model to generate guidance feedback information for the answering context in the corresponding teaching sample pair based on the understanding state information of each teaching sample pair in the teaching sample set; The processing unit is also used to generate N feedback sample pairs based on the generated guidance feedback information and N teaching sample pairs, and to perform quality evaluation processing on each feedback sample pair to obtain the quality evaluation results of the N feedback sample pairs. The processing unit is further configured to select M feedback sample pairs that meet the quality requirements from the N feedback sample pairs based on the quality assessment results of each feedback sample pair. M∈[1,N]; The processing unit is further configured to train the second language model using the selected M feedback samples; the trained second language model is configured to output guiding feedback information that matches the understanding state information of any question-answering context.

9. A computer device, characterized in that, The device includes a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images