An intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy

Abstract

The present invention discloses an intelligent question-and-answer method for educational knowledge that integrates the Socratic teaching concept, comprising the following steps: S1, establishing a Socratic teaching question-and-answer instruction to guide a model to generate answers to Socratic questions; S2, performing a Socratic teaching evaluation on the answers to obtain preference data and non-preference data; S3, training an original educational macromodel based on the preference data and non-preference data to obtain a Socratic educational macromodel; S4, conducting a Socratic heuristic teaching question-and-answer method using the Socratic educational macromodel. The present invention introduces a training process and method for enhancing the Socratic teaching ability based on the original educational macromodel, so that the trained model has heuristic teaching ability, can better adapt to educational scenarios, and enhances the applicability of the macromodel in educational scenarios. The products involved have broad market prospects and a relatively broad range of applications in the field of education.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence, large models and education, and specifically relates to an intelligent question-answering method for education knowledge that integrates Socratic teaching philosophy. Background Art

[0002] Currently, the next generation of artificial intelligence (AI) technologies is developing rapidly, and related technologies and products, represented by big models, are rapidly leading the development of the AI ​​industry. Big models across various industries are rapidly being integrated into various fields and processes, enabling intelligent upgrades across numerous sectors and contributing to a leap in social productivity. Among many industries, education is a field where AI big models are currently playing a significant role. Educational big models can be widely applied in this field. For example, in teaching scenarios for teachers, they can assist teachers in setting questions, generating lesson plans, and grading homework, effectively unleashing their productivity. In the student learning process, they can provide knowledge Q&A, analysis of incorrect questions, and the generation of personalized learning suggestions, improving student learning efficiency.

[0003] The transition from a general-purpose big model to an education-specific big model involves more than simply fine-tuning and optimizing the general big model. Instead, a learner-centered approach must be incorporated into the underlying logic of R&D, integrating it throughout the entire process of algorithm model architecture design and prototyping. This results in a big model tailored to educational scenarios. Compared to big models in other fields, educational big models require not only extensive educational knowledge and the ability to perform complex reasoning and teach knowledge problems, but also a deep understanding of education, with corresponding educational philosophies and methods. In educational settings, heuristic teaching is a common teaching method. Based on the teaching task and the objective laws of learning, and taking students' actual circumstances into consideration, it employs a variety of approaches, centered around inspiring students' thinking and mobilizing their initiative and enthusiasm for learning. A typical approach in heuristic education is the Socratic method. This method promotes problem solving through an interactive question-and-answer format, in which both teachers and students contribute their own questions to drive the conversation forward. The teacher, through questioning, continually exposes contradictions in the students' answers, guiding them to draw general conclusions. Students can also seek answers to any questions they have.

[0004] Existing large educational models are usually constructed using two methods. One is to obtain them based on a general domain large model by using domain knowledge-enhanced training and instruction fine-tuning. Typical implementations include Educhat (Yuhao D, Zhikai L, Yiyang G, Yong L, Jianghao Y, Jiaju L, Linhao Y, Zhiyan T, Yougen Z, Yilei W, Aimin Z, Ze Z, Qin C, Jie Z, Liang H, Xipeng Q, et al. EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education[J], CoRR, 2023, abs / 2308.02773) and CourseGPT-zh (Zheyan Q, Lu Y, Zitong Y, Wenbo W, Xing Z, et al. CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimization[J], CoRR, 2024, abs / 2405.04781). Another approach is to directly implement it based on a general large model with search enhancement. This involves sharding specialized educational data into a vector database, then using search enhancement to retrieve relevant context from the vector database to support the Q&A. A typical example of this approach is Tianjin University's Tianwen Campus Intelligent Question-Answering System, Twen.ai (https: / / news.tju.edu.cn / info / 1003 / 71646.htm). While both approaches have the ability to train models that solve problems, they lack the appropriate understanding of education and teaching methods, resulting in limited professionalism and applicability.

[0005] Existing large-scale educational models primarily utilize pre-training, fine-tuning, and knowledge retrieval enhancements to answer educational questions and complete specific tasks in the education field. These applications significantly improve teaching efficiency and facilitate student knowledge acquisition. However, these models neglect the fundamental principle of education: fostering students' independent learning abilities. They directly output answers when answering educational questions, which hinders the development of students' independent exploration, thinking, and problem-solving abilities and can easily lead to a lack of learning. Summary of the Invention

[0006] In response to the above-mentioned deficiencies in the existing technology, the present invention proposes an intelligent question-answering method for educational knowledge that integrates the Socratic teaching philosophy, which solves the problem that the existing large-scale education model ignores the cultivation of students' autonomous learning ability in the teaching process.

[0007] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: an intelligent question-answering method for educational knowledge that integrates Socratic teaching philosophy, comprising the following steps:

[0008] S1. Establish Socratic teaching question-answering instructions to guide the model to generate answers to Socratic questions;

[0009] S2, conduct Socratic teaching evaluation on the answers to obtain preference data and non-preference data;

[0010] S3. Train the original education model based on the preference data and non-preference data to obtain the Socrates education model;

[0011] S4. Conduct Socratic-inspired teaching and answering through the Socratic education model.

[0012] Furthermore: the S1 is specifically:

[0013] Question-answering data is collected from public datasets as original seeds. Based on the original seeds, Socrates instruction prompt templates are used to input questions into the model and guide the model to generate answers to Socrates questions.

[0014] Furthermore, S2 includes the following steps:

[0015] S21. Input the answer into the evaluation model, evaluate it based on the evaluation indicators of Socratic teaching through the evaluation model, and calculate the indicator value of each evaluation indicator;

[0016] S22. Calculate the evaluation score based on the index values ​​of all evaluation indicators, select the answer with the highest score as the preferred data, and select the answer with the lowest score as the non-preferred data.

[0017] Further: in said S21, the evaluation indicators of Socratic teaching include question-solving, relevance of the topic, helpfulness, and whether the answer is revealed;

[0018] The method for calculating the index value of questionability is as follows: determine whether the answer contains at least one question directed to the student. If so, the index value is 1; if not, the index value is 0;

[0019] The method for calculating the index value of the relevance of the topic is as follows: the index value is set according to the relevance of the answer to the ongoing conversation, and the index value ranges from 1 to 5;

[0020] The method for calculating the index value of the helpfulness of the topic is as follows: the index value is set according to the degree of usefulness of the answer in providing guidance and support to students, and the index value ranges from 1 to 5;

[0021] The method for calculating the index value of whether the answer is revealed is specifically as follows: determine whether the answer directly reveals the answer. If so, the index value is 0; if not, the index value is 1.

[0022] Further: In said S22, the evaluation score is calculated The specific expression is:

[0023]

[0024] Where, is the problematic indicator value, is the index value of the relevance of the topic, The helpfulness indicator value of the theme, is the indicator value of whether to reveal the answer.

[0025] Furthermore, in S3, the original educational model includes a role-playing model and a reference model, and S3 includes the following sub-steps:

[0026] S31, concatenating the preference data and non-preference data and inputting them into the role-playing model and the reference model, respectively calculating the scores of the corresponding role-playing model and the reference model on the preference data and non-preference data;

[0027] S32. Calculate the reward of the role-playing model and the reward of the reference model based on the score;

[0028] S33. Train the role-playing model based on the reward calculation loss function to obtain the Socrates education model.

[0029] The beneficial effect of the above further solution is: the present invention adopts the DPO preference optimization algorithm to directly use the preference data to fine-tune the original education model, and finally makes the model generation mode conform to the user preference.

[0030] Further: In said S32, the reward for playing the model and the reference model's rewards The specific expression is:

[0031]

[0032]

[0033] Where, To score the role-playing model on the preference data, To score the role model on non-preferred data, Score the reference model on the preference data, Score the reference model on the non-biased data.

[0034] Further: In said S33, the loss function The specific expression is:

[0035]

[0036] Where, To take the logarithm, is the sigmoid activation function, is the weight parameter.

[0037] The beneficial effects of the present invention are:

[0038] (1) The present invention provides an intelligent question-answering method for educational knowledge that integrates the Socratic teaching philosophy. On the basis of the original large-scale educational model, the present invention introduces the training process and method for enhancing the Socratic teaching ability, so that the trained model has heuristic teaching ability and can better adapt to educational scenarios, thereby enhancing the applicability of the large-scale model in educational scenarios. The products involved have broad market prospects and a relatively broad application range in the field of education.

[0039] (2) The method of the present invention uses a direct preference optimization algorithm to train the Socratic teaching ability of the model and conducts feedback enhancement training on the model, so that the generated Socratic education model has the Socratic heuristic teaching question-answering ability and enhances its teaching characteristics, thereby better providing intelligent teaching services for students and teachers, achieving convenience and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of an intelligent question-answering method for educational knowledge that integrates Socratic teaching philosophy.

[0041] Figure 2 The process for generating answers to Socratic questions.

[0042] Figure 3 Training process for the Socratic education model.

[0043] Figure 4 The present invention is designed to provide an overall process for an intelligent question-answering method for educational knowledge that integrates Socratic teaching philosophy. DETAILED DESCRIPTION

[0044] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0045] like Figure 1 As shown, in one embodiment of the present invention, an intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy includes the following steps:

[0046] S1. Establish Socratic teaching question-answering instructions to guide the model to generate answers to Socratic questions;

[0047] S2, conduct Socratic teaching evaluation on the answers to obtain preference data and non-preference data;

[0048] S3. Train the original education model based on the preference data and non-preference data to obtain the Socrates education model;

[0049] S4. Conduct Socratic-inspired teaching and answering through the Socratic education model.

[0050] The S1 is specifically:

[0051] Question-answering data is collected from public datasets as original seeds. Based on the original seeds, Socrates instruction prompt templates are used to input questions into the model and guide the model to generate answers to Socrates questions.

[0052] like Figure 2 As shown, in this embodiment, the present invention first randomly samples 1000 question-answer pairs from the School-Math math question-answering dataset in the BELL dataset and extracts questions from them as the original seeds. Then, a Socratic instruction prompt template is input into the model, guiding the model to generate questions for these questions and generate answers to the Socratic questions.

[0053] In this embodiment, in order to enhance the model's questioning ability, a Socratic instruction prompt template was designed based on the principles of the Socratic questioning method as follows: "You are a Socratic mentor. When responding to students, please follow the following principles: - Ask thought-provoking, open-ended questions that challenge students' preconceptions, - Promote open and respectful dialogue among students, create an environment where different perspectives are valued, and allow students to feel comfortable sharing their ideas. - Actively listen to students' answers, pay close attention to their underlying thought processes, and truly strive to understand their perspectives. - Guide students to explore topics by encouraging them to discover answers independently rather than providing direct answers, so as to improve their reasoning and analytical skills. Please respond to students' questions next."

[0054] In this embodiment, for each question, the present invention allows the model to sample and generate multiple different answers, and uses a referee model to evaluate these generated questions from different dimensions to evaluate whether the model's answers conform to the Socratic heuristic questioning method.

[0055] The S2 comprises the following sub-steps:

[0056] S21. Input the answer into the evaluation model, evaluate it based on the evaluation indicators of Socratic teaching through the evaluation model, and calculate the indicator value of each evaluation indicator;

[0057] S22. Calculate the evaluation score based on the index values ​​of all evaluation indicators, select the answer with the highest score as the preferred data, and select the answer with the lowest score as the non-preferred data.

[0058] In Socratic heuristic teaching, the core is to let the model learn to ask Socratic heuristic questions to the questions raised by students, and guide students to think for themselves and then discover the essence of the problem. Therefore, in this embodiment, it is crucial to evaluate whether the answers generated by the model have Socratic heuristic questions. The present invention extracts a comprehensive evaluation framework to evaluate whether it is suitable for subsequent training models. Specifically, in the present invention, a third-party closed-source model GPT4o is used as an evaluation model to evaluate the monads generated in the first step of the model. In terms of evaluation indicators, the present invention evaluates based on four evaluation criteria, namely: question nature (questions), relevance of the topic (on-topics), helpfulness (helpful), and whether the answer is revealed (reveal answer). The specific indicator description and indicator score are shown in the following table.

[0059] Table 1 Description of the Socratic teaching evaluation dimensions and corresponding indicator values

[0060]

[0061] In S21, the evaluation indicators of Socratic teaching include question-solving, relevance of the topic, helpfulness, and whether the answer is revealed;

[0062] The method for calculating the index value of questionability is as follows: determine whether the answer contains at least one question directed to the student. If so, the index value is 1; if not, the index value is 0;

[0063] The method for calculating the index value of the relevance of the topic is specifically as follows: setting an index value according to the relevance of the answer to the ongoing conversation, and the index value ranges from 1 to 5. In this embodiment, a value of 1 indicates that the answer is completely irrelevant to the conversation, and a value of 5 indicates that the answer is completely relevant to the topic;

[0064] The method for calculating the index value of the helpfulness of the topic is specifically as follows: an index value is set according to the degree of usefulness of the answer in providing guidance and support to students, and the index value ranges from 1 to 5. In this embodiment, a value of 1 indicates that the answer is not helpful to students at all, and a value of 5 indicates that the answer is completely useful in guiding students;

[0065] The method for calculating the index value of whether the answer is revealed is specifically as follows: determine whether the answer directly reveals the answer. If so, the index value is 0; if not, the index value is 1.

[0066] In this example, the present invention uses the GPT4o model as the judge for evaluation. During the evaluation, the present invention designed a specific prompt word model to enable the judge model to output specific indicator values ​​for each data point of the model based on four dimensions. The prompt words designed by the present invention are as follows: "You are a professional conversation evaluator. Your goal is to evaluate whether the professor engages students in deeper reflection and critical thinking. Professors are instructed to apply the Socratic method, in which they should not immediately reveal the answers to students' questions but guide them to discover the answers themselves. You will see excerpts of conversations between professors and students. You must evaluate the professor's response based on the following criteria: 1) Does the professor ask questions? Select "Yes" or "No". If there is no question mark in the answer, select "No". 2) Is the answer on the same topic as the conversation? The score range is 1 to 5, with 1 indicating that the professor's answer deviates from the original topic and 5 indicating that the answer is completely consistent. 3) Is the answer helpful to the student? The score range is 1 to 5, with 1 indicating that the professor's answer may mislead the student's thinking and 5 indicating that the answer is perfect." 4) Does it immediately reveal the answer? Select "Yes" or "No". Select "Yes" if the professor's response reveals the answer, thus not requiring any reflection from the student. Be very strict when performing assessments. Only assess the answers. Finally, provide the final assessment according to the following JSON format: {"questions":Result Criteria 1, "on-topic":Result Criteria 2, "helpful":Result Criteria 3, "reveal-answer":Result Criteria 4} Do not generate any opening or closing instructions.

[0067] In S22, the evaluation score is calculated. The specific expression is:

[0068]

[0069] Where, is the problematic indicator value, is the index value of the relevance of the topic, The helpfulness indicator value of the theme, is the indicator value of whether to reveal the answer.

[0070] In this example, after calculating the evaluation scores for the five different answers generated for each question, all answers are further sorted by score. The highest-scoring answer is selected as a good answer and marked as selected, thus meeting Socrates's preferences. The lowest-scoring answer is marked as a poor answer and rejected. After labeling the answers based on the scores, each question has two answers: preferred data and non-preferred data. Finally, this labeled preference data is used to train the model using a preference optimization algorithm, resulting in a large model with Socrates's teaching capabilities.

[0071] In S3, the original educational model includes a role-playing model and a reference model. S3 includes the following steps:

[0072] S31, concatenating the preference data and non-preference data and inputting them into the role-playing model and the reference model, respectively calculating the scores of the corresponding role-playing model and the reference model on the preference data and non-preference data;

[0073] S32. Calculate the reward of the role-playing model and the reward of the reference model based on the score;

[0074] S33. Train the role-playing model based on the reward calculation loss function to obtain the Socrates education model.

[0075] In this embodiment, if Figure 3 As shown, a preference optimization algorithm is used to train the model based on the labeled preference data generated in the second step. This ensures that the responses generated by the model align with human preferences, specifically making the model more inclined to generate preferred answers. To achieve this, the present invention employs the Determined Point-Out (DPO) algorithm. The DPO preference optimization algorithm directly fine-tunes the original educational model using the preference data, ultimately ensuring that the model's generated patterns align with user preferences.

[0076] In said S32, the reward for playing the model and the reference model's rewards The specific expression is:

[0077]

[0078]

[0079] Where, To score the role-playing model on the preference data, To score the role model on non-preferred data, Score the reference model on the preference data, Score the reference model on the non-biased data.

[0080] In this embodiment, during training, each question and the preferred answer and the non-preferred answer are spliced ​​together and input into the role-playing model and the reference model respectively. Then, the scores of the corresponding role-playing model and the reference model on the preferred data and the non-preferred data are calculated respectively. The preference and non-preference scores are calculated based on the maximum probability of relief predicted by the model. The reward of the role-playing model and the reward of the reference model are calculated according to the scores, and the KL divergence of the two rewards is calculated as the loss to train the role-playing model. After the training is completed, the role-playing model is the Socratic education model obtained through training.

[0081] In S33, the loss function The specific expression is:

[0082]

[0083] Where, To take the logarithm, is the sigmoid activation function, is the weight parameter.

[0084] like Figure 4 As shown, in this embodiment, an intelligent question-answering method for educational knowledge designed by the present invention that integrates Socratic teaching philosophy includes three steps: generating Socratic teaching question-answering instructions, constructing Socratic preference data and non-preference data, and training the Socratic education model.

[0085] For the Socratic Q&A instruction generation process, the process includes random sampling of instructions, prompt instructions based on a fixed template, and generating five different responses for each prompt.

[0086] For Socrates' method of constructing preference data and non-preference data, the evaluation dimensions of Socrates' instructions include four dimensions: problematic nature, relevance of the topic, helpfulness, and whether the answer is revealed. The scores are calculated using uniform weighting and normalization to obtain specific scores.

[0087] For the Socratic Education model training method, the model is trained using a direct preference optimization algorithm to enhance the model's Socratic question-answering capabilities.

[0088] The beneficial effects of the present invention are as follows: the present invention provides an intelligent question-answering method for knowledge in the field of education that integrates the Socratic teaching philosophy, and introduces a training process and method for enhancing the Socratic teaching ability on the basis of the original large education model, so that the trained model has heuristic teaching ability, can better adapt to the educational scene, and enhances the applicability of the large model in the educational scene. The products involved have broad market prospects and a relatively broad application range in the field of education.

[0089] The method of the present invention uses a direct preference optimization algorithm to train the Socratic teaching ability of the model, and performs feedback enhancement training on the model, so that the generated Socratic education large model has Socratic heuristic teaching question-answering ability, enhances its teaching characteristics, and thus better provides intelligent teaching services for students and teachers, achieving convenience and low cost.

[0090] In the description of the present invention, it should be understood that the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, the features defined by "first", "second", and "third" may explicitly or implicitly include one or more of such features.

Claims

1. An intelligent question-answering method for educational knowledge that integrates Socratic teaching philosophy, characterized by: The following steps are involved: S1. Establish Socratic teaching question-answering instructions to guide the model to generate answers to Socratic questions; S2. Conduct Socratic teaching evaluation on the answers to obtain preference data and non-preference data, specifically: S21. Input the answer into the evaluation model, evaluate it based on the evaluation indicators of Socratic teaching through the evaluation model, and calculate the indicator value of each evaluation indicator; S22. Calculate the evaluation score based on the index values ​​of all evaluation indicators, select the answer with the highest score as the preferred data, and select the answer with the lowest score as the non-preferred data; S3. Train the original education model based on the preference data and non-preference data to obtain the Socrates education model; S4. Conduct Socratic-inspired Q&A teaching through the Socratic education model; In S21, the evaluation indicators of Socratic teaching include question-solving, relevance of the topic, helpfulness, and whether the answer is revealed; The method for calculating the index value of questionability is as follows: determine whether the answer contains at least one question directed to the student. If so, the index value is 1; if not, the index value is 0; The method for calculating the index value of the relevance of the topic is as follows: the index value is set according to the relevance of the answer to the ongoing conversation, and the index value ranges from 1 to 5; The method for calculating the index value of the helpfulness of the topic is as follows: the index value is set according to the degree of usefulness of the answer in providing guidance and support to students, and the index value ranges from 1 to 5; The method for calculating the index value of whether the answer is revealed is specifically as follows: determine whether the answer directly reveals the answer. If so, the index value is 0; if not, the index value is 1.

2. The intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy according to claim 1 is characterized in that: The S1 is specifically: Question-answering data is collected from public datasets as original seeds. Based on the original seeds, Socrates instruction prompt templates are used to input questions into the model and guide the model to generate answers to Socrates questions.

3. The intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy according to claim 1 is characterized in that: In S22, the evaluation score is calculated. The specific expression is: Where, is the problematic indicator value, is the index value of the relevance of the topic, The helpfulness indicator value of the theme, is the indicator value of whether to reveal the answer.

4. The intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy according to claim 1 is characterized in that: In S3, the original educational model includes a role-playing model and a reference model. S3 includes the following steps: S31, concatenating the preference data and non-preference data and inputting them into the role-playing model and the reference model, respectively calculating the scores of the corresponding role-playing model and the reference model on the preference data and non-preference data; S32. Calculate the reward of the role-playing model and the reward of the reference model based on the score; S33. Train the role-playing model based on the reward calculation loss function to obtain the Socrates education model.

5. The intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy according to claim 4 is characterized in that: In said S32, the reward for playing the model and the reference model's rewards The specific expression is: Where, To score the role-playing model on the preference data, To score the role model on non-preferred data, Score the reference model on the preference data, Score the reference model on the non-biased data.

6. The intelligent question-answering method for educational knowledge integrating Socratic teaching philosophy according to claim 5 is characterized in that: In S33, the loss function The specific expression is: Where, To take the logarithm, is the sigmoid activation function, is the weight parameter.

Images