Education large model method and system integrating retrieval enhancement and efficient fine tuning

Abstract

The invention provides a retrieval enhancement and efficient fine tuning fused education large model method and system, and the system comprises a data collection and knowledge base construction module, a retrieval enhancement generation module, a model efficient fine tuning module, a model deployment and management module, and a teaching assistance and education evaluation module. According to the method applied to the system, through construction of an education knowledge base, combination of semantic retrieval and generation, efficient fine adjustment of parameters, localized deployment and model management and multi-dimensional analysis of teaching data, the painful points of one-sided evaluation of a traditional education intelligent system and high adaptive cost of a general large model are broken through; according to the invention, low-cost, high-specialty and high-stability education scene landing of an education large model is completed, personalized learning guidance is provided for students, precise teaching support is provided for teachers, and teaching and learning efficiency is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of educational technology, specifically to a method and system for a large-scale educational model that integrates retrieval enhancement and efficient fine-tuning. Background Technology

[0002] With the rapid development of artificial intelligence and large language models, the education industry is exploring their application in scenarios such as teaching assistance, automated grading, intelligent question answering, and personalized learning path planning. The introduction of large language models has endowed education systems with entirely new capabilities, from knowledge retrieval and text generation to learning process analysis, which can alleviate teachers' workload to some extent and provide students with more flexible learning support. However, the challenges that follow are equally evident: how to ensure the accuracy and professionalism of the content generated by the model, and how to achieve efficient deployment under limited computing power.

[0003] Existing AI systems for education largely rely on keyword matching or simple vector retrieval, lacking semantic understanding capabilities. This leads to a mismatch between search results and user needs when faced with complex questions. Meanwhile, while large language models possess powerful generative capabilities, they lack deep integration with educational knowledge, making them prone to issues such as biased answers, omissions of knowledge points, or inaccurate generated content.

[0004] Traditional educational AI systems, when processing complex data such as homework, exams, and classroom interactions, often rely on algorithms limited to single-dimensional analysis, making it difficult to comprehensively evaluate multiple dimensions such as knowledge mastery, problem-solving steps, and learning processes. Furthermore, some systems lack the ability to integrate with large-scale model reasoning, resulting in simplistic and shallow evaluation results that fail to provide teachers with valuable suggestions for improving their teaching.

[0005] General-purpose models are not optimized for educational tasks, and their generated results lack subject-specificity. Directly fine-tuning all parameters often requires massive amounts of data and computing power, which is difficult for educational institutions to undertake. Furthermore, existing fine-tuning methods suffer from low efficiency and insufficient versatility, lacking efficient parameter fine-tuning mechanisms for educational scenarios.

[0006] For example, the Chinese invention patent application CN202411043303.1 proposes a subject-based interactive question-answering method and teaching assistant agent based on a large model. This method improves the accuracy and relevance of the large model's answers through a subject knowledge base. Furthermore, by using implicit interactive prompts, the large language model can conduct interactive multi-turn questions for specific subjects, thus subtly guiding and gradually providing solutions to student questions. While focusing on optimizing the interaction of subject-based question-answering, it does not address the limitations of the general large model itself, such as a lack of subject-specific specialization and low fine-tuning efficiency. It still suffers from limitations such as inaccurate generated results and poor adaptability to educational scenarios. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for integrating retrieval enhancement and efficient fine-tuning into a comprehensive educational model. The goal is to construct a comprehensive educational model that integrates intelligent teaching assistance, personalized learning support, and precise educational evaluation through large-scale model technology, retrieval enhancement generation, efficient parameter fine-tuning, and localized deployment. This aims to solve the current problems of uneven distribution of educational resources, insufficient personalized learning support, and strong subjectivity in teaching evaluation.

[0008] This invention provides a method for integrating retrieval enhancement and efficient fine-tuning into a large-scale educational model, comprising the following steps:

[0009] The knowledge base construction steps include collecting multi-source educational data, performing structured processing and vectorized modeling, constructing a dynamically updatable basic education knowledge base, and storing it in a database. The enhanced question-answering retrieval step, based on the basic education knowledge base, uses a combination of vectorized retrieval and re-ranking techniques to complete the semantic retrieval of the question, and inputs it into the education big data model to obtain the question-answering results; The model's efficient fine-tuning steps employ a combination of low-rank adaptation and cue word fine-tuning, using dynamic mixed precision training to efficiently fine-tune the educational domain parameters of the large language model. The model deployment and management steps involve local deployment and distributed management of the finely tuned educational model. The educational support and evaluation process involves collecting students' dynamic learning data in real time, matching it with the basic education knowledge base, and analyzing students' learning status through multi-dimensional analysis strategies to generate quantitative learning assessment results and teaching support suggestions.

[0010] Furthermore, the multi-source educational data includes student learning behavior data, subject knowledge point data, and learning outcome data. The student learning behavior data includes login frequency and homework submission status; the subject knowledge point data includes textbooks, question banks, and test papers; and the learning outcome data includes exam scores and learning curves.

[0011] By covering multi-dimensional data, we can ensure the integrity of the knowledge base and avoid the problems of one-sided learning analysis and inaccurate question-and-answer results caused by a single data source.

[0012] Furthermore, the knowledge base construction step includes a data processing strategy, specifically: The multi-source educational data is cleaned and denoised. Redundant symbols and invalid characters in text data are removed using regular expressions. Stop words are filtered for answer texts and classroom interaction texts, and invalid records and duplicate data are deleted to ensure the semantic purity of the input data. Natural language processing techniques are used to segment textual data, identify entities, and extract relationships, transforming it into structured knowledge entries. The structured knowledge items are vectorized using an embedding model based on the Transformer architecture to obtain semantic vectors for the knowledge items. Furthermore, basic information files are established for each student, linking their learning behaviors and learning outcomes data to construct a learning profile for each student.

[0013] By employing natural language processing technology and the Transformer embedding model, this approach differs from traditional knowledge bases that only perform simple classification and lack vectorization. It enhances the structure and searchability of the knowledge base and establishes basic student information profiles, thus linking data with individual students.

[0014] Furthermore, the database includes a relational database and a vector database, wherein the vector database is used to store the semantic vectors of the knowledge entries; and the relational database is used to store structured knowledge entries and multi-source educational data.

[0015] By designing a dual database, a relational database is used for the standardized management of structured data, and a vector database is used to achieve efficient retrieval of vectorized data.

[0016] Furthermore, the enhanced question-answering steps specifically include: An embedding model using the Transformer architecture encodes the input question and generates a corresponding query vector; The query vector is compared with the semantic vector in the database to obtain candidate knowledge document fragments and recall the Top-K candidate knowledge document fragments. The semantic relevance of the Top-K candidate knowledge document fragments is scored using a re-ranking model, and the Top-N optimal knowledge fragments are selected based on the score results. The Top-N optimal knowledge fragments are concatenated with the input question and then input into the large language model to generate the final question-answering result.

[0017] By implementing a complete workflow from encoding to retrieval, and then from reordering to question and answer generation, this approach solves the problems of low accuracy and inability to adapt to complex educational questions caused by existing simple keyword searches.

[0018] Furthermore, the efficient fine-tuning steps of the model specifically include: The LoRA technique is used to insert a low-rank matrix into the attention module of the embedding model of the Transformer architecture, and only the adapter parameters corresponding to the low-rank matrix are trained and adjusted. The rank value of the LoRA technology can be dynamically adjusted according to the complexity of the educational scenario tasks, which include mathematical reasoning, essay generation, and learning analysis. Furthermore, by fine-tuning the educational model using prompt words, we can guide the large-scale educational model to aggregate semantics related to educational scenarios, resulting in a fine-tuned educational model that adapts to specific educational tasks.

[0019] By combining low-rank matrices and prompt word fine-tuning, the educational adaptability of the model is improved, ensuring that the model output conforms to the semantics of educational scenarios and solving the pain point of unprofessional output of general large models.

[0020] Furthermore, the model's efficient fine-tuning step includes an effectiveness evaluation and verification strategy, specifically: The pre-defined quantitative evaluation indicators are used to determine whether the large-scale education model is suitable for the needs of the education field. If the evaluation results meet the requirements, the model deployment and management steps are initiated; if not, the model fine-tuning steps are carried out based on the evaluation results.

[0021] Furthermore, the specific details of the model deployment management steps are as follows: The fine-tuned educational model is centrally stored, with multiple copies backed up and version snapshots managed. Containerization technology is used to isolate and deploy different versions of the large educational model, and computing resources are dynamically allocated based on access volume and task complexity. The deployment and management process also provides hierarchical control of model access permissions, log auditing, and security detection mechanisms.

[0022] By using containerized isolated deployment and multiple copy backups, it solves the problems of interference and poor stability in the existing deployment environment. It also features dynamic resource scheduling to adapt to educational scenarios and provides security mechanisms such as access control and log auditing to meet the data security and compliance requirements of the education sector.

[0023] Furthermore, the multi-dimensional analysis strategy specifically includes: The homework and exam data in the basic education knowledge base are matched with knowledge points to calculate the students’ mastery curves and error distribution for each knowledge point, and to pinpoint weak knowledge points. Sequence modeling algorithms are used to serialize students' answer steps, reconstruct the logical chain of problem-solving, and determine the reasons for students' wrong answers. Semantic analysis, sentiment analysis, and topic modeling are performed on classroom interaction data to extract student questioning frequency, answering logic, and thought processes, and to analyze students' classroom participation and knowledge acquisition difficulties.

[0024] By employing a multi-dimensional analysis strategy, we ensure the professionalism and accuracy of the analysis results, break through the limitations of traditional evaluations that only consider scores, avoid unfounded subjective evaluations, and enhance the credibility of teaching evaluations.

[0025] The present invention also provides a system applicable to the above-mentioned method for integrating retrieval enhancement and efficient fine-tuning of educational intelligent large model, the system comprising a data acquisition and knowledge base construction module, a retrieval enhancement generation module, a model efficient fine-tuning module, a model deployment and management module, and a teaching assistance and educational evaluation module; The data acquisition and knowledge base construction module is used to execute the knowledge base construction step; the retrieval enhancement generation module is used to execute the retrieval enhancement question answering step; the model efficient fine-tuning module is used to execute the model efficient fine-tuning step; the model deployment and management module is used to execute the model deployment and management step; and the teaching assistance and education evaluation module is used to execute the teaching assistance and evaluation step.

[0026] The beneficial effects of this invention are: This invention overcomes the pain points of traditional educational intelligent systems, such as one-sided evaluation and high cost of adapting general large models, by constructing an educational knowledge base, combining semantic retrieval and generation, efficiently fine-tuning parameters, localized deployment and model management, and multi-dimensional analysis of teaching data. It achieves low-cost, highly professional and stable implementation of educational large models in educational scenarios, providing students with personalized learning guidance and teachers with precise teaching support, thus greatly improving teaching and learning efficiency.

[0027] This invention achieves automatic collaboration between knowledge retrieval and content generation through in-depth processing and modeling of educational data. It combines efficient parameter fine-tuning technology to adapt to the needs of educational scenarios, and ensures the efficiency and controllability of the system through localized deployment and dynamic resource management. Furthermore, it improves the accuracy and efficiency of question answering and evaluation in educational scenarios by combining a comprehensive solution of knowledge base vector retrieval and multi-dimensional learning data analysis. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the overall steps of the method for integrating retrieval enhancement and efficient fine-tuning in the educational big data model of the present invention.

[0029] Figure 2 This is a flowchart of the fusion retrieval enhancement and efficient fine-tuning large-scale educational model method of the present invention.

[0030] Figure 3 This is a flowchart illustrating the teaching assistance and evaluation steps in the educational big data model that integrates retrieval enhancement and efficient fine-tuning of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] It should be noted that when a component is described as "fixed to" another component, it can be directly on the other component or may have a component in between. When a component is considered "connected to" another component, it can be directly connected to the other component or may have a component in between. When a component is considered "set on" another component, it can be directly set on the other component or may have a component in between. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0034] This invention provides a system for a method of integrating retrieval enhancement and efficient fine-tuning of an educational intelligent large model. The system includes a data acquisition and knowledge base construction module, a retrieval enhancement generation module, a model efficient fine-tuning module, a model deployment and management module, and a teaching assistance and educational evaluation module. Specifically, the data acquisition and knowledge base construction module is used to execute the knowledge base construction steps; the retrieval enhancement generation module is used to execute the retrieval enhancement question answering steps; the model efficient fine-tuning module is used to execute the model efficient fine-tuning steps; the model deployment and management module is used to execute the model deployment and management steps; and the teaching assistance and education evaluation module is used to execute the teaching assistance and evaluation steps.

[0035] The methods applied to the aforementioned educational intelligence big data model system that integrates retrieval enhancement and efficient fine-tuning, such as... Figure 1 As shown, it includes the following steps: The knowledge base construction steps include collecting multi-source educational data, performing structured processing and vectorized modeling, constructing a dynamically updatable basic education knowledge base, and storing it in a database. Among them, the multi-source education data includes students' learning behavior data, subject knowledge point data, and learning result data. Specifically, the students' learning behavior data reflects the dynamic data of students' daily learning participation status and learning behavior habits, including the daily or weekly login frequency and homework submission status of students in the education system; the subject knowledge point data includes teaching materials, question banks, and test papers of each subject; the learning result data reflects the quantitative data of students' knowledge mastery effect and learning progress changes, including exam scores and learning curves.

[0036] The database is a dual database composed of a relational database and a vector database. Among them, the vector database is used to store the semantic vectors of knowledge entries; the relational database is used to store multi-source education data and processed structured knowledge entries. The dual database design is to solve the problems of inconvenient management or inefficient retrieval of a single database, and the real-time synchronization trigger mechanism of the dual database is used to ensure the data consistency of the dual database.

[0037] A data processing strategy is also set in this step, specifically: Clean and denoise the collected multi-source education data. Write data processing rules through regular expressions, scan and identify all text-based data, remove meaningless redundant symbols and invalid characters in the text, and then filter and remove stop words from the answer text and classroom interaction text. Stop words include modal particles, auxiliary words, conjunctions, etc., such as "de", "le", "ne", "he", "danshi", etc.; and delete invalid records in the data, such as blank answer data, abnormal login data, question data without matching knowledge points, etc., and delete duplicate data to ensure the semantic purity of the input data; After completing data cleaning and denoising, use natural language processing technology to perform structured transformation on text-based data. Natural language processing technology refers to using computer technology to perform word segmentation, entity recognition, and relationship extraction on human natural language. Specifically, word segmentation refers to the process of splitting continuous text-based data into independent lexical units according to semantics; entity recognition refers to the process of identifying entities with specific meanings from text-based data; relationship extraction refers to the process of extracting the association relationships between different entities from text-based data; Furthermore, through natural language processing technology, the original text data without a fixed format is transformed into structured knowledge entries with a clear logical structure, clear entities and relationships, and all structured knowledge entries are stored in the relational database; After completing the generation of structured knowledge entries, use an embedding model based on the Transformer architecture to perform vectorization processing on the structured knowledge entries. Among them, the Transformer architecture is a deep learning model architecture based on the self-attention mechanism, with powerful natural language understanding and feature extraction capabilities. It is a deep learning model that can map text-based data into low-dimensional dense numerical vectors, thereby obtaining the semantic vectors of knowledge entries; The specific implementation process involves taking each structured knowledge entry stored in the relational database as input, substituting it into the embedding model for feature extraction and mapping calculation, and then using the formula... The corresponding semantic vector of the knowledge item is calculated, in the formula. For embedding model mapping functions, For knowledge entry text; Then, integrate the semantic vectors of all knowledge items to construct a semantic vector set. ,in, This is the semantic vector corresponding to the first structured knowledge entry. This is the semantic vector corresponding to the second structured knowledge item. Let m be the semantic vector corresponding to the m-th structured knowledge entry, where m is the total number of structured knowledge entries. This set is stored in the Milvus vector database, which is a database specifically used for storing, managing, and retrieving numerical vectors.

[0038] A basic education knowledge base that can be incrementally learned and dynamically updated is constructed by using vector databases and relational databases; This step also establishes basic information files for individual students, linking their learning behavior and learning outcome data to ensure that every piece of student learning data can be traced back to the corresponding individual student. Then, each student's individual learning data is compared with the group data. Through data comparison and analysis, the student's learning performance and relative position in the group are analyzed. Combined with multi-dimensional information such as students' learning behavior data, learning outcome data, and knowledge point mastery, a student learning profile is generated, providing accurate individual student data support for subsequent teaching assistance and evaluation.

[0039] The knowledge base construction process ensures the integrity, accuracy, and searchability of educational knowledge data, providing a solid foundation for the generation of enhanced question-and-answer steps.

[0040] The retrieval-enhanced question-answering process is based on a basic education knowledge base and employs vectorized retrieval and re-ranking techniques combined with a retrieval enhancement generation algorithm. It is used for semantic retrieval and intelligent question answering of student questions, specifically as follows: Similarly, the input question is encoded using the embedding model of the Transformer architecture, transforming unstructured natural language text into a structured low-dimensional dense numerical vector. This vector is the query vector, and the dimension and numerical format of the query vector are completely consistent with the semantic vector of the knowledge items stored in the basic education knowledge base, ensuring that the two can be used for subsequent similarity retrieval operations. By using vector similarity retrieval, the system quickly matches a set of semantic vectors of knowledge items that are semantically related to the question from the vector database of the basic education knowledge base. The generated query vector is then compared with the semantic vectors of all knowledge items in the vector database one by one to calculate the similarity. This quantitatively determines the degree of semantic association between the query vector and the semantic vectors of each knowledge item. The higher the similarity value, the stronger the semantic relevance between the corresponding knowledge item and the user's question. Based on the similarity calculation results, all knowledge item semantic vectors are sorted from high to low. The original structured knowledge items corresponding to the top K knowledge item semantic vectors are selected and these knowledge items are used as relevant document fragments for recall, i.e., the Top-K candidate knowledge document fragments are recalled. The formula for calculating similarity is:

[0041] in, For query vector, For the semantic vector of knowledge entries; query vector The modulus length; semantic vectors for knowledge entries The modulus length; Next, a re-ranking model is used to score the semantic relevance of the Top-K candidate knowledge document fragments. These fragments serve as input to the re-ranking model, and their corresponding scores are calculated as numerical values. Higher scores indicate a higher semantic match and better fit between the document fragment and the input question. Therefore, based on the re-ranking model's scores, the Top-K candidate document fragments are sorted from highest to lowest. The top N fragments are selected as the optimal knowledge fragments, i.e., the Top-N optimal knowledge fragments are selected. Here, N and K are both positive integers, with N being less than or equal to K. Furthermore, N and K can be dynamically adjusted based on the semantic complexity of the input question.

[0042] In one embodiment of the present invention, the re-ranking model may be the BCE-Reranker model, which is a model optimized for text semantic relevance analysis. It has a more refined text semantic analysis capability in the education field than basic vector similarity calculation, and can accurately identify the semantic matching degree and answer fit between knowledge document fragments and user questions in educational scenarios.

[0043] The Top-N optimal knowledge fragments are concatenated with the input question and then fed into the large-scale education model that has been efficiently fine-tuned using LoRA and PromptTuning dual paths to generate the final question-and-answer result.

[0044] Furthermore, this step, for multi-hop reasoning problems, also supports cross-paragraph retrieval and answer aggregation. Building upon conventional vectorized retrieval and reordering, it breaks the limitations of single knowledge items and paragraphs. Based on the semantic logic of the user's question, it performs multiple rounds of semantic retrieval in the vector database of the basic education knowledge base, retrieving all knowledge items related to each reasoning stage of the question from different knowledge paragraphs. At the same time, based on the logical connections of the question, it establishes semantic links between multiple different knowledge items, thereby generating more interpretive answers.

[0045] This step not only improves the accuracy and professionalism of the answers, but also supports multi-hop reasoning and contextual question answering for complex questions in teaching scenarios; it also effectively makes up for the knowledge blind spots of large models in educational scenarios, and provides high-quality contextual data support for the efficient fine-tuning of the model.

[0046] The efficient fine-tuning steps for the model involve combining low-rank adaptation with cue word fine-tuning to efficiently fine-tune the parameters of a large language model for the educational domain. Specifically: LoRA (Lower Rank Adaptation) technology is employed. LoRA is a high-efficiency parameter fine-tuning technique for large language models. It inserts a low-rank matrix into the attention module of the embedded model within the Transformer architecture. Only the adapter parameters corresponding to the low-rank matrix are trained and adjusted, achieving training with a small number of parameters. This allows for domain adaptation of the model without modifying the massive pre-trained parameters of the large model, significantly reducing memory usage. Furthermore, during training, the LoRA rank value can be dynamically adjusted based on the complexity of the educational task, which includes mathematical reasoning, essay generation, and learning analysis. The rank value is the rank of the low-rank matrix, and its magnitude is positively correlated with the number of adapter parameters. A larger rank value indicates more adapter parameters, resulting in stronger model fine-tuning and a slightly higher memory usage. Conversely, a smaller rank value indicates fewer adapter parameters, faster model fine-tuning, and lower memory usage. In one embodiment of the present invention, the LoRA Rank value is set to 8, and the scaling factor is set to 32, thereby reducing the number of parameters while maintaining the model's expressive power. Regarding training parameter settings, the single-device training batch size is 8, and the evaluation batch size is 8; the precision is set to bfloat16 to improve training efficiency and reduce GPU memory usage; the initial learning rate is set to 5e-5 and dynamically adjusted using a cosine annealing strategy; the gradient accumulation step is set to 4 to simulate the effect of large-batch training under small-batch conditions; and the number of iterations is set to 5 epochs. With these settings, the system can perform efficient fine-tuning under limited computing power, quickly converge, and stably optimize model performance.

[0047] After completing the deployment and training of the LoRA low-rank adaptation technique, the deployment and training of the PromptTuning prompt word fine-tuning technique are implemented. PromptTuning is an efficient fine-tuning technique for soft prompt parameters of large language models. It inserts learnable virtual vectors before the model's input sequence to realize the model. These learnable virtual vectors, i.e., soft prompt parameters, are a set of randomly initialized low-dimensional dense numerical vectors, which guide the model to generate content that conforms to the educational context without modifying most of the parameters, so as to adapt to specific educational tasks.

[0048] During the training process of LoRA and PromptTuning dual-path fine-tuning techniques, a dynamic mixed-precision training method is configured simultaneously. Dynamic mixed-precision training is a computational optimization technique for training large models. By combining single-precision and half-precision numerical computation methods, it accelerates the fine-tuning process while ensuring the numerical stability of the training process. In order to further ensure the convergence stability of the large model fine-tuning training process, the AdamW optimizer and cosine annealing learning rate scheduler are used simultaneously in the training optimization.

[0049] like Figure 2 As shown, this step also includes an effectiveness evaluation and verification strategy. Specifically, this strategy uses pre-defined quantitative evaluation indicators to determine whether the model is suitable for the needs of educational tasks. If the evaluation results meet the preset requirements, the model deployment and management step is initiated. If not, the model is optimized and fine-tuned based on the evaluation results, and fine-tuning training continues. Specifically, the pre-defined quantitative evaluation indicators can be set before model fine-tuning, based on the core tasks of the educational field. By inputting the preset educational test set into the fine-tuned educational model, the model results and the actual values ​​of each indicator are obtained. Then, the actual values ​​of each indicator are compared with the set indicator thresholds to determine whether the model is suitable for the needs of educational tasks.

[0050] This step also configures breakpoint resume training and visual monitoring functions for educational institutions, facilitating flexible adjustments to the training progress based on actual computing resources and training progress. Specifically, the breakpoint resume training function configures a timed parameter saving mechanism within the large model training framework. At preset time intervals or training steps, it automatically saves key training information such as LoRA adapter parameters, the current parameter values ​​of the learnable virtual vectors in PromptTuning, training epochs, loss function values, and optimizer status to a designated storage path. When the training process is stopped due to insufficient computing resources, equipment failure, or human error, the breakpoint resume training function can retrieve the previously saved key training information and resume fine-tuning from the interrupted point, eliminating the need to start training from scratch and effectively saving computing resources and training time. The visual monitoring function integrates a training monitoring panel into the large model training framework, enabling real-time collection and visualization of core performance indicators during the training process.

[0051] After effective evaluation and verification that the fine-tuned education model meets the requirements, the model deployment and management step begins. This involves local deployment and distributed management of the fine-tuned education model, specifically: The fine-tuned educational models are centrally stored. All educational models fine-tuned for different educational scenarios are centrally stored, and model parameters are stored using block encryption to ensure data security during transmission and storage. Multiple backups are used to ensure that model resources can be quickly restored from backup replica nodes in case of hardware failure or data corruption on the primary storage node, thus avoiding model data loss. At the same time, a version snapshot management function is configured for the educational models. After each iteration of fine-tuning and parameter update, a snapshot of the current version status of the model is automatically saved to form a complete model version traceability system. Containerization technology is used to isolate and deploy different versions of the large-scale education model, achieving environment decoupling and resource isolation. Specifically, Kubernetes is used for container orchestration. Kubernetes is an open-source container orchestration platform that enables automated deployment, scaling, and operation and maintenance management of containers. All the built large-scale education model container images are connected to the Kubernetes platform. This platform integrates resource monitoring and dynamic scheduling components, and collects the running status data of each model container in real time, including CPU utilization, GPU utilization, memory usage, network bandwidth, user access volume, and task processing complexity. Therefore, GPU / CPU computing resources can be dynamically allocated according to user access volume and task complexity, realizing on-demand allocation and efficient utilization of computing resources, and ensuring stable response in high-concurrency scenarios. Subsequently, load balancing among multiple instances is achieved by combining Nginx and gRPC. Nginx is a high-performance HTTP server and reverse proxy server, while gRPC is a high-performance remote procedure call framework. Nginx is configured as the reverse proxy server for the model service, receiving all model call requests from the educational intelligence system. gRPC is responsible for efficient data transmission between Nginx and each model container instance. A round-robin weighted load balancing strategy is configured in Nginx to ensure service stability. This step also provides hierarchical control of model access permissions, log auditing, and security detection mechanisms to prevent unauthorized access and the generation of inappropriate content; This step also configures quantization inference functionality for the inference service of the large educational model, supporting large model quantization methods such as GPTQ and AWQ, quantizing model weights into INT8 or INT4 format, thereby significantly reducing resource consumption without significantly reducing performance.

[0052] The model deployment and management steps provide an efficient and controllable operating environment for the teaching support and evaluation steps, such as... Figure 3 As shown, the teaching support and evaluation steps involve real-time collection of students' dynamic learning data. This dynamic learning data refers to all homework and exam data collected by students during their daily learning process, including answer results, answer time, error records, and correction status. Combined with pre-built structured knowledge items in the basic education knowledge base, and leveraging the reasoning and statistical analysis capabilities of the large-scale education model, a multi-dimensional analysis strategy is employed to analyze students' learning status, generating quantitative learning assessment results and teaching support suggestions. The multi-dimensional analysis strategy includes: By matching real-time collected learning data with data in the basic education knowledge base, each student's answer to a homework or exam question is associated with its corresponding subject knowledge point based on the unique identifier of the question. This establishes a one-to-one correspondence between student answers and knowledge points. Based on the matching results, if no matching result is found, it means that there is no corresponding question information in the knowledge base, and the knowledge base needs to be replenished in time. If a matching result is found, the mastery curve and error distribution of each student's knowledge point are calculated based on the matching results. Error distribution and mastery analysis are conducted to accurately identify weak knowledge points and recommend personalized learning paths and exclusive question banks for students to improve learning outcomes. Sequence modeling algorithms are used to serialize students' answer steps, reconstruct the problem-solving logic chain, and determine the reasons for students' mistakes. The sequence modeling algorithm can employ either Bi-LSTM or TransformerEncoder to reconstruct the problem-solving logic chain. Bi-LSTM, or Bidirectional Long Short-Term Memory network algorithm, can capture bidirectional contextual information of the sequence data. TransformerEncoder, based on a self-attention mechanism, can capture long-distance dependencies between steps in the sequence data. Both algorithms can achieve feature extraction and structured analysis of answer step sequence data. By parsing the student's answer step sequence through the algorithm model, the student's problem-solving logic chain can be reconstructed, thus not only determining the correctness of the answer but also analyzing the student's thinking process. Semantic analysis, sentiment analysis, and topic modeling are performed on classroom interaction data to extract student questioning frequency, answering logic, and thought processes. This analysis helps to differentiate students' classroom participation and areas of confusion in knowledge acquisition, and a comprehensive assessment is conducted by combining this with a knowledge acquisition vector.

[0053] The evaluation report generated in this step not only includes numerical indicators, but also provides visual charts and dynamic feedback, providing teachers with a basis for decision-making. It realizes intelligent teaching assistance and objective quantitative evaluation in the education scenario, forming a complete closed loop for the application of the large education model.

[0054] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for a comprehensive educational model that integrates retrieval enhancement and efficient fine-tuning, characterized in that: Includes the following steps: The knowledge base construction steps include collecting multi-source educational data, performing structured processing and vectorized modeling, constructing a dynamically updatable basic education knowledge base, and storing it in a database. The enhanced question-answering retrieval step, based on the basic education knowledge base, uses a combination of vectorized retrieval and re-ranking techniques to complete the semantic retrieval of the question, and inputs it into the education big data model to obtain the question-answering results; The model's efficient fine-tuning steps employ a combination of low-rank adaptation and cue word fine-tuning, using dynamic mixed precision training to efficiently fine-tune the educational domain parameters of the large language model. The model deployment and management steps involve local deployment and distributed management of the finely tuned educational model. The educational support and evaluation process involves collecting students' dynamic learning data in real time, matching it with the basic education knowledge base, and analyzing students' learning status through multi-dimensional analysis strategies to generate quantitative learning assessment results and teaching support suggestions.

2. The method for integrating enhanced retrieval and efficient fine-tuning of an educational large-scale model according to claim 1, characterized in that, The multi-source educational data includes student learning behavior data, subject knowledge point data, and learning outcome data. The student learning behavior data includes login frequency and homework submission status; the subject knowledge point data includes textbooks, question banks, and test papers; and the learning outcome data includes exam scores and learning curves.

3. The method for integrating enhanced retrieval and efficient fine-tuning of the educational large-scale model according to claim 2, characterized in that, The knowledge base construction steps include a data processing strategy, specifically: Natural language processing techniques are used to segment textual data, identify entities, and extract relationships, transforming it into structured knowledge entries. The structured knowledge items are vectorized using an embedding model based on the Transformer architecture to obtain semantic vectors for the knowledge items.

4. The method for integrating enhanced retrieval and efficient fine-tuning of the educational large-scale model according to claim 3, characterized in that, The enhanced question-answering steps specifically include: An embedding model using the Transformer architecture encodes the input question and generates a corresponding query vector. The query vector is compared with the semantic vector in the database to obtain candidate knowledge document fragments and the Top-K candidate knowledge document fragments are retrieved. The semantic relevance of the Top-K candidate knowledge document fragments is scored using a re-ranking model, and the Top-N optimal knowledge fragments are selected based on the score results. The Top-N optimal knowledge fragments are concatenated with the input question and then input into the large language model to generate the final question-answering result.

5. The method for integrating enhanced retrieval and efficient fine-tuning of an educational large-scale model according to claim 4, characterized in that, The efficient fine-tuning steps of the model specifically include: The LoRA technique is used to insert a low-rank matrix into the attention module of the embedded model of the Transformer architecture. Only the adapter parameters corresponding to the low-rank matrix in the model are trained and adjusted. The rank value of the LoRA technique can be dynamically adjusted according to the task complexity in the educational scenario. The large model is then guided to aggregate semantics of educational scenarios through fine-tuning of prompt words, resulting in a fine-tuned educational model.

6. The method for integrating enhanced retrieval and efficient fine-tuning of an educational large-scale model according to claim 5, characterized in that, The specific details of the model deployment and management steps are as follows: The fine-tuned educational model is centrally stored, with multiple copies backed up and version snapshots managed. Containerization technology is used to isolate and deploy different versions of the large educational model, and computing resources are dynamically allocated based on access volume and task complexity. The deployment and management process also includes configuring hierarchical control of model access permissions, log auditing, and security detection mechanisms.

7. The method for integrating enhanced retrieval and efficient fine-tuning of an educational large-scale model according to claim 1, characterized in that, The multi-dimensional analysis strategy specifically includes: The real-time learning data is matched with the data in the basic education knowledge base for knowledge points. The matching results are used to determine whether there are any matching results. If there are no matching results, the basic education knowledge base is supplemented. If there are matching results, the mastery curve of each knowledge point and the distribution of wrong questions of students are calculated based on the matching results. Sequence modeling algorithms are used to serialize students' answer steps, reconstruct the logical chain of problem-solving, and determine the reasons for students' wrong answers. Semantic analysis, sentiment analysis, and topic modeling are performed on classroom interaction data to extract student questioning frequency, answering logic, and thought processes, and to analyze students' classroom participation and knowledge acquisition difficulties.

8. The method for integrating enhanced retrieval and efficient fine-tuning of an educational large-scale model according to claim 5, characterized in that, The efficient fine-tuning step of the model includes an effectiveness evaluation and verification strategy, specifically: The pre-defined quantitative evaluation indicators are used to determine whether the large-scale education model is suitable for the needs of the education field. If the evaluation results meet the requirements, the model deployment and management steps are initiated; if not, the model fine-tuning steps are carried out based on the evaluation results.

9. The method for integrating enhanced retrieval and efficient fine-tuning of an educational large-scale model according to claim 3, characterized in that, The database includes a relational database and a vector database. The vector database is used to store the semantic vectors of the knowledge entries. The relational database is used to store the processed multi-source educational data and structured knowledge entries.

10. A system applicable to the educational intelligence large-scale model method of fusion retrieval enhancement and efficient fine-tuning as described in any one of claims 1-9, characterized in that, The system includes a data acquisition and knowledge base construction module, a retrieval enhancement generation module, a model efficient fine-tuning module, a model deployment and management module, and a teaching assistance and education evaluation module; The data acquisition and knowledge base construction module is used to execute the knowledge base construction step; the retrieval enhancement generation module is used to execute the retrieval enhancement question answering step; the model efficient fine-tuning module is used to execute the model efficient fine-tuning step; the model deployment and management module is used to execute the model deployment and management step; and the teaching assistance and education evaluation module is used to execute the teaching assistance and evaluation step.

Images