A large language model training method and device based on knowledge distillation

Abstract

The application discloses a large language model training method and device based on knowledge distillation, which comprises the following steps: training a large model by using a target field corpus, fine-tuning the large model by using a data set corresponding to a target task in the target field, and obtaining a student model; the student model answers the target task, and if the answer is wrong, self-iteration is performed; if the iteration number exceeds a threshold value and still fails, a teacher model provides an answer, adds a high-quality answer corpus pool, and corrects the wrong answer of the student model, the error task of the student model, the wrong answer, and the corrected answer of the teacher model form a correction corpus; the student model is fine-tuned by using the high-quality answer corpus pool; a scoring model is trained by using the correction corpus; the student model is tested by using the scoring model, if the test score reaches a target, the training is completed; otherwise, the corpus collection and learning are re-performed for the substandard task. The application enhances the problem solving ability, high efficiency and high accuracy of the student model.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a method and apparatus for training large language models based on knowledge distillation. Background Technology

[0002] Large language models, pre-trained on extensive datasets, have demonstrated powerful capabilities in natural language understanding and generation tasks. By analyzing and learning from massive amounts of text data, these models can grasp the deep structure and meaning of language, effectively handling various tasks such as translation, chatbots, and text summarization. While pre-training across numerous domains provides a general foundation for understanding, for specific domain tasks, such as HVAC, the models require further targeted training on downstream tasks to accurately adapt to specific needs.

[0003] In the field of knowledge distillation, the performance gap between a powerful, general-purpose, high-parameter, large-language model and a low-parameter model can be narrowed by transferring knowledge from the latter. Knowledge distillation is a process of transferring knowledge from a large model (the teacher model) to a small model (the student model). In this way, the student model can achieve higher efficiency and lower resource consumption without sacrificing too much performance. This process not only simplifies computational requirements but also promotes a more inclusive and equitable AI landscape.

[0004] For example, in the HVAC field, large language models are used in various downstream tasks such as system control, load forecasting, fault diagnosis, and operation and maintenance, which places higher demands on the model's professional knowledge and real-time response capabilities. Existing large language models face challenges including a lack of domain-specific terminology and context, as well as the high energy consumption and operational resource requirements associated with large models. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a method and apparatus for training large language models based on knowledge distillation.

[0006] The objective of this invention is achieved through the following technical solution: a method for training a large language model based on knowledge distillation, comprising the following steps:

[0007] (1) Use the target domain corpus to train the large language model for masked language modeling task, so that it learns the context and meaning of the target domain terms and phrases; then use the dataset corresponding to the target task in the target domain to fine-tune the large language model so that it adapts to the question and answer form of the target task, and obtain the student model.

[0008] (2) The target task is decomposed into several sub-tasks to form a task pool for the target task;

[0009] (3) The student model attempts to answer the tasks in the task pool. If the answer is incorrect, it will iterate itself. If the number of self-iterations exceeds the first preset threshold and still fails, the task will be handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus. At the same time, the student model's incorrect answer will be handed over to the teacher model for correction, providing the corrected answer. During this process, the student model's error reporting task, incorrect answer and the teacher model's corrected answer are recorded and correspond one by one to form the correction corpus.

[0010] (4) Using the high-quality response corpus, the student model is fine-tuned with target data;

[0011] (5) Train the scoring model using the corrected corpus;

[0012] (6) Test the fine-tuned student model using the trained scoring model. If the test score reaches the target score, the training is complete; otherwise, return to step (3) and collect and learn the corpus again for the unsatisfactory task.

[0013] Furthermore, the target field includes the heating, ventilation, and air conditioning (HVAC) field; the target tasks include system control, load forecasting, fault diagnosis, and operation and maintenance tasks in the HVAC field.

[0014] Furthermore, it also includes a feedback model to verify the correctness of the student model's answers described in step (3).

[0015] Furthermore, if the test fails, the feedback model will report an error message. The error message and the student model's incorrect answer will be added to the prompt message in the next round of dialogue for self-iterative attempts.

[0016] Furthermore, if the answer is successfully obtained within the second preset threshold number of iterations, the answer at this time will be added to the high-quality answer corpus.

[0017] The present invention also provides a large language model training device based on knowledge distillation, comprising:

[0018] The student model building module is used to train a large language model for masked language modeling tasks using target domain corpus, enabling it to learn the context and meaning of the target domain terms and phrases; then, the large language model is fine-tuned using the dataset corresponding to the target task in the target domain to adapt it to the question and answer format of the target task, thus obtaining the student model.

[0019] The task pool construction module is used to decompose the target task into several sub-tasks to form a task pool for the target task.

[0020] The high-quality answer corpus pool and the correction corpus acquisition module are used by the student model to attempt to answer tasks in the task pool. If the answer is incorrect, it will iterate itself. If the number of self-iterations exceeds the first preset threshold and still fails, the task will be handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus pool. At the same time, the incorrect answers of the student model will be handed over to the teacher model for correction, providing corrected answers. During this process, the student model's error-reporting tasks, incorrect answers, and teacher model's answers are recorded and matched one by one to form the correction corpus.

[0021] The student model fine-tuning module is used to fine-tune the student model with target data using the high-quality response corpus.

[0022] The scoring model training module is used to train the scoring model using the corrected corpus.

[0023] The student model testing module is used to test the fine-tuned student model using the trained scoring model. If the test score reaches the target score, the training is complete; otherwise, the corpus is collected and the learning is repeated for the tasks that do not meet the target.

[0024] Furthermore, the target field includes the heating, ventilation, and air conditioning (HVAC) field; the target tasks include system control, load forecasting, fault diagnosis, and operation and maintenance tasks in the HVAC field.

[0025] Furthermore, it also includes a feedback model for verifying the correctness of the student model answers in the high-quality answer corpus and the correction corpus acquisition module.

[0026] Furthermore, if the test fails, the feedback model will report an error message. The error message and the student model's incorrect answer will be added to the prompt message in the next round of dialogue for self-iterative attempts.

[0027] Furthermore, if the answer is successfully obtained within the second preset threshold number of iterations, the answer at this time will be added to the high-quality answer corpus.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] First, by introducing a large language model, the intelligence level and efficiency of various specific tasks in the target domain are significantly improved. Users do not need in-depth professional knowledge to complete various tasks in the target domain through natural language, greatly simplifying the operation process and lowering the professional threshold for users. This approach improves the user experience and reduces reliance on technical personnel.

[0030] Secondly, this invention utilizes knowledge alignment fine-tuning and task adjustment fine-tuning methods to supplement and strengthen the professional terminology and expressions of the target domain, ensuring that the model can accurately adapt to specific downstream tasks. By combining the powerful capabilities of the teacher model and the problem-exploration capabilities of the student model, as well as the real-time evaluation of the feedback model, this invention achieves a dynamic iterative process. In this process, the teacher model makes further adjustments and iterative optimizations based on the results provided by the feedback model. Simultaneously, various problems encountered by the student model during its self-iteration process are collected and analyzed, and the teacher model provides correct solutions to these problems. This interaction not only deepens the student model's understanding of the target domain problems but also greatly enhances the student model's processing capabilities on the target task by using these problems and their best solutions as new training data, enabling it to even surpass the performance of the teacher model. The application of this method not only enhances the model's professional knowledge and problem-solving capabilities in the target domain but also ensures high efficiency and accuracy in solving practical problems through continuous learning and iteration, further enhancing its capabilities and application value in specific tasks.

[0031] Then, this invention addresses the problems arising from the large number of model parameters, such as limited computing resources, offline deployment requirements, and data security considerations. This method optimizes the use of computing resources, reduces operating costs, ensures data security, and improves environmental sustainability. It supports the application of the model in resource-constrained and offline deployment environments, enhancing practicality and flexibility.

[0032] Finally, by employing knowledge distillation, this invention significantly narrows the performance gap between large-parameter and small-parameter models, enabling the latter to achieve high efficiency and performance with lower computational resources. This advancement simplifies computational requirements, promotes environmental sustainability for AI operations, and makes small-parameter models more suitable for addressing specific domain tasks. Furthermore, this invention fosters a more inclusive and equitable AI ecosystem, encouraging broader participation in advancing and diversifying AI technologies. This democratization of technology drives the development of more powerful, versatile, and readily available large language model solutions for specific scenarios, thereby fostering innovation and growth across various industries and research fields. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1A flowchart illustrating a multi-step HVAC functional large language model training method based on knowledge distillation, provided for an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of a knowledge distillation process provided in an embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of target task decomposition provided in Embodiment 1 of the present invention. Detailed Implementation

[0037] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0038] This invention provides a method for training a large language model based on knowledge distillation, such as... Figure 1 As shown, it includes the following steps:

[0039] (1) Use the target domain corpus to train the large language model for masked language modeling task, so that it learns the context and meaning of the target domain terms and phrases; then use the dataset corresponding to the target task in the target domain to fine-tune the large language model so that it adapts to the question and answer form of the target task, and obtain the student model.

[0040] In one embodiment, taking the HVAC (Heating, Ventilation, and Air Conditioning) field as an example, the model is trained using an HVAC-specific corpus for a masked language modeling task. This allows the model to learn the context and meaning of domain-specific terms and phrases, adjusting the model to adapt to the text requirements of general scenarios in the HVAC field. Next, a fine-tuning dataset corresponding to specific types of downstream tasks is selected to fine-tune the model, adapting it to the question and answer format of the target task. These specific types of downstream tasks include system control, load forecasting, fault diagnosis, and operation and maintenance tasks within the HVAC field.

[0041] The fine-tuning datasets corresponding to the specific types of downstream tasks include: the system control task dataset, which contains control parameters and operation guidelines for various HVAC systems, as well as system response data under different control strategies; the load forecasting task fine-tuning dataset, which contains a corpus related to Python programming; the fault diagnosis task fine-tuning dataset, which contains possible fault modes and cases for various HVAC systems; and the operation and maintenance task fine-tuning dataset, which includes maintenance manuals and plans, historical operation and maintenance logs and reports for different types of HVAC systems.

[0042] In the fine-tuning dataset, the data format is "Guidance-Input-Output". This data format matches the supervised learning fine-tuning method, enabling the model to adapt to the question and answer format of the target task.

[0043] It should be noted that the model can be configured according to the characteristics of each objective task and its own conditions. For example, if a general model can be used for system control tasks, the ChatGLM3-6B model, which is good at natural language interaction and reasoning, can be selected. If a programming model can be used for load forecasting tasks, a model that is good at programming, such as the CodeGeex2-6B model, can be selected.

[0044] In some embodiments, those skilled in the art can also perform knowledge alignment fine-tuning and task adjustment fine-tuning using other fields as examples to obtain student models, which will not be elaborated here.

[0045] (2) The target task is decomposed into several sub-tasks to form a task pool for the target task;

[0046] In one embodiment, taking the HVAC field as an example, the task pool for the target task is set for specific tasks in the HVAC field, such as detailed decomposition of tasks like system control, load forecasting, fault diagnosis, and operation and maintenance. By breaking down the target task into sub-tasks and connecting the sub-tasks in a process flow, complex target tasks can be completed. For example, for a system control task, it can be broken down into control equipment and variable extraction, control strategy formulation, strategy execution and adjustment, etc.; for a load forecasting task, it can be broken down into data preprocessing, feature engineering, model training, model evaluation, result visualization, model interpretation, etc.

[0047] In some embodiments, those skilled in the art may also take other fields as examples to decompose the target tasks in that field to form a task pool of target tasks, which will not be elaborated here.

[0048] (3) The student model attempts to answer the tasks in the task pool. If the answer is incorrect, it will iterate itself. If the number of self-iterations exceeds the first preset threshold and still fails, the task will be handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus. At the same time, the student model's incorrect answer will be handed over to the teacher model for correction, providing the corrected answer. During this process, the student model's error reporting task, incorrect answer and the teacher model's corrected answer are recorded and correspond one by one to form the correction corpus.

[0049] In one embodiment, such as Figure 2As shown, the accuracy of the student model's answer is verified by the feedback model. If it fails the verification, its answer and the error message from the feedback will be added to the prompt information in the next round of dialogue for self-iteration. If the self-iteration count is within the second preset threshold and the answer is successfully answered, its answer will be added to the high-quality answer corpus as a high-quality answer. If the self-iteration count exceeds the first preset threshold and the answer is still unsuccessful, the task will be handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus. At the same time, the student model's incorrect answers will be handed over to the teacher model for correction, providing corrected answers. During this process, the student model's error task, incorrect answers, and teacher model's answers are recorded and correspondingly formed into a correction corpus.

[0050] It should be noted that the knowledge distillation process described above will result in tasks with lower difficulty being solved by the student model through self-iteration, with a smaller corpus for those tasks. For more difficult tasks, the corpus will be larger, which is beneficial for data augmentation of key improvement tasks.

[0051] It should be noted that the format of the high-quality response corpus is "Guidance-Input-Output", which matches the supervised learning fine-tuning method; the format of the correction corpus is "Guidance-Input-Output1-Output2", where output1 corresponds to the incorrect answers of the student model and output2 corresponds to the high-quality answers of the teacher model.

[0052] It should be noted that the number and specific functions of the teacher models are not limited; multiple teacher models can be used to be responsible for different target tasks.

[0053] (4) Using the high-quality answer corpus, the student model is fine-tuned with target data to adapt the student model to each sub-task in the task pool, which can significantly improve the performance of the student model.

[0054] It should be noted that the target data fine-tuning stage is compatible with both full parameter fine-tuning and efficient parameter fine-tuning methods, allowing users to choose according to their individual computing resources. Efficient parameter fine-tuning methods, including QLoRA and BitFit, effectively reduce computing resource consumption and improve inference speed while maintaining fine-tuning effectiveness, thus providing users with a fine-tuning strategy that can maintain high performance even under resource constraints.

[0055] (5) The scoring model is trained using the corrected corpus to evaluate and improve the task processing ability of the fine-tuned student model;

[0056] It should be noted that the training of the scoring model using the corrected corpus can involve steps such as data preprocessing, model selection, model training, and model evaluation. The specific steps depend on the model itself and the training method. For example, a pre-trained vector model such as Word2Vec can be used, and a scoring model such as the transformer model can be selected for model training. Alternatively, the reinforcement learning PPO algorithm can be used to train an agent for scoring.

[0057] (6) Test the fine-tuned student model using the trained scoring model. If the test score reaches the target score, the training is complete; otherwise, return to step (3) and collect and learn the corpus again for the unsatisfactory task.

[0058] It should be noted that if the test score does not reach the target score, the process will return to step (3) to collect and learn corpus separately for the unsatisfactory task. This step will result in the student model being able to solve the task with lower difficulty by iterating on its own, and the corresponding corpus pool will have a smaller number of corpus; while for the more difficult task, the number of corpus will be larger, which is conducive to completing the data augmentation for the key improvement task.

[0059] Furthermore, the trained student model can be deployed offline or online in environments with limited computing resources. When deployed online, the student model can collect information and feedback from the target application scenario for online learning, further improving model performance.

[0060] In one embodiment, a self-learning mechanism is also included to continuously train the student model based on a historical experience database and usage log data after online deployment, thereby continuously improving the system's performance and accuracy. The historical experience database originates from previous training datasets, metadata from earlier versions, etc.

[0061] Example 1:

[0062] Take the load forecasting task of building heating, ventilation and air conditioning systems as an example.

[0063] This embodiment uses operational data collected from a public building in 2016 to verify the method proposed in this invention. The sampling interval for the operational data is 1 hour. The operational data includes the following available variables: month, day of the month, hour of the day, day of the week, outdoor air temperature, outdoor relative humidity, building cooling load, and the building's historical cooling load from the previous 1 to 24 hours. The operational data is stored in a file named "Historical_operational_data.csv".

[0064] S1: Knowledge Alignment Fine-tuning. The model was trained using professional field textbooks (such as HVAC design manuals and building environmental science textbooks), papers on load forecasting, and Python coding textbooks for masked language modeling tasks. The trained model was the CodeGeeX2-6B model.

[0065] S2: Task Fine-tuning. This involves fine-tuning a large language model using open-source Python code. The dataset contains problem descriptions and Python code. It is taken from sahil2801 / code_instructions_120k and includes an Alpaca-style hint column. Its format is as follows:

[0066] {

[0067] “Instruction”:

[0068] “Input”:

[0069] "Output":

[0070] }

[0071] Its content includes dialogues related to code at different difficulty levels in the Python language. To further reduce the demand for experimental equipment for training the model, the implementation example uses Freeze tuning, which has the advantages of parameter efficiency and suitability for extracting features at specific levels. Freeze tuning only fine-tunes the parameters of the last few fully connected layers of the model, which can retain most of the knowledge of the pre-trained model, while adapting to the specific requirements of the task through fine-tuning.

[0072] S3: Task Decomposition. The load forecasting task is decomposed into a task pool, consisting of six sub-tasks: data preprocessing, feature engineering, model training, model evaluation, result visualization, and model interpretation. Each sub-task can be further divided into multiple task requirements, as shown in Table 1. A local Python compilation and runtime environment is used as the feedback model to provide feedback on the correctness of the code, the accuracy of the results, and any errors.

[0073] Table 1 Load Forecasting Task Pool

[0074]

[0075]

[0076] This embodiment designs a set of efficient prompt words to give commands to a large language model, which improves the accuracy of command parsing and the quality of response.

[0077] Before each specific prompt for a subtask, add the following: "You are a programming expert. Please answer the following questions and provide the corresponding code, or modify / improve the code based on the errors. You do not need to provide additional textual explanations, only provide the code or make modifications. Because your answers will be copied and run as Python code, any extra text that is not commented out will produce errors. Regular code in the answer section does not need comments, but any text in the answer section needs to be separated by # as code comments. Uncommented text is not allowed:." This prompt provides the large language model with self-awareness, improving the accuracy of command parsing and the quality of responses, enabling its generated answers to run directly as Python code after formatting.

[0078] Following the above prompts, further prompts are provided for the six sub-tasks: data preprocessing, feature engineering, model training, model evaluation, result visualization, and model interpretation. The file paths, data variables, expected outputs, and task requirements for each step are automatically embedded in the prompts. For example, the prompt for the data preprocessing sub-task is:

[0079] "I have a file named '{file path}{data variables}'. This file contains some missing data. Please write a Python program to process the missing data, normalize all variables, remove time-related variables, and then split the cleaned and normalized data into training and test sets. Note that feature selection should not be performed in this task. The execution order of the task is as follows: 1. Missing data processing: Complete the missing data in the office building operation data. 2. Data normalization: Normalize all variables in the dataset, excluding time-related variables. 3. Data splitting: Split the processed data into training and test sets and save them as `Training_set.csv` and `Testing_set.csv`. Please provide the Python code to complete all these tasks in sequence."

[0080] The overall process is as follows Figure 3 As shown in the diagram, the above process can automatically complete the process from inputting information such as the location, features, expected output, and evaluation threshold of the initial dataset to outputting the load forecasting results and the load forecasting model interpretation, thus completing the data-driven load forecasting task.

[0081] The feedback model utilizes Python's native compilation and execution environment to provide feedback on the correctness of the code, the accuracy of the results, and error reports. For example, the formula for compilation and execution is:

[0082] If process.returncode! = 0:

[0083] pre_request_gai="\n\nPrevious run code:\n"+responseglm+"\nRun errormessage:\n"+stderr.decode('utf-8','ignore')+"\nPlease regenerate the code."

[0084] S4: Knowledge Distillation. The first step involves the student model attempting tasks from the task pool extracted in S3. If the feedback model reports an error, the student's answer and error message will be automatically added to the prompts in the next round of dialogue for iterative self-attempts. A self-iteration threshold of 15 iterations is set. If the self-iteration exceeds this threshold and still fails, the task is considered a failure, and the task is handed over to the teacher model to complete alone. The teacher model provides excellent code examples for solving the problem and adds them to the high-quality answer corpus. Simultaneously, the student model's incorrect answers and error messages are handed over to the teacher model for resolution, providing high-quality code examples for resolving the error issues. During this process, the student model's errors, poor answers, and error messages are recorded and correspondingly formed into a correction corpus.

[0085] If the student model completes the task in less than 3 iterations, its answer will be added to the high-quality answer corpus as a high-quality answer. This indicates that the student model already has a strong ability for the task and does not require instruction from the teacher model.

[0086] In this embodiment, the CodeGeeX2-6B model, after fine-tuning in steps S1 and S2, is used as the student model, and GLM-4 is used as the teacher model.

[0087] Based on this learning framework, after several rounds of collection, a total of 3,000 high-quality code examples were added to the high-quality answer corpus, and 2,300 corrected responses were collected. This knowledge distillation process results in a smaller corpus for tasks of lower difficulty, while a larger corpus is available for more difficult tasks. This is beneficial for data augmentation of tasks requiring focused improvement.

[0088] S5: Target Data Fine-tuning. Utilizing the 3000 high-quality response corpus collected in the S4 learning framework, Freeze fine-tuning is applied to fine-tune the student model using target data, adapting the student model to the target task. This significantly improves the performance of the student model.

[0089] S6: Scoring Model Training. Using 2300 corrected corpora collected through the S4 learning framework, a scoring model was trained using the PPO algorithm of reinforcement learning to evaluate and improve the task processing capabilities of the fine-tuned model.

[0090] S7: Student Model Testing. The student model is tested using a scoring model to evaluate learning outcomes. If the test score reaches the target score, proceed to S8 for model deployment; otherwise, return to step S4 to re-collect corpus and relearn the material for the unsatisfactory task.

[0091] S8: Model Deployment. After training, deploy the model on a consumer-grade graphics card to test the success rate of the load prediction task.

[0092] The final test results, including the sub-task success rates before and after model training, are shown in Table 2. Table 3 shows a comparison of the average number of iterations required for the trained model and the teacher model to complete the task. This demonstrates that the large language model trained based on this invention, applied to load prediction tasks in HVAC systems, can be completed with a basic consumer-grade graphics card in terms of parameter count. Its task completion capability surpasses that of the teacher model GLM-4, significantly improving the application potential and value of the large language model, as well as the operability and computational efficiency of model deployment. It solves a series of challenges related to the large number of parameters in the model while enhancing the model's professional understanding and problem-solving capabilities in the HVAC field.

[0093] Table 2 Subtask Success Rates Before and After Student Model Training

[0094]

[0095] Table 3 shows the average number of iterations required for the trained student and teacher models to complete the task.

[0096]

[0097]

[0098] The present invention also provides a large language model training device based on knowledge distillation, comprising:

[0099] The student model building module is used to train a large language model for masked language modeling tasks using target domain corpus, enabling it to learn the context and meaning of the target domain terms and phrases; then, the large language model is fine-tuned using the dataset corresponding to the target task in the target domain to adapt it to the question and answer format of the target task, thus obtaining the student model.

[0100] The task pool construction module is used to decompose the target task into several sub-tasks to form a task pool for the target task;

[0101] The high-quality answer corpus pool and the correction corpus acquisition module are used by the student model to attempt to answer tasks in the task pool. If the answer is incorrect, it will iterate itself. If the number of self-iterations exceeds the first preset threshold and still fails, the task will be handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus pool. At the same time, the incorrect answers of the student model will be handed over to the teacher model for correction, providing corrected answers. During this process, the student model's error-reporting tasks, incorrect answers, and teacher model's answers are recorded and matched one by one to form the correction corpus.

[0102] The student model fine-tuning module is used to fine-tune the student model with target data using the high-quality response corpus.

[0103] The scoring model training module is used to train the scoring model using the corrected corpus.

[0104] The student model testing module is used to test the fine-tuned student model using the trained scoring model. If the test score reaches the target score, the training is complete; otherwise, the corpus is collected and the learning is repeated for the tasks that do not meet the target.

[0105] It should be noted that the device embodiment shown in this embodiment matches the content of the above method embodiment, and the content of the above method embodiment can be referred to, and will not be repeated here.

[0106] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for training a large language model based on knowledge distillation, characterized in that, Includes the following steps: Step 1: Use the target domain corpus to train the large language model for a masked language modeling task, enabling it to learn the context and meaning of the target domain terms and phrases; The large language model is then fine-tuned using the dataset corresponding to the target task in the target domain to adapt it to the question and answer format of the target task, thus obtaining the student model; the target domain includes the HVAC field; the target tasks include system control, load forecasting, fault diagnosis, and operation and maintenance tasks in the HVAC field; Step 2: Decompose the target task into several sub-tasks to form a task pool for the target task; Step 3: The student model attempts to answer the tasks in the task pool. If the answer is incorrect, it iterates itself. If the number of iterations exceeds the first preset threshold and still fails, the task is handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus. At the same time, the student model's incorrect answers are handed over to the teacher model for correction, providing corrected answers. During this process, the student model's error-reporting tasks, incorrect answers, and corrected answers are recorded and matched one-to-one to form a correction corpus. Step 4: Using the high-quality response corpus, fine-tune the student model using target data; Step 5: Train the scoring model using the corrected corpus; Step Six: Test the fine-tuned student model using the trained scoring model. If the test score reaches the target score, the training is complete; otherwise, return to Step Three and collect and learn the corpus again for the unsatisfactory tasks.

2. The method according to claim 1, characterized in that, It also includes a feedback model, used to verify the correctness of the student model's answers described in step three.

3. The method according to claim 2, characterized in that, If the test fails, the feedback model will report an error message. The error message and the student model's incorrect answer will be added to the prompt message in the next round of dialogue for self-iterative attempts.

4. The method according to claim 3, characterized in that, If the answer is successfully obtained within the second preset threshold number of iterations, the answer at this time will be added to the high-quality answer corpus.

5. A large language model training device based on knowledge distillation, characterized in that, include: The student model building module is used to train a large language model to perform masked language modeling tasks using target domain corpus, enabling it to learn the context and meaning of the target domain terms and phrases; The large language model is then fine-tuned using the dataset corresponding to the target task in the target domain to adapt it to the question and answer format of the target task, thus obtaining the student model; the target domain includes the HVAC field; the target tasks include system control, load forecasting, fault diagnosis, and operation and maintenance tasks in the HVAC field; The task pool construction module is used to decompose the target task into several sub-tasks to form a task pool for the target task; The high-quality answer corpus pool and the correction corpus acquisition module are used by the student model to attempt to answer tasks in the task pool. If the answer is incorrect, it will iterate itself. If the number of self-iterations exceeds the first preset threshold and still fails, the task will be handed over to the teacher model to complete the task alone, providing a solution example and adding it to the high-quality answer corpus pool. At the same time, the incorrect answers of the student model will be handed over to the teacher model for correction, providing corrected answers. During this process, the student model's error-reporting tasks, incorrect answers, and teacher model's answers are recorded and matched one by one to form the correction corpus. The student model fine-tuning module is used to fine-tune the student model with target data using the high-quality response corpus. The scoring model training module is used to train the scoring model using the corrected corpus. The student model testing module is used to test the fine-tuned student model using the trained scoring model. If the test score reaches the target score, the training is complete; otherwise, the corpus is collected and the learning is repeated for the tasks that do not meet the target.

6. The apparatus according to claim 5, characterized in that, It also includes a feedback model, used to verify the correctness of the student model answers in the high-quality answer corpus and the correction corpus acquisition module.

7. The apparatus according to claim 6, characterized in that, If the test fails, the feedback model will report an error message. The error message and the student model's incorrect answer will be added to the prompt message in the next round of dialogue for self-iterative attempts.

8. The apparatus according to claim 7, characterized in that, If the answer is successfully obtained within the second preset threshold number of iterations, the answer at this time will be added to the high-quality answer corpus.

Images