Large language model knowledge fine-tuning method

Abstract

The application provides a large language model knowledge fine-tuning method, which aims at professional fields to improve the understanding and reasoning ability of the model. The method first extracts knowledge from professional literature, generates question and answer pairs, including correct and incorrect answers, and conducts comparative analysis to enhance the depth of understanding of the model. Further, the question restatement mechanism is introduced to improve the clarity of question expression and the critical thinking of the model. The application of ChoiceBoost technology expands the data set by adjusting the answer order, reduces the position bias, and improves the data utilization. Overall, the application reduces the model reasoning time, enhances the generalization ability, improves the processing efficiency and accuracy of professional field tasks such as network communication tasks, and overcomes the current lack of high-quality training data problem, providing a new fine-tuning paradigm for the development of large language models.

Description

Technical Field

[0001] This invention relates to artificial intelligence large language model technology, and in particular to a method for fine-tuning knowledge of large language models. Background Technology

[0002] With the rise of large language models (MLMs), research on their applications has become a major focus. In the field of network communication, recent research has explored various applications of MLMs, such as network architecture design, network parameter configuration, network performance optimization, and network information security. The pre-training process of MLMs endows them with basic problem understanding capabilities and a large amount of relatively universal common sense. However, in solving highly specialized problems, MLMs still perform poorly due to their weak reasoning ability and lack of specialized knowledge. Existing technologies mainly employ prompting engineering techniques, such as thought chains and context learning, to guide MLMs in generating answers. In addition, a few works have fine-tuned MLMs for specific tasks. Through task-specific model fine-tuning, providing MLMs with longer context, and enabling MLMs to actively generate reasoning steps, the performance of MLMs in handling specific network system tasks has been significantly improved.

[0003] Recent studies have shown that large language models are capable of performing highly specialized network communication tasks; however, existing solutions still have some shortcomings:

[0004] a. The model inference process is inefficient, and the cue word engineering is cumbersome. Although cue word adjustment methods such as thought chains can quickly demonstrate proof of concept, the numerous intermediate steps significantly impact their efficiency. This impact is particularly pronounced in large-scale network applications and those requiring low latency. Furthermore, creating effective cue words requires extensive trial and error, making it a very time-consuming task for engineers.

[0005] b. Weak generalization ability. Since the basic parameters of the large language model itself are not modified, different network tasks require the design of a completely different set of prompt words for the large language model to complete the task. For tasks solved by model fine-tuning, although the computational cost of model inference is reduced, the resulting solution still cannot be generalized to different types of tasks, thus the solution lacks versatility.

[0006] c. The lack of high-quality open-source datasets in the field of network communication leads to low data utilization in model fine-tuning methods. The understanding ability of large language models largely depends on training data. However, the scarcity of high-quality open-source datasets in network communication hinders further improvements in model performance. Furthermore, naive model fine-tuning has limited effectiveness in enhancing model understanding, further limiting the improvement of large language model capabilities in the network communication field.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide a method for fine-tuning knowledge in a large language model.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A method for fine-tuning knowledge in a large language model includes the following steps:

[0011] S1. Extract raw text materials from documents containing professional domain knowledge; analyze the raw text materials using a large language model combined with context learning techniques to extract professional domain knowledge;

[0012] S2. Input the professional domain knowledge into the large language model to generate questions based on the professional domain knowledge points;

[0013] S3. The large language model generates corresponding answers to the question by combining the knowledge of the relevant professional domain, including expected correct answers and deliberately constructed incorrect answers; it guides the large language model to analyze the core points of the correct answer and supplement relevant information of the correct answer; it guides the large language model to identify the specific reasons for the incorrect answers and explain the reasons for the incorrect answers; and it establishes a comparative relationship between the correct and incorrect answers.

[0014] S4. Using a large language model, combine the content generated in steps S2 and S3 with the original text material to generate high-quality question-answer pair data;

[0015] S5. Use the question-and-answer pairs generated in step S4 to fine-tune the parameters of the target baseline model, thereby reducing the inference time of the baseline model while enhancing the model's deep understanding of the professional domain knowledge and its critical thinking and reasoning abilities.

[0016] Furthermore, in step S2, the problem is restated, specifically including: the large language model restates the content of the problem from different perspectives to make the problem description clearer and more specific; and the large language model is guided to supplement background information and details about the problem to eliminate any ambiguity in the problem.

[0017] Furthermore, in step S3, the comparative relationship between the established correct and incorrect answers is explained.

[0018] Furthermore, in step S4, the ChoiceBoost technique is applied to the generated question-answer pairs to create multiple data variants by adjusting the order of the answer options in the question pairs, thereby expanding the dataset size.

[0019] Furthermore, in step S5, the baseline model parameters are fine-tuned using LoRA fine-tuning technology to improve its interpretability.

[0020] Furthermore, in step S5, the change in the loss function during model training is observed, and the training stopping time is determined based on the loss function to avoid overfitting.

[0021] Furthermore, the literature containing professional domain knowledge includes high-quality textbooks in the field of network communication, and the baseline model is a large language model designed to accomplish highly specialized network communication tasks.

[0022] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the large language model knowledge fine-tuning method.

[0023] A computer program product includes a computer program that, when executed by a processor, implements the large language model knowledge fine-tuning method.

[0024] The present invention has the following beneficial effects:

[0025] This invention provides a method for fine-tuning knowledge in large language models, proposing an innovative answer comparison and analysis mechanism. This mechanism significantly improves the understanding and reasoning capabilities of large language models in the field of network communication by introducing comparative analysis of correct and incorrect answers during the fine-tuning process. The innovative fine-tuning mechanism of this invention effectively improves the model's response efficiency, reduces the model's inference time when processing tasks, and enhances the model's deep understanding of network professional knowledge. Furthermore, by introducing a question restatement mechanism, this invention further enhances the model's depth of understanding of questions, enabling the model to understand questions from different perspectives, thereby improving the model's critical thinking ability. In addition, the introduction of the ChoiceBoost data augmentation method, by adjusting the order of answer options to expand the data scale, effectively reduces the impact of answer position bias and enhances the model's utilization of training data, further improving the model's generalization ability. The method of this invention also overcomes the current problem of a lack of high-quality training data in professional fields and the limitations of traditional prompt word engineering. Overall, this invention provides an advanced model fine-tuning framework that not only improves the model's task processing speed and shortens the model's response time, but also enhances the model's generalization ability, comprehension ability, and logical reasoning ability, providing a comprehensive data fine-tuning paradigm for the development of large language models for handling professional tasks such as network communication.

[0026] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the overall process of the large language model knowledge fine-tuning method according to an embodiment of the present invention. Detailed Implementation

[0028] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0029] This invention provides a method for fine-tuning knowledge of a large language model, which significantly improves the professional knowledge understanding and reasoning ability of the large language model. Through automated data mining, question restatement, answer comparison and analysis, data generation and model fine-tuning, it achieves effective fine-tuning of the large language model to adapt to tasks in specific professional fields (such as network communication).

[0030] See Figure 1 This invention provides a method for fine-tuning knowledge in a large language model, comprising the following steps:

[0031] Step S1. Data Mining: Extract raw text materials from documents containing professional domain knowledge; analyze the raw text materials using a large language model combined with context learning techniques to extract professional domain knowledge;

[0032] Step S2. Question Generation: Input the professional domain knowledge into the large language model to generate questions based on the professional domain knowledge points;

[0033] Step S3. Answer Comparison and Analysis: The large language model generates corresponding answers for the question based on the relevant professional domain knowledge, including expected correct answers and intentionally constructed incorrect answers; guides the large language model to analyze the core points of the correct answers and supplement relevant information; guides the large language model to identify the specific reasons for incorrect answers and explain the reasons for the incorrect answers; establishes a comparative relationship between correct and incorrect answers.

[0034] Step S4. Data Generation: Using a large language model, combine the content generated in steps S2 and S3 with the original text material to generate high-quality question-answer pair data;

[0035] Step S5. Model fine-tuning: Use the question-and-answer pairs generated in step S4 to fine-tune the parameters of the target baseline model, thereby reducing the inference time of the baseline model while enhancing the model's deep understanding of the professional domain knowledge and its critical thinking and reasoning abilities.

[0036] For example, in one particular embodiment, the literature containing domain-specific knowledge includes high-quality textbooks in the field of network communications, and the baseline model is a large language model designed to accomplish highly specialized network communication tasks.

[0037] In a preferred embodiment, step S2 further includes restating the problem, specifically including: restating the content of the problem from different perspectives by the large language model to make the problem description clearer and more specific; and guiding the large language model to supplement background information and details about the problem to eliminate any ambiguity in the problem.

[0038] In some embodiments, step S3 involves interpreting the established comparison between correct and incorrect answers.

[0039] In some embodiments, in step S4, the ChoiceBoost technique is applied to the generated question-answer pairs to create multiple data variants by adjusting the order of the answer options in the question pair (e.g., including one correct answer and multiple incorrect answers), thereby expanding the dataset size.

[0040] This invention proposes a knowledge fine-tuning method for large language models, specifically targeting specialized domains. By introducing mechanisms such as answer comparison analysis, it significantly enhances the model's comprehension and reasoning abilities. The method first extracts knowledge from specialized literature, generating question-answer pairs (including correct and incorrect answers), and then performs comparative analysis to enhance the model's deep understanding. Furthermore, a question restatement mechanism is introduced to improve the clarity of question formulation and the model's critical thinking capabilities. In addition, ChoiceBoost technology is applied to expand the dataset by adjusting the order of answers, reducing positional bias and improving data utilization. Overall, this invention reduces model inference time, enhances generalization ability, and significantly improves the efficiency and accuracy of large models in handling specialized tasks, providing a new fine-tuning paradigm for the development of large language models. This invention also overcomes the current lack of high-quality training data in specialized domains and the limitations of traditional prompt word engineering. By simulating question restatement and answer comparison analysis in the human learning process, it significantly improves model performance.

[0041] The following describes specific embodiments of the present invention.

[0042] System framework:

[0043] AI-assisted data mining: Using large language models to extract training data from documents (such as textbooks) containing professional domain knowledge.

[0044] ChoiceBoost data augmentation: Expand your dataset up to four times its original size by creating variants.

[0045] The restatement and contrast model fine-tuning framework: The mined data is processed into question-answer pairs including questions, restates, answers, and answer parsing, and LoRA (low-rank adaptation) fine-tuning is performed on the large language model.

[0046] Model evaluation phase: The model's comprehension and logical reasoning abilities are assessed.

[0047] Model encapsulation and application deployment: The trained model is encapsulated into an interface for easy API calls or cloud deployment.

[0048] Workflow

[0049] Process 1. Data Preparation Stage

[0050] Phase 1.1 Data Mining Process

[0051] Step 1.1.1 Extract raw text materials from documents containing professional knowledge (such as high-quality network communication textbooks).

[0052] Step 1.1.2 Utilize AI-assisted data mining by using large language models (e.g., GPT-4) combined with context learning techniques to analyze the original materials.

[0053] Automated data mining can be used to efficiently transform teaching materials into high-quality training data.

[0054] Phase 1.2 Problem Restatement Processing

[0055] Step 1.2.1 Input the network communication knowledge content into the large language model to generate questions.

[0056] Step 1.2.2 The large language model rephrases the problem content, making the original problem description clearer and more specific.

[0057] Step 1.2.3 guides the large language model to supplement necessary background information and details, eliminating possible ambiguities in the problem.

[0058] Phase 1.3 Answer Comparison and Analysis

[0059] Step 1.3.1 The large language model combines network communication knowledge to generate answers to the questions, including expected correct answers and intentionally constructed incorrect answers, such as one correct answer and multiple incorrect answers. The large language model is then guided to analyze the core points of the correct answer and supplement relevant information.

[0060] Step 1.3.2 guides the large language model to identify the specific reasons for incorrect answers and explains the reasons for the incorrect answers.

[0061] Step 1.3.3 Establish a comparison between the correct and incorrect answers and provide an explanation.

[0062] Phase 1.4 Data Generation

[0063] Step 1.4.1 Utilize the large language model to combine the restated and analyzed content above with the extracted raw data materials to automatically generate high-quality question-answer pair data.

[0064] Phase 1.5 Data Augmentation Process

[0065] Step 1.5.1 applies ChoiceBoost technology to the generated question-answer pairs, creating multiple variants by adjusting the order of options, thus expanding the dataset size to four times its original size.

[0066] Data augmentation is achieved by adjusting the order of options, which expands the data scale while keeping the knowledge content unchanged. This effectively reduces the impact of answer position bias and enhances the model's utilization of training data.

[0067] Process 2. Model Fine-tuning Stage

[0068] Phase 2.1 Parameter Optimization Process

[0069] Step 2.1.1 Input the restated problem and comparative analysis data into the baseline model. In this embodiment, the baseline model used is Llama-2-7B.

[0070] Step 2.1.2 Use LoRA (low-rank adaptation) fine-tuning technique to fine-tune the baseline model parameters to improve its interpretability.

[0071] Step 2.1.3 Observe the changes in the loss function during model training, and determine the training stopping time based on the loss function to avoid overfitting.

[0072] Process 3. Model Evaluation Phase

[0073] Phase 3.1 Multi-dimensional Testing

[0074] Step 3.1.1 Design simple, difficult, and comprehensive test sets. Evaluate the trained model on these three test sets, measuring its accuracy performance across different difficulty levels. Assess the model's logical reasoning, comprehension, and generalization abilities based on the accuracy of its responses.

[0075] Phase 3.2 Performance Verification

[0076] Step 3.2.1 Compare and test with other baseline models, analyze the accuracy of different models' answers, and analyze the improvement in accuracy.

[0077] Step 3.2.2 Design K-Fold cross-validation experiments and demonstrate the effectiveness of the restatement and comparison fine-tuning framework.

[0078] Step 3.2.3 Design ablation experiments and demonstrate that each part of the restatement and comparison fine-tuning framework is important and indispensable.

[0079] Process 4. Application Deployment Phase

[0080] Phase 4.1 Model Encapsulation

[0081] Step 4.1.1 Package the trained model.

[0082] Step 4.1.2 Based on the training data and related code, the model can be further developed and studied.

[0083] This invention addresses the problems encountered by large language models in specialized fields such as network communication applications, as mentioned above. By fine-tuning large language models using the method described in this invention, the model's comprehension and reasoning abilities are enhanced while reducing inference time. This invention proposes an innovative mechanism for model fine-tuning using a question restatement and comparison method: by simulating human learning through question restatement and comparative reflection, a question restatement mechanism and an answer comparison analysis mechanism are introduced into the model fine-tuning framework. Multi-angle restatement enhances the model's depth of understanding of the question and improves its comprehension and logical reasoning abilities in the network domain. Furthermore, the model fine-tuning method also solves the problem of low model response efficiency caused by long contexts, shortening the model's response time. Moreover, when applied to the network communication field, the data mined by the method of this invention can cover various categories in the network communication field, thus adapting to various network communication tasks and exhibiting stronger generalization ability compared to other model fine-tuning methods.

[0084] Specifically, the embodiments of the present invention have the following advantages:

[0085] a. Improve model response efficiency: By fine-tuning the model, the time-consuming prompt word design process is avoided, improving the model's efficiency in adapting to network tasks. At the same time, the shorter context information also greatly reduces the time required for model inference.

[0086] b. Enhance the model's generalization ability. Most existing large language models in the field of network communication are based on specialized data from specific subdomains. To enable the model to perform various tasks, we carefully selected teaching materials in the field of network communication and conducted data mining on them. By introducing question restatement and answer comparison analysis mechanisms, we enhanced the model's deep understanding of network professional knowledge and its critical thinking ability.

[0087] c. To address the current lack of high-quality training data, a data mining method based on a large language model is provided to automatically generate suitable training data from professional textbooks, improving data acquisition efficiency. Furthermore, a model fine-tuning method based on restatement and comparison, as well as the ChoiceBoost data augmentation method, are proposed to solve the problem of low utilization of training data in the naive model fine-tuning method. The restatement and comparison method overcomes the limitations of traditional prompt word engineering, significantly improving model performance by simulating question restatement and answer comparison analysis in the human learning process, with a test accuracy improvement of 63.73% compared to the base model.

[0088] This invention improves the response efficiency and generalization ability of the model. Compared with existing technologies, the embodiments of this invention reduce the inference time of the model when processing tasks and enhance the understanding of network professional knowledge by the large language model. The model exhibits better generalization performance when solving various network professional domain tasks. Regarding training data utilization, a restatement and comparison method is designed, which significantly improves the model's understanding and logical reasoning abilities. In summary, this invention provides an advanced model fine-tuning framework that improves the overall performance of network communication large language models in terms of task processing speed, model generalization ability, understanding ability, and logical reasoning ability, providing a data fine-tuning paradigm for the future development of network communication large language models.

[0089] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0090] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0091] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0092] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk drive or magnetic tape drive. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0093] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0094] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0095] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0096] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0098] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0099] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0100] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0101] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. A method for fine-tuning knowledge in a large language model, characterized in that, Includes the following steps: S1. Extract raw text materials from documents containing professional domain knowledge; analyze the raw text materials using a large language model combined with context learning techniques to extract professional domain knowledge; S2. Input the professional domain knowledge into the large language model to generate questions based on the professional domain knowledge points; S3. The large language model generates corresponding answers to the question by combining the knowledge of the relevant professional field, including expected correct answers and intentionally constructed incorrect answers; Guide the large language model to analyze the core points of the correct answer and supplement relevant information; guide the large language model to identify the specific reasons for the incorrect answer and explain the reasons for the incorrect answer; establish a comparative relationship between the correct answer and the incorrect answer; S4. Using a large language model, combine the content generated in steps S2 and S3 with the original text material to generate high-quality question-answer pair data; S5. Use the question-and-answer pairs generated in step S4 to fine-tune the parameters of the target baseline model, thereby reducing the inference time of the baseline model while enhancing the model's deep understanding of the professional domain knowledge and its critical thinking and reasoning abilities.

2. The large language model knowledge fine-tuning method of claim 1, wherein, In step S2, the problem is restated, which specifically includes: the large language model restating the content of the problem from different perspectives to make the problem description clearer and more specific; and guiding the large language model to supplement background information and details about the problem to eliminate any ambiguity in the problem.

3. The large language model knowledge fine-tuning method of claim 1 or 2, wherein, In step S3, the comparison between the established correct and incorrect answers is explained.

4. The large language model knowledge fine-tuning method of any one of claims 1-3, wherein, In step S4, the ChoiceBoost technique is applied to the generated question-answer pairs to create multiple data variants by adjusting the order of the answer options in the question pairs, thereby expanding the dataset size.

5. The large language model knowledge fine-tuning method of any one of claims 1 to 4, wherein, In step S5, the LoRA fine-tuning technique is used to fine-tune the baseline model parameters to improve its interpretability.

6. The large language model knowledge fine-tuning method of any one of claims 1-5, wherein, In step S5, observe the changes in the loss function during model training and determine the training stopping time based on the loss function to avoid overfitting.

7. The method for fine-tuning knowledge of a large language model as described in any one of claims 1 to 6, characterized in that, The literature containing professional domain knowledge includes high-quality textbooks in the field of network communication, and the baseline model is a large language model designed to accomplish highly specialized network communication tasks.

8. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the large language model knowledge fine-tuning method as described in any one of claims 1 to 7.

9. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the large language model knowledge fine-tuning method as described in any one of claims 1 to 7.

Images