A fine-tuning method, system, device and medium based on a large language model
By introducing knowledge graphs as control modalities into large language models and utilizing logistic chain decomposition triples for training data and attention-controlled fine-tuning networks, the problems of insufficient specialization and uncertainty in vertical domains of large language models are solved, achieving efficient and reliable fine-tuning results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CLP JIUTIAN INTELLIGENT TECH CO LTD
- Filing Date
- 2023-09-27
- Publication Date
- 2026-06-05
AI Technical Summary
Existing large language models lack specialization capabilities in vertical business and professional fields, suffer from factual errors and illusions, and existing fine-tuning methods are costly, have uneven data distribution, lose model versatility, and have uncertain results.
By using knowledge graphs as control modalities for fine-tuning, and utilizing knowledge graph triples trained on logistic chain decomposition data, a high-quality LLM fine-tuning dataset is generated. This dataset is then combined with an attention-controlled fine-tuning network and a low-rank attention decomposition fine-tuning module to enhance the reliability and determinism of large language models.
It achieves reliable and deterministic output of large language models in vertical domains, enhances the understanding and reasoning ability of knowledge graphs, reduces illusion phenomena, and maintains the general reasoning ability of the model.
Smart Images
Figure CN117290480B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial automation technology, and more specifically, to a fine-tuning method, system, device, and medium based on a large language model. Background Technology
[0002] Natural Language Processing (NLP) tasks have experienced rapid development since the advent of Transformer models. In November 2022, OpenAI's ChatGPT achieved over 100 million monthly active users in just three months, becoming the fastest-growing consumer application in history and bringing Large Language Models (LLMs) into the public eye. The massive number of parameters in Large Language Models (e.g., GPT-3 with 175 billion parameters and PaLM with 540 billion parameters) enables them to exhibit emergent capabilities not found in previously smaller pre-trained Language Models (PLMs), demonstrating remarkable performance in a range of complex tasks such as dialogue, retrieval, question answering, and reasoning.
[0003] While the general language capabilities of large language models are already quite impressive, there is still a lack of expertise in how to specialize LLM capabilities for specific tasks and in vertical business and professional fields. Large language models that have not been fine-tuned suffer from serious factual errors in subdivided fields with ambiguous boundary conditions and precise professional knowledge requirements. The information generated by large language models conflicts with existing sources or cannot be verified by existing knowledge tracing (illusion), which severely restricts the application of large language models in industrial fields.
[0004] To adapt the general capabilities of large language models to specific application domains, it is usually necessary to perform adaptation and fine-tuning based on domain-specific knowledge after pre-training. Existing methods for adapting pre-trained LLMs include instruction tuning and alignment tuning. However, due to the large number of parameters in LLMs, the lack of open-source pre-training datasets, the relative scarcity of domain-specific data, and the catastrophic forgetting of models, these two full-parameter tuning methods are extremely costly in practical applications and training.
[0005] To address the issue of high overhead in full parameter fine-tuning of LLM while preserving its good performance as much as possible, we propose an efficient fine-tuning method for large language models, including four methods: adapter tuning, prefix tuning, prompt tuning, and low-rank tuning (LoRA).
[0006] Existing LLM relies heavily on Prompt prompts and is quite sensitive to changes in Prompt prompts. It requires careful manual design of Prompt prompts to achieve the desired good results.
[0007] Existing fine-tuning methods do not include graph structure data in the form of knowledge graphs, and cannot efficiently utilize the rich, reliable, and trustworthy knowledge graph data that already exists in the industry;
[0008] Existing fine-tuning methods cannot solve the problem of uneven distribution of training data caused by small local datasets and limited expertise, and the fine-tuning effect cannot be guaranteed.
[0009] Existing fine-tuning methods use existing local data for updates, and the updated domain knowledge needs to be retrained; otherwise, illusions will still exist at knowledge boundaries or in vertical domains.
[0010] Existing fine-tuning training methods often result in a certain degree of forgetting of the model's original knowledge after incorporating new knowledge, leading to a loss of the generality of LLM.
[0011] Existing fine-tuning methods input new knowledge into the LLM for fine-tuning, but the reasoning process is uncertain, cannot be accurately traced, and cannot meet the reliability requirements of industrial fields for results. Summary of the Invention
[0012] This invention addresses the shortcomings of existing fine-tuning methods in terms of reliability and determinism by proposing a fine-tuning method, system, device, and medium based on a large language model. By using a knowledge graph as the control mode for fine-tuning and generating a high-quality LLM fine-tuning dataset based on knowledge graph triples trained using logistic chain decomposition, this invention fully utilizes the content and relationships of the knowledge graph, alleviates the problem of uneven distribution of fine-tuning data, and achieves contextual learning based on the knowledge graph. Furthermore, by using multiple industrial knowledge graphs as input to the large language model, the reliability and determinism of the large language model's output are enhanced.
[0013] The specific implementation details of this invention are as follows:
[0014] A fine-tuning method based on a large language model first obtains an industrial knowledge graph from an intelligent middleware platform, generates knowledge graph triple training data based on the input query question and the industrial knowledge graph, then constructs a fine-tuning model based on the LLM large language model, and finally inputs the triple training data into the fine-tuning model to obtain the knowledge graph logical chain tracing answer.
[0015] To better realize the present invention, the fine-tuning method based on a large language model further includes the following steps:
[0016] Step S1: Obtain the industrial knowledge graph from the intelligent platform, and generate knowledge graph triple training data based on the query question to be input and the industrial knowledge graph;
[0017] Step S2: Construct a fine-tuned training model based on the LLM large language model, and call the attention-controlled fine-tuning network to train the fine-tuned training model;
[0018] Step S3: Input the knowledge graph triple training data into the fine-tuning training model to obtain the knowledge graph logical chain source tracing answer.
[0019] To better realize the present invention, step S1 further includes the following steps:
[0020] Step S11: Call the logical chain decomposition method to decompose the query problem and obtain the query subproblems;
[0021] Step S12: Obtain the industrial knowledge graph from the intelligent platform, and generate knowledge graph triple training data based on the query sub-question and the industrial knowledge graph;
[0022] Step S13: Use the knowledge graph triple training data as the control modality to generate query data based on the query question corresponding to the knowledge graph triple data.
[0023] To better realize the present invention, step S12 further includes the following steps:
[0024] Step S121: Search the sub-question of the query and retrieve entities and relationships related to the query question based on the knowledge graph;
[0025] Step S122: Based on the entities and relationships, obtain a knowledge graph subgraph related to the query question;
[0026] Step S123: Decompose the knowledge graph subgraph into atomic-level single logic, and construct knowledge graph triple training data by combining the atomic-level single logic with the query question according to the set format.
[0027] To better realize the present invention, step S2 further includes the following steps:
[0028] Step S21: Based on the LLM large language model, reason about the query question and predict the reasoning result;
[0029] Step S22: Invoke the graph neural network to extract features from the knowledge graph triple training data;
[0030] Step S23: Add a convolutional layer initialized to 0 and connect it to the frozen parameter residual of the LLM large language model, and concatenate the inference result with the features of the knowledge graph triple training data to obtain hybrid features;
[0031] Step S24: Construct a fine-tuned training model based on the hybrid features, and train the fine-tuned training model by calling the low-rank factorization matrix combined with channel and spatial attention mechanisms.
[0032] To better realize the present invention, step S3 further includes the following steps:
[0033] Step S31: Convert the query question in the triplet training data into a word feature vector;
[0034] Step S32: Fuse the word feature vector with the acquired location information to obtain a word feature vector containing the location information;
[0035] Step S33: Reason the word feature vector containing the location information to obtain the reasoning feature vector;
[0036] Step S34: Fuse the reasoning feature vector with the industrial knowledge graph to obtain a feature vector that fuses the knowledge graph information;
[0037] Step S35: Restore the feature vector of the fused knowledge graph information into natural language to obtain the knowledge graph logical chain tracing answer.
[0038] Based on the above-mentioned fine-tuning method based on a large language model, in order to better realize the present invention, a fine-tuning system based on a large language model is further proposed, including a preprocessing unit, a construction unit, and a prediction unit.
[0039] The preprocessing unit is used to obtain industrial knowledge graphs from the intelligent middleware platform and generate knowledge graph triple training data based on the query question to be input and the industrial knowledge graphs.
[0040] The construction unit is used to construct a fine-tuning model based on the LLM large language model;
[0041] The prediction unit is used to input the triple training data into the fine-tuning model to obtain the knowledge graph logic chain tracing answer.
[0042] Based on the above-mentioned fine-tuning method based on a large language model, in order to better realize the present invention, an electronic device is further proposed, including a memory and a processor; the memory stores a computer program; when the computer program is executed on the processor, the above-mentioned fine-tuning method based on a large language model is implemented.
[0043] Based on the above-mentioned fine-tuning method based on a large language model, in order to better realize the present invention, a computer-readable storage medium is further proposed, wherein computer instructions are stored on the computer-readable storage medium; when the computer instructions are executed on the above-mentioned electronic device, the above-mentioned fine-tuning method based on a large language model is implemented.
[0044] The present invention has the following beneficial effects:
[0045] (1) This invention uses knowledge graphs as control modes for fine-tuning and generates high-quality LLM fine-tuning datasets based on knowledge graph triples trained by logical chain decomposition. It can make full use of the content and relationships of knowledge graphs, alleviate the problem of uneven distribution of fine-tuning data, realize context learning based on knowledge graphs, and use multiple industrial knowledge graphs as inputs to large language models, thereby enhancing the reliability and determinism of the output of large language models.
[0046] (2) Based on the knowledge graph triple training data of knowledge graph logical chain decomposition, this invention effectively improves the understanding and reasoning ability of large language models to the logical relationships between entities in knowledge graphs, enhances the zero-sample reasoning ability of large language models, and improves the general reasoning ability of large language models to knowledge graphs.
[0047] (3) This invention uses the attention controllable fine-tuning module ACNet and the low-rank attention decomposition fine-tuning module LRA to connect with the frozen parameter residuals of the large language model. While enhancing the understanding and reasoning ability of the large language model on the knowledge graph, it retains the original general reasoning ability of the large language model to the maximum extent. Attached Figure Description
[0048] Figure 1 This is a schematic diagram of the fine-tuning process provided by the present invention.
[0049] Figure 2 This is a schematic diagram of the knowledge graph input provided by the present invention.
[0050] Figure 3 This is a schematic diagram of the fine-tuning architecture provided by the present invention. Detailed Implementation
[0051] To more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments, and therefore should not be regarded as a limitation on the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0052] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "set up," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0053] Example 1:
[0054] This embodiment proposes a fine-tuning method based on a large language model. First, an industrial knowledge graph is obtained from an intelligent middleware platform. Based on the query question to be input and the industrial knowledge graph, knowledge graph triple training data is generated. Then, a fine-tuning model is constructed based on the LLM large language model. Finally, the triple training data is input into the fine-tuning model to obtain the knowledge graph logical chain tracing answer.
[0055] Furthermore, the fine-tuning method based on a large language model specifically includes the following steps:
[0056] Step S1: Obtain the industrial knowledge graph from the intelligent platform, and generate knowledge graph triple training data based on the query question to be input and the industrial knowledge graph.
[0057] Furthermore, step S1 specifically includes the following steps:
[0058] Step S11: Call the logical chain decomposition method to decompose the query problem and obtain the query subproblems.
[0059] Step S12: Obtain the industrial knowledge graph from the intelligent platform, and generate knowledge graph triple training data based on the query sub-question and the industrial knowledge graph.
[0060] Furthermore, step S12 specifically includes the following steps:
[0061] Step S121: Search the sub-question of the query and retrieve entities and relationships related to the query question based on the knowledge graph.
[0062] Step S122: Based on the entities and relationships, obtain a knowledge graph subgraph related to the query question.
[0063] Step S123: Decompose the knowledge graph subgraph into atomic-level single logic, and construct knowledge graph triple training data by combining the atomic-level single logic with the query question according to the set format.
[0064] Step S13: Use the knowledge graph triple training data as the control modality to generate query data based on the query question corresponding to the knowledge graph triple data.
[0065] Step S2: Construct a fine-tuned training model based on the LLM large language model, and call the attention-controlled fine-tuning network to train the fine-tuned training model.
[0066] Furthermore, step S2 specifically includes the following steps:
[0067] Step S21: Based on the LLM large language model, reason about the query question and predict the reasoning result;
[0068] Step S22: Invoke the graph neural network to extract features from the knowledge graph triple training data;
[0069] Step S23: Add a convolutional layer initialized to 0 and connect it to the frozen parameter residual of the LLM large language model, and concatenate the inference result with the features of the knowledge graph triple training data to obtain hybrid features;
[0070] Step S24: Construct a fine-tuned training model based on the hybrid features, and train the fine-tuned training model by calling the low-rank factorization matrix combined with channel and spatial attention mechanisms.
[0071] Step S3: Input the knowledge graph triple training data into the fine-tuning training model to obtain the knowledge graph logical chain source tracing answer.
[0072] Furthermore, step S3 specifically includes the following steps:
[0073] Step S31: Convert the query question in the triplet training data into a word feature vector;
[0074] Step S32: Fuse the word feature vector with the acquired location information to obtain a word feature vector containing the location information;
[0075] Step S33: Reason the word feature vector containing the location information to obtain the reasoning feature vector;
[0076] Step S34: Fuse the reasoning feature vector with the industrial knowledge graph to obtain a feature vector that fuses the knowledge graph information;
[0077] Step S35: Restore the feature vector of the fused knowledge graph information into natural language to obtain the knowledge graph logical chain tracing answer.
[0078] Working Principle: This embodiment first obtains an industrial knowledge graph from the intelligent platform. Based on the query question to be input and the industrial knowledge graph, it generates knowledge graph triple training data. Then, it constructs a fine-tuning model based on the LLM large language model. Finally, it inputs the triple training data into the fine-tuning model to obtain the knowledge graph logic chain source tracing answer. By using the knowledge graph as a control mode for fine-tuning and generating a high-quality LLM fine-tuning dataset based on the knowledge graph triple training data decomposed by the logic chain, it can fully utilize the content and relationships of the knowledge graph, alleviate the problem of uneven distribution of fine-tuning data, realize contextual learning based on the knowledge graph, and use multiple industrial knowledge graphs as input to the large language model, thereby enhancing the reliability and determinism of the large language model output.
[0079] Example 2:
[0080] This embodiment is based on the above embodiment 1, such as... Figure 1 , Figure 2 , Figure 3 As shown, a specific embodiment will be described in detail.
[0081] The overall flowchart of the efficient fine-tuning method proposed in this embodiment is as follows: Figure 1 As shown, the specific steps include:
[0082] Step S1: Relying on the knowledge graph-based logical chain decomposition method built into the JTian intelligent platform, complex problems are decomposed into single-step queries. Combined with high-quality industrial knowledge graphs, knowledge graph triple data is generated. The constructed knowledge graph triple data is used as the knowledge boundary of the control modality enhancement large language model to enhance the certainty of the generated answer. The query questions corresponding to the triple data are used as normal query data.
[0083] Step S2: Input the knowledge graph triple training data obtained in S1 into the LLM fine-tuning training architecture, and combine it with the Attention Control Network (ACNet) to fine-tune the LLM training, thereby enhancing the understanding and learning of industrial knowledge graphs by the large language model, enhancing the ability of the large language model to decompose complex problems using logical chains and deduce step by step based on facts, and reducing the illusion phenomenon of LLM.
[0084] Step S3: The fine-tuned LLM can automatically associate factual knowledge in the query knowledge graph with the query question and the input relevant knowledge graph triple data, perform reasoning and answer step by step, and finally generate a credible answer result based on logical chains, so as to realize the traceability of the results generated by the large language model and enhance the accuracy and reliability of the results generated by the large language model.
[0085] The knowledge graph triple input data construction method proposed in this embodiment is as follows: Figure 2 As shown.
[0086] During the fine-tuning process of LLM, since the fine-tuning training dataset is much smaller in size and volume than the pre-training dataset, higher requirements are placed on the quality of the fine-tuning dataset in order to minimize the loss of the general capabilities of large language models. At the same time, because LLM has poor understanding and reasoning capabilities for complex problems, CoT (Consciousness Chain Tips) is usually introduced to decompose complex problems into multi-step simple reasoning queries.
[0087] S101: Leveraging the JTian intelligent platform, the query is logically decomposed, and the resulting atomic questions are searched for connections. Entities and relationships related to the query are retrieved from the knowledge graph, resulting in a subgraph of the knowledge graph concerning the query. For example... Figure 2 As shown, to understand the battery's recent power status, the JTian intelligent platform retrieves battery-related entities and attributes from the knowledge graph, including data such as battery power, power supply charging status, and sensor power consumption, and constructs a subgraph of the knowledge graph related to the battery.
[0088] S102: Decompose the retrieved knowledge graph subgraph into atomic-level single logic, and construct knowledge graph triple training data by combining the decomposed triple data with the query question and the confirmed answer according to a preset format.
[0089] The large language model fine-tuning training method proposed in this embodiment is as follows: Figure 3 As shown.
[0090] The efficient fine-tuning method for large language models proposed in this embodiment includes two parts: a parameter freezing part of the large language model ontology, as shown in S201; a knowledge graph control modality input part, as shown in S202; an attention-controllable fine-tuning network ACNet, as shown in S203; and a low-rank attention decomposition fine-tuning network LRA, as shown in S404. The large language model in this embodiment is ChatGLM-6B as an example. Other large language models can adopt the efficient fine-tuning method proposed in this patent. S203 and S204 are the core parts of the fine-tuning model.
[0091] S201: This part represents the original structure of the large language model, performing reasoning on the input query question. All parameters of the original model are frozen and not used in training, preserving the general reasoning capabilities of the large language model to the greatest extent possible. Taking ChatGLM-6B as an example, it includes an Embedding layer that encodes the input, a RotaryEmbedding layer that encodes the position, 28 forward propagation GLMBlock modules (each module includes RMSnorm, SelfAttention, and an MLP layer), and finally outputs the predicted reasoning result.
[0092] S202: This part is the fine-tuning control modal input. The fine-tuning training method proposed in this patent uses knowledge graph as the control modal input. The input content is as shown in S102. After being retrieved and filled, it contains training data including knowledge graph triples. After extracting features through graph neural network GCN, it is input into the fine-tuning model for fine-tuning training.
[0093] S203: This part is the attention-controlled fine-tuning network ACNet. It achieves overall fine-tuning of the large language model module by adding convolutional layers initialized to 0. By linking with the frozen parameter residuals of the large language model itself, it preserves the general reasoning ability of the large language model as much as possible.
[0094] The ACNet module accepts features extracted from the knowledge graph triple training data from the GCNS402 input. After passing through a convolutional layer initialized to 0, the feature dimensions are restored and then concatenated with the question query encoding features extracted from the embedding layer S401 of the ChatGLM-6B model to obtain the mixed input features.
[0095] The input features are passed through the copied GLMBlock module for forward inference. Each GLMBlock module completely copies the weights and parameters of the frozen large language model GLMBlock. The blocks that need to be adjusted are connected to convolutional layers initialized to 0 before and after them, so as to realize the mapping adjustment of the module input and output.
[0096] S204: This part is the Low-Rank Attention Decomposition Fine-Tuning Network (LRA). To reduce the computational cost required for large language models with hundreds of billions of parameters, and to reduce the resources required for inference, a low-rank decomposition matrix combined with channel and spatial attention mechanisms is used to equivalently replace and train the linear layer part of SelfAttention in the GLMBlock module.
[0097] The input of SelfAttention is used as the input of LRA, and the features after multi-head attention passes through the linear layer Ar are concatenated according to the Attention dimension.
[0098] Channel attention weights are obtained using global max pooling (MaxPool) and global average pooling (AvgPool) based on a single attention direction.
[0099] The obtained features are concatenated and then passed through a multilayer perceptron (MLP) to obtain feature attention weights with the same number of output channels as the Ar layer.
[0100] After passing through the Channel Attention Fusion (CAF) module, the feature attention weights obtained from the MLP are multiplied with the corresponding features of the Ar layer to obtain the fused channel attention features.
[0101] The features processed by the CAF module are used in each layer to obtain spatial attention using MaxPool and AvgPool based on the attention channel direction, respectively.
[0102] Spatial attention reduces the dimensionality to the same dimension as a single attention feature through 1*1 convolution;
[0103] The dimensionality-reduced features are processed by the Spatial Attention Fusion (SAF) module and then dot-producted with the fused channel attention features to obtain the fused attention features.
[0104] The fused attention features are residually linked with the features from Ar after passing through a ReLU activation layer, and then restored to the output dimension of SelfAttention in GLMBlock through a linear layer Br;
[0105] The features of Br are fed as input to RMSnorm for inference training of the model.
[0106] Overall training process:
[0107] The input to S201 is the query question (Language Input) in the knowledge graph triple training data, which is transformed into a word feature vector through the encoding layer Embedding;
[0108] The word feature vector is fused with positional information through the RotaryEmbedding positional encoding layer, becoming a feature vector containing positional information;
[0109] The feature vectors containing location information are fed into the inference module GLMBlock and the S203 attention-controlled fine-tuning network ACNet, respectively. In addition, the fine-tuning module also receives input from the S202 graph neural network GCN.
[0110] The feature vector containing location information enters the inference module and is then processed by mean square layer normalization (RMSNorm), self-attention layer (SelfAttention), mean square layer normalization (RMSNorm), and multilayer perceptron (MLP) to perform inference on the feature vector containing location information.
[0111] The features output by the inference module are added to the features after passing through the attention-controlled fine-tuning network ACNet to obtain a feature vector that integrates knowledge graph information.
[0112] After 28 inference modules GLMBlock infer sequentially, each inference module shares input with the attention-controlled fine-tuning network ACNet, and the fine-tuned feature vectors output by the summation of the attention-controlled fine-tuning network ACNet are fused together to obtain the inference feature vectors of the question based on knowledge graph information.
[0113] The fused and added inference feature vectors are normalized by the mean square layer RMSNorm and then input into the output layer to restore the feature vectors into natural language, thus obtaining a credible answer based on the knowledge graph.
[0114] The initial input of the S203 Attention Controllable Fine-Tuning Network ACNet is the triple data KG Control Input from the knowledge graph triple training data. After being compressed into the feature space by the graph neural network GCN, it is mapped to the knowledge graph triple feature vector.
[0115] The feature vector of the knowledge graph triple is input into the attention-controlled fine-tuning network ACNet. After feature space adjustment by the convolutional layer Zero Conv initialized to 0, it is concatenated with the feature vector containing positional information input by S201 to obtain the feature vector of the knowledge graph triple training data.
[0116] Subsequently, each inference module GLMBlock corresponds to an attention-controlled fine-tuning network ACNet;
[0117] The input of each attention-controlled fine-tuning network ACNet is the feature vector after inference based on knowledge graph information, which has been adjusted and mapped by the convolutional layer ZeroConv initialized to 0 in the previous layer. It then passes through the mean square layer normalization RMSNorm, the self-attention layer SelfAttention connected to the residual of the low-rank attention decomposition fine-tuning network LRA in S204, the mean square layer normalization RMSNorm, and the multilayer perceptron MLP for inference. Each module passes through a convolutional layer Zero Conv initialized to 0 to map and adjust the inferred feature vector. The mapped and adjusted feature vector is shared as the input of the inference module GLMBLock in S201 and S203.
[0118] Each SelfAttention layer in the attention-controlled fine-tuning network ACNet is connected to the residual of the S204 low-rank attention decomposition fine-tuning network LRA.
[0119] The answers and conclusions defined in the knowledge graph triple training data are transformed into result feature vectors through the same encoding layer, Embedding. These feature vectors, along with the feature vectors obtained after ChatGLM-6B inference, are used as inputs to the ChatGLM-6B loss function to perform backpropagation and update the model. In this process, S201 freezes all parameters and does not participate in the training.
[0120] This embodiment uses an industrial knowledge graph as the control modal input of a large language model, which enhances the large language model's understanding of the knowledge graph and its ability to query complex questions based on the knowledge graph's logical chain decomposition. It defines knowledge graph triple training data suitable for the large language model based on logical chain decomposition, weakening the large language model's sensitivity to prompt words and enhancing its ability to align with human intent. Using knowledge graph logical chain reasoning as the output of the large language model enables traceability of the generated results, enhancing the reliability and accuracy of the large language model's output.
[0121] This embodiment proposes an efficient fine-tuning training framework for large language models. It uses GCN fused with knowledge graph as control input and employs attention controllable mechanism and low-rank attention mechanism to achieve efficient and controllable incremental fine-tuning of large language models, thereby enhancing the question answering and querying capabilities of large language models based on knowledge graph content and relationships.
[0122] This embodiment proposes an attention-controllable neural network module, ACNet, which replicates the modules that need fine-tuning in a large language model. By using convolutional layers with pre- and post-connections initialized to 0, the features of the input and output are adjusted. The features output by the fine-tuning module are aligned with and added to the features output by the frozen module to form the new output. The fusion of the fine-tuning module and the frozen module is achieved through residual connections, which can effectively enhance the understanding of new knowledge in a large language model while preserving the original general reasoning ability as much as possible.
[0123] The Low-Rank Attention Decomposition Fine-Tuning Network Module (LRA) proposed in this embodiment decomposes the linear layers in the SelfAttention model of a large language model into low-rank matrices. At the same time, it introduces the Channel Attention Fusion Module (CAF) and the Spatial Attention Fusion Module (SAF) in the low-rank space to enhance the extraction and recognition of key features. While reducing the training parameters of the large language model, it enhances the semantic understanding of the control modality and improves the fine-tuning effect.
[0124] This embodiment proposes a method for constructing training data for knowledge graph triples based on logical chain decomposition. Taking JTian intelligent platform as an example, it realizes context learning (ICL) based on knowledge graphs, which can directly use various industrial knowledge graphs as input for large language models. The invention proposes using knowledge graphs as control modes for fine-tuning, and using knowledge graph triple training data based on logical chain decomposition to generate high-quality LLM fine-tuning datasets. This can make full use of the content and relationships of knowledge graphs and alleviate the problem of uneven distribution of fine-tuning data.
[0125] The knowledge graph triple training data based on knowledge graph logical chain decomposition proposed in this embodiment can effectively improve the understanding and reasoning ability of large language models to logical relationships between entities in knowledge graphs, enhance the zero-shot reasoning ability of large language models, and improve the general reasoning ability of large language models to knowledge graphs.
[0126] The efficient fine-tuning method for large language models proposed in this embodiment uses a fine-tuning method based on the attention-controllable fine-tuning module ACNet and the low-rank attention decomposition fine-tuning module LRA, which is connected to the frozen parameter residuals of the large language model. This method enhances the understanding and reasoning ability of the large language model on the knowledge graph while preserving the original general reasoning ability of the large language model to the maximum extent.
[0127] The efficient fine-tuning method and knowledge graph triple training data based on knowledge graph logic chains proposed in this embodiment fine-tune the output of the large language model to traceable answers based on knowledge graph logic chain decomposition, thereby enhancing the reliability and determinism of the large language model output.
[0128] This embodiment is applied to the field of industrial automation. It uses existing multimodal data such as archival texts, fault records, and knowledge graphs in the industrial field to fine-tune the large language model, enhance the large language model's dialogue and question-answering capabilities in vertically segmented fields, alleviate the illusion problem of the large language model, and can provide deterministic answers to questions such as the current factory production environment, product quality, and equipment status. It also proposes constructive improvement plans and optimization suggestions based on existing methods.
[0129] This embodiment takes the JTian-GLM intelligent platform as an example. The model uses a local knowledge vector library built by the intelligent platform, which includes a multimodal dataset of industrial fields such as archival text, fault records, and knowledge graphs. It automatically constructs knowledge graph triple training data, uses an enhanced retrieval query method, and uses a fine-tuned large language model for reasoning to obtain reliable deterministic answers and suggestions based on the enterprise's local knowledge base.
[0130] The method for fine-tuning large language models includes constructing training data for knowledge graph triples by integrating enterprise knowledge graph logical chains and causal graphs for context learning (ICL) and thought chain hints (CoT) for index enhancement; it also includes using an efficient and controllable neural network based on an attention mechanism; and it achieves efficient and controllable fine-tuning of existing large language models in professional knowledge and vertical domains.
[0131] The fine-tuning training method proposed in this embodiment uses knowledge graphs as control modalities, is insensitive to prompt words, enhances the robustness and applicability of LLM, and improves the user experience; it can efficiently fine-tune large language models by inputting knowledge graphs as new modalities into the large language model, achieving efficient utilization of existing knowledge graphs in the industry; it relies on the JTian-GLM intelligent platform to decompose complex problems into logical chains, and constructs a knowledge graph triple training dataset in combination with industrial knowledge graphs, improving the large language model's understanding and reasoning ability of knowledge graphs; it achieves the integration of knowledge graph reasoning ability with the general capabilities of LLM, enhancing the large language model's ability to use knowledge graphs, achieving zero-shot reasoning ability on new knowledge graphs, and effectively avoiding illusions; it integrates a new attention network for fine-tuning while retaining the original weights of LLM, achieving the integration of vertical domain professional knowledge, and retaining the general capabilities of the large language model as much as possible; it achieves the integration of LLM with knowledge graph logical chains, and can output the reasoning process in the form of knowledge graph logical chains, realizing the traceability of reasoning results and reasoning processes, and improving the reliability of LLM reasoning results.
[0132] The other parts of this embodiment are the same as those in Embodiment 1 above, so they will not be described again.
[0133] Example 3:
[0134] Based on any one of Embodiments 1-2 above, this embodiment proposes a fine-tuning system based on a large language model, including a preprocessing unit, a construction unit, and a prediction unit;
[0135] The preprocessing unit is used to obtain industrial knowledge graphs from the intelligent middleware platform and generate knowledge graph triple training data based on the query question to be input and the industrial knowledge graphs.
[0136] The construction unit is used to construct a fine-tuning model based on the LLM large language model;
[0137] The prediction unit is used to input the triple training data into the fine-tuning model to obtain the knowledge graph logic chain tracing answer.
[0138] This embodiment also proposes an electronic device, including a memory and a processor; the memory stores a computer program; when the computer program is executed on the processor, it implements the above-described fine-tuning method based on a large language model.
[0139] This embodiment also proposes a computer-readable storage medium storing computer instructions; when the computer instructions are executed on the aforementioned electronic device, the aforementioned fine-tuning method based on a large language model is implemented.
[0140] The other parts of this embodiment are the same as any one of the above embodiments 1-2, so they will not be described again.
[0141] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.
Claims
1. A fine-tuning method based on a large language model, characterized in that, First, the industrial knowledge graph is obtained from the intelligent middleware platform. Based on the query question to be input and the industrial knowledge graph, knowledge graph triple training data is generated. Then, a fine-tuning model is constructed based on the LLM large language model. Finally, the triple training data is input into the fine-tuning model to obtain the knowledge graph logical chain tracing answer. The fine-tuning method based on a large language model specifically includes the following steps: Step S1: Obtain the industrial knowledge graph from the intelligent platform, and generate knowledge graph triple training data based on the query question to be input and the industrial knowledge graph; Step S2: Construct a fine-tuned training model based on the LLM large language model, and call the attention-controlled fine-tuning network to train the fine-tuned training model; Step S3: Input the knowledge graph triple training data into the fine-tuning training model to obtain the knowledge graph logical chain source tracing answer; Step S1 specifically includes the following steps: Step S11: Call the logical chain decomposition method to decompose the query problem and obtain the query subproblems; Step S12: Obtain the industrial knowledge graph from the intelligent platform, and generate knowledge graph triple training data based on the query sub-question and the industrial knowledge graph; Step S13: Using the knowledge graph triple training data as the control modality, generate query data based on the query question corresponding to the knowledge graph triple training data; Step S12 specifically includes the following steps: Step S121: Search the sub-question of the query and retrieve entities and relationships related to the query question based on the knowledge graph; Step S122: Based on the entities and relationships, obtain a knowledge graph subgraph related to the query question; Step S123: Decompose the knowledge graph subgraph into atomic-level single logic, and construct knowledge graph triple training data by combining the atomic-level single logic with the query question according to the set format; Step S2 specifically includes the following steps: Step S21: Based on the LLM large language model, reason about the query question and predict the reasoning result; Step S22: Invoke the graph neural network to extract features from the knowledge graph triple training data; Step S23: Add a convolutional layer initialized to 0 and connect it to the frozen parameter residual of the LLM large language model, and concatenate the inference result with the features of the knowledge graph triple training data to obtain hybrid features; Step S24: Construct a fine-tuned training model based on the hybrid features, and train the fine-tuned training model by calling the low-rank factorization matrix combined with channel and spatial attention mechanisms.
2. The fine-tuning method based on a large language model according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step S31: Convert the query question in the triplet training data into a word feature vector; Step S32: Fuse the word feature vector with the acquired location information to obtain a word feature vector containing the location information; Step S33: Reason the word feature vector containing the location information to obtain the reasoning feature vector; Step S34: Fuse the reasoning feature vector with the industrial knowledge graph to obtain a feature vector that fuses the knowledge graph information; Step S35: Restore the feature vector of the fused knowledge graph information into natural language to obtain the knowledge graph logical chain tracing answer.
3. A fine-tuning system based on a large language model, used to execute the fine-tuning method based on a large language model as described in claim 1; characterized in that, It includes a preprocessing unit, a building unit, and a prediction unit; The preprocessing unit is used to obtain industrial knowledge graphs from the intelligent middleware platform and generate knowledge graph triple training data based on the query question to be input and the industrial knowledge graphs. The construction unit is used to construct a fine-tuning model based on the LLM large language model; The prediction unit is used to input the triple training data into the fine-tuning model to obtain the knowledge graph logic chain tracing answer.
4. An electronic device, characterized in that, It includes a memory and a processor; the memory stores a computer program; when the computer program is executed on the processor, it implements the fine-tuning method based on a large language model as described in any one of claims 1-2.
5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions; when the computer instructions are executed on the electronic device as described in claim 4, they implement the fine-tuning method based on a large language model as described in any one of claims 1-2.