Large language model fine tuning method for power supply industry knowledge retrieval

Abstract

The invention provides a power supply industry knowledge retrieval-oriented large language model fine tuning method. The method comprises the following steps of 1, deploying an RAG system in a working environment of a power supply industry; 2, inputting a query text, and returning a plurality of query results by the knowledge base of the RAG system; step 3, using the query text and the query result to construct a training corpus record, and performing preprocessing to obtain a word index sequence; step 4, constructing a pseudo prompt sequence string, and adding an adjustable module in the large language model LLM; and step 5, training and optimizing the large language model LLM added with the adjustable module by using the word index sequence, and finishing fine tuning of the large language model for power supply industry knowledge retrieval after repeated iteration.

Description

Technical Field

[0001] The invention relates to a large language model fine-tuning method, in particular to a large language model fine-tuning method for knowledge retrieval in the power supply industry. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] In the field of natural language processing, pre-trained large language models (LLM) have achieved great success in text classification, knowledge retrieval, knowledge question answering, etc. However, the corpus used in pre-trained LLM comes from open datasets and is not optimized for specific industries, which results in poor performance in applications in specific industries. It is usually necessary to fine-tune it using corpus from specific industries.

[0004] There are two main methods for fine-tuning LLM when retraining with new corpus. One is to adjust all or part of the parameters of the original model; the other is to freeze all the parameters of the pre-trained model and only adjust the newly added parameters. The amount of parameters that need to be adjusted in the former method is usually much larger than the latter. The more parameters that need to be adjusted, the more corpus is required. The present invention adopts the latter method, and the reference idea is derived from the P-Tuning v2 algorithm (reference: Liu X, Ji K, FuY, et al. P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks[J]. 2021. DOI: 10.48550 / arXiv.2110.07602.). The core idea of ​​P-Tuning v2 is to introduce continuous trainable prompts at each layer of LLM to guide the model to generate outputs that meet the requirements. It has the following advantages: First, it is highly memory efficient. Since there is no need to store and update all parameters, the memory consumed during training is greatly reduced. Second, it is highly flexible. Users can easily adjust the model output by modifying the prompt prefix, which enables the model to adapt to a variety of different tasks and scenarios without making a lot of modifications to the model itself. Third, it is highly scalable and can be used in combination with any pre-trained language model, which means that users can choose the most suitable pre-trained model according to their specific needs to achieve the best performance.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to provide a large language model fine-tuning method for knowledge retrieval in the power supply industry in view of the shortcomings of the prior art.

[0007] In order to solve the above technical problems, the present invention discloses a large language model fine-tuning method for knowledge retrieval in the power supply industry, comprising the following steps:

[0008] Step 1: deploy the RAG system in the working environment of the power supply industry;

[0009] Step 2, input the query text, and the knowledge base of the RAG system returns multiple query results;

[0010] Step 3, constructing a training corpus record using the query text and query results, and performing preprocessing to obtain a word index sequence;

[0011] Step 4: construct a pseudo prompt sequence string and add an adjustable module to the large language model LLM;

[0012] Step 5, using the word index sequence, training and optimizing the large language model LLM after adding the adjustable module, and completing the fine-tuning of the large language model for knowledge retrieval in the power supply industry after repeated iterations.

[0013] Further, the step of obtaining the word index sequence in step 3 includes:

[0014] Step 3-1, let the query text be x, and the text after the corresponding multiple query results are merged, that is, the reference text, be y, and combine to obtain a training corpus record (x, y);

[0015] Step 3-2, use the word segmentation tool to convert the training corpus record (x, y) into a word index sequence (x_i, y_i), where the sequence elements are discrete integers, x_i is the index number of the word in the prompt text in the dictionary, and y_i is the index number of the word in the reference text in the dictionary.

[0016] Furthermore, the construction of the pseudo prompt sequence string described in step 4 includes:

[0017] The pseudo prompt sequence string p_i is a discrete integer sequence generated by a random function.

[0018] Furthermore, the adjustable module is added to the large language model LLM described in step 4, including:

[0019] Pseudo-prompt vector conversion module and pseudo-prompt string generation module; wherein,

[0020] The structure of the pseudo-prompt vector conversion module is a two-dimensional matrix, which is used to convert integers into vectors as the first layer of the adjustable module;

[0021] The number of the pseudo-prompt string generation modules is consistent with the number of layers of the large language model LLM. Multiple pseudo-prompt string generation modules form a layered structure and are sequentially arranged after the pseudo-prompt vector conversion module. Their outputs are respectively used as the input of each layer of the encoder in the large language model LLM.

[0022] Furthermore, the pseudo prompt string generation module described in step 4 adopts a multi-layer perceptron model.

[0023] Further, the pseudo prompt vector conversion module described in step 4 converts the pseudo prompt sequence string p_i into a pseudo prompt vector p_v;

[0024] The word index sequence (x_i, y_i) is mapped to a word index vector (x_v, y_v) through the embedding layer in the large language model LLM, where x_v is the query vector and y_v is the result vector;

[0025] The pseudo-prompt vector p_v is concatenated with the query vector x_v to obtain the input prompt vector [p_v, x_v];

[0026] The input prompt vector [p_v, x_v] is calculated by the pseudo prompt string generation module to obtain a new vector, which is concatenated with the query vector x_v and used as the input of the encoder of the current layer in the large language model LLM.

[0027] Furthermore, the large language model LLM after adding the adjustable module as described in step 5 is trained and optimized, that is, the parameters of the large language model LLM are fixed, and the parameters in the adjustable module are adjusted during the training process.

[0028] Furthermore, the training and optimization of the large language model LLM after adding the adjustable module described in step 5 specifically includes:

[0029] Step 5-1, taking the index number y_i of the word in the reference text described in step 3-2 in the dictionary as a reference label, and performing a smoothing operation on it;

[0030] Step 5-2, the index number y_i after the smoothing operation is used as the output of the large language model LLM after adding the adjustable module, that is, the prediction result, and the autoregressive mode is used for training, that is, the loss of the prediction result is calculated according to the reference label one by one, where the loss function loss is measured by the KL distance, which is expressed as follows:

[0031]

[0032] Where L represents the number of words, Q(j) represents the jth element value of the probability vector, Q is the probability vector, and P is the predicted probability vector;

[0033] Step 5-3, calculate the total loss LOSS, as follows:

[0034]

[0035] Among them, λ is the discount value, t represents the word y t The position in the sequence y, the first word is numbered 0, and the last word is numbered T;

[0036] Step 5-4, using the gradient descent method to update the large language model LLM after adding the adjustable module, that is, freezing all parameters in the large language model LLM during gradient back propagation, and only updating the parameters related to the adjustable module.

[0037] Furthermore, the smoothing operation described in step 5-1 includes:

[0038] That is, the label bit corresponding to each word in the index number y_i is set to: 1-∈, which is used to represent the true probability of the word; ∈ is a constant between (0,1) and is used to assign the probability of the remaining labels. The vector after label smoothing is expressed as follows:

[0039]

[0040] Where D represents the total distance between words; d i is the distance between the word to be predicted and other words in the dictionary.

[0041] Furthermore, the total distance between words described in step 5-1 is calculated as follows:

[0042]

[0043] Where T represents the number of words.

[0044] Beneficial effects:

[0045] The pre-trained LLM can be better adapted to applications in specific industries or application scenarios after fine-tuning; freezing all parameters of LLM and only adjusting the parameters of the added prompt module reduces the number of parameters that need to be adjusted, which not only reduces the number of training samples but also shortens the time for model fine-tuning; allocating weights for label smoothing operations based on the closeness of the reference text can demonstrate the generalization ability of the model; the text generated by the model depends on the previous text, and the loss function gives greater weight to the text that is earlier in time, which is conducive to rapid convergence of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0047] Figure 1 It is a schematic diagram of the model fine-tuning process.

[0048] Figure 2 It is a schematic diagram of the model structure.

[0049] Figure 3 It is a schematic diagram of the fine-tuned model for archiving internal documents and industry data.

[0050] Figure 4 Schematic diagram of building a local knowledge base using archived documents in the fine-tuned model.

[0051] Figure 5 It is a schematic diagram of the local knowledge base and intelligent retrieval system in the fine-tuned model. DETAILED DESCRIPTION

[0052] The reason why the pre-trained model performs poorly in knowledge retrieval and knowledge question-answering in specific industry applications is that the corpus used for pre-training LLM comes from different fields and has no bias towards the industry, which requires fine-tuning it with knowledge corpus focused on the industry. Therefore, the LLM fine-tuning method proposed in the present invention needs to be combined with the industry knowledge base, and the corpus provided by the industry knowledge base is used to fine-tune the LLM, while retaining and making full use of the LLM's own ability to generate fluent and natural knowledge organization that conforms to the user's expression habits.

[0053] Based on the above ideas, the present invention proposes an LLM fine-tuning method that integrates the industry knowledge base and the LLM knowledge organization capability. The fine-tuned LLM can better adapt to industry-oriented knowledge retrieval. Figure 1 As shown, the specific technical solutions of the present invention are as follows:

[0054] First, deploy the industry knowledge base, which requires the knowledge support to be able to find related content based on text, extract text from documents and dynamically update the knowledge base based on text; select the pre-trained LLM, which can generate smooth and natural text content based on the prompt text. The LLM model adopts the transformer structure (reference: Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need [J]. arXiv, 2017. DOI: 10.48550 / arXiv.1706.03762.), which has been widely used in many large language models and has shown good performance and flexible structure.

[0055] Step 1: Add an adjustable module for generating pseudo-prompts to the pre-trained prediction model structure, including a pseudo-prompt vector conversion module ( Figure 2 The P-embedding module in the module is used to convert integers into vectors, and its structure is a two-dimensional matrix), and the pseudo prompt string generation module for each layer of LLM ( Figure 2 The MLP_1, MLP_2, ..., MLP_n modules in the , are multi-layer perceptron network structures, which are used to transform word vectors into new vector representations). Input the query text, the knowledge base obtains the related content according to the query text, and selects the top K query results with the closest correlation. The K value is an empirical parameter and can be adjusted according to the retrieval performance of the knowledge base.

[0056] Step 2: Let the user query text be x, and the returned multiple query results are merged into a query result y. The query text and the result together constitute a training corpus record (x, y), where x is the input of the LLM encoder and y is the input of the decoder, and is also the reference text for the prediction result. Before x and y are sent to the LLM model, they are first segmented using the word segmentation tool to convert the text sequence into a word index sequence (x_i, y_i). The sequence elements are integers, x_i is the index number of the word in the prompt text in the dictionary (in natural language processing, a word dictionary is constructed to index the words in the sentence, thereby converting the text into a numerical value, and then into a vector for calculation), and y_i is the index number of the word in the reference text in the dictionary.

[0057] Step 3: Construct a pseudo-prompt sequence string p_i. p_i is a discrete integer sequence, which is similar to the query prompt, but its value does not point to a word with specific semantics. Instead, it serves as a guiding value for forward calculation. p_i is an integer sequence generated by a random function and is a constant during model training and prediction.

[0058] Step 4. The p_i sequence is transformed into a vector sequence p_v through the pseudo-prompt P-embedding module, that is, the word mapping matrix. The function of this matrix is ​​the same as that of word mapping to vector, except that the word vector mapping matrix is ​​obtained through training (usually initialized with random quantities and parameters are continuously updated during model iterations) instead of the word mapping matrix in the LLM model.

[0059] Step 5. (x_i, y_i) is mapped to a vector sequence (x_v, y_v) through the embedding layer of LLM; the pseudo-prompt vector sequence p_v is concatenated with the x_v vector sequence to form the input prompt [p_v, x_v]. In the forward propagation, [p_v, x_v] is calculated through the multi-layer perceptron model to obtain a new vector sequence, which is concatenated with the x_v vector as the input of the encoder in the current layer of the LLM model. Each layer of the LLM model input has a corresponding perceptron model and uses the same operation. These perceptron model parameters are parameters that can be modified during model fine-tuning, while the parameters of the LLM model are not updated.

[0060] Step 6. The y_i generated in step 2 is used as the reference label. The reference label is smoothed before calculating the loss. Each word in y_i has a value of 1-∈, which is used to represent the true probability of the word; ∈ is a value between (0,1) and is used to allocate the probability of the remaining labels. When allocating the probability of the remaining labels, it is allocated according to the distance between the word vector and the word vector in the dictionary. The closer the distance, the greater the weight assigned. The vector representation after label smoothing is:

[0061]

[0062] in d i is the distance between the word vector to be predicted and other word vectors i in the dictionary.

[0063] Step 7: Take y_i as the predicted output and use the autoregressive model for training, that is, calculate the loss of word prediction for each label corresponding to y_i. The loss of word prediction is measured using the KL distance: Where L represents the number of words, j represents the jth element value of the probability vector, Q is the reference probability vector after label smoothing, and P is the predicted probability vector.

[0064] Step 8. Calculate the total loss of y_i for word sequence prediction. The loss of a word is inversely proportional to the time sequence. The larger the loss weight of the word with the smallest t, the greater the total loss:

[0065] λ is the discount value of (0,1), t represents the position of the word in the y sequence, the first word is 0, and the last word is T.

[0066] Step 9: Use the gradient descent method to update the prompt parameters of the model. Freeze all LLM parameters during gradient backpropagation and only update the model parameters related to p, that is, the embedding parameters of p_i and the multilayer perceptron model parameters corresponding to p at each layer.

[0067] Step 10: After the model is stable after multiple iterations, use the prediction results of LLM as retrieval prompts to retrieve related knowledge z from RAG, and combine it with x in step 2 (x, z) as a new training corpus record, and repeat steps 2 to 9.

[0068] like Figure 2 As shown, it is the model structure involved in the present invention. The gray background modules represent the modules that need parameter learning, and the other modules represent the modules whose parameters are frozen and not adjusted but participate in the calculation.

[0069] Example:

[0070] In order to make the pre-trained LLM better adapt to industry applications, the model needs to be fine-tuned, which requires the support of the industry knowledge base. Taking the power supply industry as an example, combined with the attached drawings, the specific implementation plan is described in detail according to the technical solution.

[0071] According to the processing flow, the retrieval-augmented generation system (RAG) needs to be deployed in the user's working environment. RAG is an artificial intelligence technology that combines information retrieval technology with language generation models. It is based on the existing large-scale language model and enhances the model's capabilities by accessing an external knowledge base, so that the model can generate more accurate and context-sensitive responses. The RAG system can be divided into a knowledge base subsystem and a knowledge retrieval subsystem. The data used to build the local knowledge base are firstly knowledge documents from the industry field, such as basic knowledge of the power grid, such as the composition, structure, and operation principle of the power transmission and distribution system, power supply industry standards and specifications, power supply equipment and technology, etc.; secondly, work documents, reports, logs, etc. from within the organization; thirdly, production data; and fourthly, structured data from business systems. The knowledge base system must have the ability to dynamically update knowledge, because local documents and materials will continue to accumulate, and the knowledge contained in these documents needs to be dynamically added to the knowledge base. The knowledge base can be stored in a graph database, such as Neo4j. Entities and relationships are extracted from documents using manual and auxiliary tools, and stored in a graph database to form a local knowledge graph. During interactive question-and-answer sessions, the question text is converted into a graph query language, such as Cypher, and relevant information is retrieved from the knowledge graph. A vector database, such as Faiss, can also be used to store text and build a knowledge vector library. The document can be divided into multiple text segments by paragraph or sentence, and the segmented segments can be vectorized and stored in the knowledge base. During interactive question-and-answer sessions, the question text can be converted into a vector, and the vector can be used to retrieve the related text paragraphs.

[0072] like Figure 3 As shown in Figure 1, the system functions of archiving internal documents and industry data by applying the fine-tuned model; Figure 4 The above demonstrates the system's ability to build a local knowledge base using archived documents.

[0073] The text is first segmented and transformed into a word sequence, and then transformed into a word index sequence according to the LLM dictionary. The prompt file is combined with the text retrieved from RAG as the model fine-tuning corpus record, the input is used as the input of the LLM encoding, and the retrieval results are used as the input and reference text of the LLM solver.

[0074] The probability ∈ assigned to the remaining words during the label smoothing operation after quantization of the reference text is set to 0.2. The word vectorization uses the word embedding table of the LLM model for vectorization, and the distance between word vectors uses the cosine distance:

[0075]

[0076] The vectors corresponding to the words A and B, i represents the i-th element of the vector, and n represents the length of the word vector, that is, the number of vector elements.

[0077] The discount λ is taken as 0.99 when calculating the total loss.

[0078] like Figure 5 As shown in FIG. 1 , the system function of combining the local knowledge base and the fine-tuned LLM model for intelligent knowledge retrieval and document query is shown.

[0079] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of the large language model fine-tuning method for knowledge retrieval in the power supply industry provided by the present invention and some or all of the steps in each embodiment can be run. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0080] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of computer programs, i.e., software products, which can be stored in a storage medium and include several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0081] The present invention provides a method and idea for fine-tuning a large language model for knowledge retrieval in the power supply industry. There are many methods and approaches to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A large language model fine-tuning method for knowledge retrieval in the power supply industry, characterized in that: The following steps are involved: Step 1: deploy the RAG system in the working environment of the power supply industry; Step 2, input the query text, and the knowledge base of the RAG system returns multiple query results; Step 3, constructing a training corpus record using the query text and query results, and performing preprocessing to obtain a word index sequence; Step 4: construct a pseudo prompt sequence string and add an adjustable module to the large language model LLM; Step 5, using the word index sequence, training and optimizing the large language model LLM after adding the adjustable module, and completing the fine-tuning of the large language model for knowledge retrieval in the power supply industry after repeated iterations.

2. According to claim 1, a large language model fine-tuning method for knowledge retrieval in the power supply industry is characterized in that: The step of obtaining the word index sequence described in step 3 includes: Step 3-1, let the query text be x, and the text after the corresponding multiple query results are merged, that is, the reference text, be y, and combine to obtain a training corpus record (x, y); Step 3-2, use the word segmentation tool to convert the training corpus record (x, y) into a word index sequence (x_i, y_i), where the sequence elements are discrete integers, x_i is the index number of the word in the prompt text in the dictionary, and y_i is the index number of the word in the reference text in the dictionary.

3. According to claim 2, a large language model fine-tuning method for knowledge retrieval in the power supply industry is characterized in that: The construction of the pseudo prompt sequence string described in step 4 includes: The pseudo prompt sequence string p_i is a discrete integer sequence generated by a random function.

4. The large language model fine-tuning method for power supply industry knowledge retrieval according to claim 3 is characterized in that: Adding adjustable modules to the large language model LLM described in step 4 includes: Pseudo-prompt vector conversion module and pseudo-prompt string generation module; wherein, The structure of the pseudo-prompt vector conversion module is a two-dimensional matrix, which is used to convert integers into vectors as the first layer of the adjustable module; The number of the pseudo-prompt string generation modules is consistent with the number of layers of the large language model LLM. Multiple pseudo-prompt string generation modules form a layered structure and are sequentially arranged after the pseudo-prompt vector conversion module. Their outputs are respectively used as the input of each layer of the encoder in the large language model LLM.

5. The large language model fine-tuning method for power supply industry knowledge retrieval according to claim 4 is characterized in that: The pseudo prompt string generation module described in step 4 adopts a multi-layer perceptron model.

6. The large language model fine-tuning method for knowledge retrieval in the power supply industry according to claim 5 is characterized in that: The pseudo prompt vector conversion module described in step 4 converts the pseudo prompt sequence string p_i into a pseudo prompt vector p_v; The word index sequence (x_i, y_i) is mapped to a word index vector (x_v, y_v) through the embedding layer in the large language model LLM, where x_v is the query vector and y_v is the result vector; The pseudo-prompt vector p_v is concatenated with the query vector x_v to obtain the input prompt vector [p_v, x_v]; The input prompt vector [p_v, x_v] is calculated by the pseudo prompt string generation module to obtain a new vector, which is concatenated with the query vector x_v and used as the input of the encoder of the current layer in the large language model LLM.

7. The large language model fine-tuning method for power supply industry knowledge retrieval according to claim 6 is characterized in that: The step 5 described in which the large language model LLM after adding the adjustable module is trained and optimized, that is, the parameters of the large language model LLM are fixed, and the parameters in the adjustable module are adjusted during the training process.

8. The large language model fine-tuning method for power supply industry knowledge retrieval according to claim 7 is characterized in that: The training and optimization of the large language model LLM after adding the adjustable module described in step 5 specifically includes: Step 5-1, taking the index number y_i of the word in the reference text described in step 3-2 in the dictionary as a reference label, and performing a smoothing operation on it; Step 5-2, the index number y_i after the smoothing operation is used as the output of the large language model LLM after adding the adjustable module, that is, the prediction result, and the autoregressive mode is used for training, that is, the loss of the prediction result is calculated according to the reference label one by one, where the loss function loss is measured by the KL distance, which is expressed as follows: Where L represents the number of words, Q(j) represents the jth element value of the probability vector, Q is the probability vector, and P is the predicted probability vector; Step 5-3, calculate the total loss LOSS, as follows: Among them, λ is the discount value, t represents the word y t The position in the sequence y, the first word is numbered 0, and the last word is numbered T; Step 5-4, using the gradient descent method to update the large language model LLM after adding the adjustable module, that is, freezing all parameters in the large language model LLM during gradient back propagation, and only updating the parameters related to the adjustable module.

9. The large language model fine-tuning method for knowledge retrieval in the power supply industry according to claim 8 is characterized in that: The smoothing operation described in step 5-1 includes: That is, the label bit corresponding to each word in the index number y_i is set to: 1-∈, which is used to represent the true probability of the word; ∈ is a constant between (0,1) and is used to assign the probability of the remaining labels. The vector after label smoothing is expressed as follows: Where D represents the total distance between words; d i is the distance between the word to be predicted and other words in the dictionary.

10. The large language model fine-tuning method for power supply industry knowledge retrieval according to claim 9 is characterized in that: The total distance between words described in step 5-1 is calculated as follows: Where T represents the number of words.

Images