Pure text application-oriented large language model fine tuning training method and related device

Abstract

The invention discloses a plain text application-oriented large language model fine tuning training method and a related device, and the method comprises the steps: obtaining a vertical field data set corresponding to a target vertical field, and carrying out the arrangement in a data set format required by the fine tuning of a large language model; decomposing a weight matrix of the pre-training model into a directivity matrix and an amplitude vector, and quantifying the directivity matrix and the amplitude vector; a rank square root method is used in the forward propagation process of model fine tuning training, the gradient collapse problem under the condition that the matrix rank is large is relieved, meanwhile, the matrix learning rate ratio in the back propagation process is adjusted, and the adaptability of the model to input feature changes and output feature generation tasks is improved; and finally, calculating gradients of the loss function for the matrix and the amplitude vector, and respectively updating the gradients to obtain a final fine tuning model. According to the method, the fine adjustment efficiency and precision of the model are improved on the premise of using small calculation overhead, and the method can be applied to various large language models and the vertical field and has good generalization.

Description

Technical Field

[0001] The present application belongs to the technical field of natural language processing text generation, and relates to a large language model fine-tuning training method and related devices for plain text applications. Background Art

[0002] In the field of natural language processing, text generation technology has become a key component in numerous applications, encompassing tasks such as dialogue systems, automated writing, and machine translation. In recent years, with the development of large language models, deep learning-based text generation technology has made significant progress. Trained with massive amounts of text data, large language models can generate fluent and coherent text content and demonstrate superior language understanding and generation capabilities. However, directly applying general large language models to specific domains (such as power system knowledge question answering) often faces challenges such as insufficient domain knowledge and inaccurate terminology. In these situations, model fine-tuning technology becomes a key solution. By applying a pre-trained model to a dataset relevant to a specific task, fine-tuning allows the model to shift from general knowledge to domain-specific knowledge, improving its performance in specific contexts. Model fine-tuning is not only a transfer learning method but also a key technology for integrating large-scale pre-trained models with specific applications.

[0003] While model fine-tuning techniques can significantly improve model performance on specific tasks, directly fine-tuning large language models faces significant computational and storage overhead, especially when the model parameters are extremely large. Low-rank adaptation (LoRA) has emerged to address this issue. Low-rank adaptation (LoRA), a classic low-rank adaptation method, inserts low-rank adapters into the weight matrix of a pre-trained model, enabling efficient fine-tuning of large models without significantly increasing the number of parameters. By confining model parameter changes to a low-rank subspace, LoRA significantly reduces computational resource requirements while maintaining high performance. To further improve fine-tuning efficiency, the DyLoRA method improves on LoRA. DyLoRA dynamically adjusts the model's rank during training, enabling the model to dynamically adapt to different rank values ​​during inference based on the requirements of different tasks, eliminating the need to search for the optimal rank. This not only further reduces training time but also improves the model's flexibility and adaptability. However, these methods cannot adapt to changes in input features and output feature generation during the training and fine-tuning of large language models, and may also lead to gradient collapse when the matrix rank is large.

[0004] Chinese patent CN118395955A discloses an improved fine-tuning method based on a large language model (LLM), specifically for sample feature completion scenarios. This method converts sample features into text, generates a prompt template, and then fine-tunes the LLM to enable it to predict missing feature values ​​based on the input text, thereby improving the model's performance in feature completion tasks. However, this method still suffers from high computational overhead and lacks the ability to flexibly adjust the rank based on task requirements, resulting in limited performance in some complex tasks. Summary of the Invention

[0005] The purpose of this application is to solve the gradient collapse problem that may occur in high-rank situations in the prior art, and to provide a large language model fine-tuning training method and related devices for plain text applications. This method decomposes the weight matrix of the pre-trained model into a directional matrix and an amplitude vector, and by assigning different learning rates to the decomposed low-rank matrix, the model can better adapt to changes in input features and the generation of output features. At the same time, the rank square root method is introduced in the weight decomposition process, so that the model can still maintain training stability when the rank is large.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions: In the first aspect, this application proposes a large language model fine-tuning training method for plain text applications, comprising the following steps: Obtain pre-trained models and weight matrices of large language models, as well as fine-tuning datasets built for the power sector; Decompose the pre-trained weight matrix into a directionality matrix and a magnitude vector; Quantize the directional matrix and amplitude vector to obtain a quantized large language model; The fine-tuning dataset is input into the quantized large language model and the rank square root factor is introduced for incremental fine-tuning to obtain the prediction score of each word in the fine-tuning dataset and the weight update of the large language model after incremental fine-tuning. Perform a softmax operation on the predicted scores and use the loss function to calculate the difference between the probability distribution and the target distribution of the fine-tuning dataset to obtain the loss; Combine the losses and use the backpropagation algorithm to update the magnitude vector and the weight update matrix; when updating the weight update matrix, a fixed learning rate ratio is set for the matrix; The updated amplitude vector is used to obtain a new quantized large language model. The steps of incremental fine-tuning, calculating the cross-entropy loss, and updating the amplitude vector and weight update matrix are repeated for training until the preset number of training rounds is reached or the stopping condition is met, thereby obtaining a fine-tuned large language model for the power field.

[0007] Furthermore, the decomposition of the pre-trained weight matrix into a directional matrix and an amplitude vector includes:

[0008] in, represents the pre-trained weight matrix, represents the magnitude vector, represents the directivity matrix, express The norm across columns, Indicates the number of columns of the matrix.

[0009] Furthermore, the quantization processing of the directivity matrix and the amplitude vector includes: The directional matrix Quantize the matrix to 8-bit precision ; The magnitude vector Vector quantized to 4 bits of precision .

[0010] Furthermore, the weight update as follows:

[0011] in, Express Incremental direction update, is the hyperparameter in the model training process, is the directional matrix The rank of the matrix and matrix Yes Low-rank decomposition of , and , Represents the number of rows of the original matrix or the dimension of the input features, Indicates the number of columns of the original matrix or the dimension of the output features, Represents the norm of the matrix across its columns.

[0012] Furthermore, the loss function is used to calculate the difference between the probability distribution and the target distribution of the fine-tuning dataset to obtain the loss, including: The loss function adopts the cross entropy loss function. For each time step, the cross entropy loss function The calculation is as follows:

[0013] in, is the true distribution of the target word, is the model word The predicted probability of is the size of the vocabulary.

[0014] Furthermore, the updating of the magnitude vector and the weight updating matrix using the back propagation algorithm includes: (1) Calculate cross entropy loss About Matrix Gradient and about the matrix Gradient :

[0015]

[0016] in, represents the identity matrix, express The transposed matrix of and The matrices and matrix The transposed matrix of is the cross entropy loss About weight update gradient; (2) For the matrix The learning rate or matrix The learning rate Use the learning rate optimization algorithm to update the amplitude vector ,matrix and matrix :

[0017]

[0018]

[0019] in, Express m Updates, express about The gradient, , Express A Updates, Express B Update of ; where the matrix and matrix The setting has a fixed learning rate ratio.

[0020] Furthermore, the steps of repeating incremental fine-tuning, calculating cross entropy loss, and updating the magnitude vector and weight update matrix for training are as follows: , , Then repeat the steps of incremental fine-tuning, calculating the cross entropy loss, and updating the magnitude vector and weight update matrix; The stopping condition is that the loss value during the model fine-tuning process drops below 0.2.

[0021] In a second aspect, the present application proposes an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0022] In a third aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0023] In a fourth aspect, the present application proposes a computer program product, which includes computer instructions. A processor of a computer device reads the computer instructions, and the processor of the computer device executes the computer instructions to implement the steps of the above method.

[0024] Compared with the prior art, this application has the following beneficial effects: This application proposes a large language model fine-tuning method based on improved weight decomposition. By decomposing the weight matrix into a directional matrix and an amplitude vector, the rank square root factor and the A and B matrix learning rate ratio are introduced in the forward propagation and backpropagation processes of the model fine-tuning, respectively. This not only improves the model's adaptability to input feature changes and output feature generation tasks, but also effectively alleviates the gradient collapse problem when the matrix rank is large, thereby improving the model's fine-tuning efficiency and accuracy. In view of the high computing resource requirements required by large language models, the model fine-tuning method designed in this application not only applies low-rank adaptation technology to improve adaptation efficiency with less resource overhead, but also uses model quantization technology to reduce computer memory requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1This is a flowchart of the large language model fine-tuning training method for plain text applications in this application.

[0027] Figure 2 This is a schematic diagram of the principle of the large language model fine-tuning training method for plain text applications in this application.

[0028] Figure 3 This is a flow chart of an embodiment of the present application. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0030] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0031] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0032] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or the orientation or position relationship in which the product of the invention is usually placed when in use. This is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, it should not be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0033] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0034] In the description of the embodiments of this application, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in this application based on specific circumstances.

[0035] The present application is described in further detail below with reference to the accompanying drawings: See also Figure 1 The present application discloses a method for fine-tuning a large language model for plain text applications, comprising the following steps: S1 obtains the pre-trained model and pre-trained weight matrix of the large language model and the fine-tuning dataset built for the power industry; In practical applications, fine-tuning the dataset Include Group data, with instructions and output The format appears in pairs, such as .

[0036] In a feasible embodiment, the pre-trained models include various large language models such as Chatglm3, Baichuan, Qwen and Glm-4, and include models with different parameter amounts such as 6B, 7B, 13B and 65B.

[0037] In practical applications, the fine-tuning dataset includes general question-and-answer data in general fields, professional terminology and explanation data for the power field, specific question-and-answer data for the power field, and solution data for specific power tasks.

[0038] S2 decomposes the pre-trained weight matrix into a directional matrix and an amplitude vector; In practical applications, the pre-trained weight matrix Decomposed into a directional matrix and magnitude vector ,Right now ,in express The norm across columns, Indicates the number of columns of the matrix; S3 quantizes the directional matrix and amplitude vector to obtain a quantized large language model; In practical applications, the directional matrix and magnitude vector Quantized to 8-bit or 4-bit precision, respectively and , and get the quantized large language model ; S4 inputs the fine-tuning dataset into the quantized large language model for incremental fine-tuning to obtain the prediction score of each word in the fine-tuning dataset and the weight update of the large language model after incremental fine-tuning; In practical applications, the dataset will be fine-tuned Input to the quantized large language model Perform incremental fine-tuning, passing through the word embedding layer, model layer and output layer in sequence; the word embedding layer is used to fine-tune the dataset The data in the word vector is encoded; the model layer includes a multi-head self-attention mechanism, a feedforward network and layer normalization; the output layer, that is, the fully connected layer, is used to output the fine-tuning dataset Prediction score for each word in ; large language model after incremental fine-tuning The weight is updated to , such as:

[0039] Where: Express Incremental direction update, is the hyperparameter in the model training process, is the directional matrix The rank of the matrix and Yes Low-rank decomposition of , and , Represents the norm of the matrix across columns; S5 performs a softmax operation on the predicted scores and uses the loss function to calculate the difference between the probability distribution and the target distribution of the fine-tuning dataset to obtain the loss; In practical applications, the loss function can use the cross entropy loss function.

[0040] S6 combines the losses and uses the backpropagation algorithm to update the magnitude vector and the weight update matrix; S6-1, Calculation of Losses About Matrix and Gradient and :

[0041] Where: represents the identity matrix, express The transposed matrix of and The matrices and The transposed matrix of for about gradient; S6-2, yes or Matrix learning rate or Use a learning rate optimization algorithm, such as Use, then let ( is a fixed learning rate ratio), and vice versa; let , use the following formula to update the amplitude vector ,matrix and matrix :

[0042] In one possible embodiment, the learning rate ratio The setting is , and in the process of model fine-tuning training, They are 2, 4, 8, and 16 respectively.

[0043] S7 uses the updated amplitude vector to obtain a new quantized large language model, and repeats the steps of incremental fine-tuning, calculating the cross-entropy loss, and updating the amplitude vector and weight update matrix for training until the preset number of training rounds is reached or the stopping condition is met, thereby obtaining a fine-tuned large language model for the power field.

[0044] In practical applications, , , , repeat steps S4-S6 until the preset number of training rounds is reached or the stopping condition is met, and a fine-tuned large language model for the power field is obtained; In practical applications, the stopping condition is that the loss value during model fine-tuning drops below 0.2.

[0045] In a feasible embodiment, the algorithm evaluation index is used to set different learning rate ratios. The model with the highest score is selected Values ​​as matrices in model backpropagation and matrix The learning rate ratio is set to obtain the final fine-tuned large language model for the power field; In practical applications, algorithm evaluation indicators include: BLEU-4: Based on the precision ratio, it is a method for automatically evaluating the similarity between generated text and reference text. 4 represents the exact match of 4 consecutive words. Rouge series metrics: Based on the recall rate design, the method uses Rouge-1 to measure the degree of overlap between the generated summary and the reference summary of a single word, Rouge-2 to measure the degree of overlap between the generated summary and two adjacent words in the reference summary, and Rouge-L to measure the similarity between the generated summary and the reference summary using the longest common subsequence; In actual applications, the ratio between the overall data consisting of professional terminology and explanation data for the power field, specific question and answer data for the power field, and solution data for specific power tasks and the general question and answer data in the general field is 1:4.

[0046] If it is determined that the prediction result does not meet the predetermined conditions, the model strategy is updated based on the feedback of the prediction result to obtain an updated large language model, and the updated large language model is determined as the new current large language model, and the data set is input into the current large language model to obtain the prediction result and the corresponding steps thereafter are iteratively executed; wherein, if it is determined that the prediction result meets the predetermined conditions, the training is stopped and the current large language model is determined as the trained large language model.

[0047] Example like Figure 2 and Figure 3 As shown, the embodiment of the present application discloses a large language model fine-tuning training method for plain text applications, comprising the following steps: S101: Create a large language model fine-tuning dataset for the power sector, which includes three related datasets: (1) 8062 power sector related terms and their definitions were extracted from the China National Knowledge Infrastructure (CNKI) Chinese Standard Terminology website (https: / / shuyu.cnki.net / index.aspx). After data processing, the first fine-tuning dataset (shuyu) was created; (2) 3500 data items were reorganized through manual screening and revision as the second fine-tuning dataset (EE); (3) Question-and-answer data were extracted from power professional books such as the "Electrical Engineer's Handbook" and "Questions and Answers on Practical Technology for Power Grid Regulation and Operation" or power sector related websites to construct the third fine-tuning dataset (wenda). Some examples of the above datasets are shown in Table 1.

[0048] Table 1 Some examples of three datasets

[0049] S102: Convert the dataset compiled in S101 into the dataset format required for large language model training and fine-tuning. The instruction format is as follows: {"instruction": "Instruction content", "response": "output content"}, The input JSON file for fine-tuning the large language model using some examples from the dataset in Table 1 is: [ {"instruction": "coupling coefficient", "response": "The ratio of the mutual inductance between two circuits to the geometric mean of the inductances of the two circuits."}, {"instruction": "Transmission line", "response": "Transmission lines are specially designed for long-distance electrical energy transmission within power systems. They typically connect power plants to remote substations, transporting the electricity generated by the power plants to where it's needed. Transmission lines are characterized by their high voltage levels to reduce energy losses during transmission. These lines typically use high voltage transmission, effectively covering long distances while maintaining transmission efficiency."}, {"instruction": "What factors affect system voltage?", "response": "System voltage is determined by the system power flow distribution. The main factors affecting it are: (1) load changes caused by production, life, meteorological factors, etc.; (2) changes in reactive power compensation capacity; (3) changes in power distribution and network impedance caused by changes in system operation mode; (4) the reactive power output of the grid-connected generator sets; (5) the impact of impact loads or unbalanced loads."}, ······ ] S103: Available large language pre-training models include: Chatglm3, Baichuan, Qwen, and Glm-4, and their existing 6B, 7B, 13B, and 65B versions of different parameter sizes. In this example, the GLM-9B-Chat model is used as the pre-training model, and the corresponding pre-training weight matrix is ​​obtained. . The weight matrix Decomposed into a directional matrix and magnitude vector ,Right now , and use pre-trained weights Initialize them separately, let , ,in Representation matrix The norm across columns, Indicates the number of columns of the matrix. Then the directional matrix and magnitude vector Quantized to 8-bit or 4-bit precision, respectively and , and get the quantized large language model .

[0050] S104: Next, the three data sets in S102 (shuyu, EE, wenda) are input into the quantized large language model Perform incremental fine-tuning, passing through the word embedding layer, model layer and output layer in sequence; the word embedding layer is used to fine-tune the dataset The data in the word vector is encoded and converted into a fixed-dimensional vector representation; the model layer (usually using the Transformer layer) includes a multi-head self-attention mechanism, a feedforward network, and layer normalization. Through these layers, the model gradually captures contextual information and outputs the encoded representation of the current word at each position: ; After processing by the model layer, the implicit representation of each time step is obtained , these representations will be passed to the output layer, that is, the fully connected layer, and the fine-tuning dataset will be output The predicted score (logits) of each word in ; the large language model after incremental fine-tuning The weight is updated to , such as:

[0051] Where: Express Incremental direction update, is the hyperparameter in the model training process, is the directional matrix The rank of the matrix and Yes Low-rank decomposition of , and , Represents the norm of the matrix across its columns.

[0052] S105: Perform softmax operation on the logits of each time step: , converting the predicted scores into probability distributions. Softmax ensures that the predicted probability values ​​are non-negative and sum to 1. To measure the difference between the word distribution predicted by the model and the true word label, this example uses the cross-entropy loss function to calculate the difference between the probability distribution and the target distribution of the fine-tuning dataset E to obtain the cross-entropy loss. For each time step, the cross entropy loss function is calculated as follows:

[0053] in, is the true distribution of the target word (usually a one-hot vector, with the correct word position being 1 and the rest being 0), is the model word The predicted probability of is the size of the vocabulary.

[0054] S106: Back propagation of the GLM-9B-Chat model 、 The matrix is ​​updated using different learning rate optimization algorithms, setting a fixed learning rate ratio , and only adjust (or ) to update two matrices simultaneously, such as Using the learning rate optimization algorithm, let , and vice versa. These ratios grow as powers of 2, creating a logarithmic search space. This allows for exploration over a wider range with fewer points, allowing us to observe the effects of both smaller and larger ratios. Furthermore, using powers of 2 exploits the binary nature of computers, improving computational efficiency.

[0055] S107: Calculating the cross entropy loss function during the backpropagation of the GLM-9B-Chat model about and The gradient of is:

[0056] Where: for about The gradient, for about The gradient, for about The gradient, represents the identity matrix, express The transposed matrix of calculate About Matrix and Gradient:

[0057] Where: for about The gradient, for about The gradient, and The matrices and The transposed matrix of Calculating the loss function About Matrix and Gradient:

[0058] Update the magnitude vector , low-rank decomposition matrix and :

[0059] S108: Order , , Repeat S104 to S107 until the preset number of training rounds is reached or the stopping condition is met (the loss value during the model fine-tuning process drops below 0.2), and a fine-tuned large language model for the power field is obtained; S108: Use BLEU (Bilingual Evaluation Understudy) and ROUGE series indicators to evaluate the model.

[0060] When using the BLUE indicator, first get the formula for calculating n-gram

[0061] In the above formula, express A set of phrases of length words, Indicates the Phrases in translation The number of times it appears in Indicates the Phrases in manual translation The number of times it appears in .

[0062] The n-gram matching may become better as the sentence length becomes shorter. To avoid this phenomenon, BLEU introduces a length penalty factor BP (Brevity Penalty) in the final scoring result.

[0063]

[0064] in, Indicates the length of the machine translation. Indicates the effective length of the reference translation.

[0065] In order to balance the effects of statistics of each order, the statistics of each order are weighted and summed, and the final formula is as follows:

[0066] Generally speaking, if N is 4, only the 4-gram accuracy is counted, thus obtaining the first evaluation metric used in this paper, BLUE-4. Other evaluation metrics used in this paper include the ROUGE series, which are mainly designed based on recall: (1) Rouge-1: It measures the degree of overlap between the generated summary and the reference summary of a single word; (2) Rouge-2: It measures the degree of overlap between the generated summary and the reference summary of two adjacent words; (3) Rouge-L: It uses the longest common subsequence (LCS) to measure the similarity between the generated summary and the reference summary.

[0067] S108: The three datasets were merged and input into the GLM-9B-Chat model fine-tuned using the low-rank decomposition method, the weighted decomposition method, and the improved weighted decomposition method. The model was evaluated using the evaluation metrics described in S107, and the evaluation results were shown in Table 2. As can be seen from Table 2, the large language model fine-tuning training method for text-only applications proposed in this application achieved the best results in all evaluation metrics.

[0068] Table 2 Evaluation results of three fine-tuning methods

[0069] Table 3 shows the model evaluation results of different learning rate ratios. As shown in Table 3, when the ratio is set to 4 or 8, the various indicators of the model achieve better scores.

[0070] Table 3 Comparison of results of different learning rates

[0071] In summary, this application first decomposes the weight matrix of the pre-trained model into a directional matrix and an amplitude vector; then gives different learning rates to the low-rank matrices B and A decomposed from the directional matrix to adapt to the tasks of input feature changes and output feature generation respectively; secondly, the rank square root method is used to alleviate the gradient collapse problem of the improved Dora method when the matrix rank is large; finally, this article also introduces a quantization method in the model training process to enable fine-tuning of larger language models with smaller video memory. The method proposed in this application has shown good stability and the highest accuracy on models of different sizes and diverse data sets, providing an effective technical reference solution for the future application of large language model fine-tuning technology in professional fields such as power systems.

[0072] One embodiment of the present application provides a computer device. The computer device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of each of the above-described method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in each of the above-described apparatus embodiments are implemented.

[0073] The computer program may be divided into one or more modules / units, and one or more modules / units may be stored in a memory and executed by a processor to complete the present application.

[0074] The computer device may be a desktop computer, a notebook computer, a PDA, a cloud server, etc. The computer device may include, but is not limited to, a processor and a memory.

[0075] The processor can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0076] The memory can be used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0077] If a module / unit integrated into a computer device is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. It should be noted that the content included in a computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunications signals.

[0078] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The methods provided in the various optional methods are therefore not described in detail here.

[0079] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0080] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or portion of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal. It can be implemented in whole or in part using software, hardware (such as processing circuits or memory), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the functionality of the module or unit.

[0081] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0082] The methods and related devices provided in the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided in the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the steps in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device, or are transmitted through a computer-readable storage medium. Computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The instruction device is implemented in the process Figure 1 Schematic diagram of one or more processes and / or structures Figure 1These computer program instructions can also be loaded onto a computer or other programmable device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0083] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0084] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0085] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A large language model fine-tuning training method for plain text applications, characterized in that: The following steps are involved: Obtain pre-trained models, pre-trained weight matrices, and fine-tuned datasets built for the power sector using large language models; Decompose the pre-trained weight matrix into a directional matrix and a magnitude vector; Quantize the directional matrix and amplitude vector to obtain a quantized large language model; The fine-tuning dataset is input into the quantized large language model and the rank square root factor is introduced for incremental fine-tuning to obtain the prediction score of each word in the fine-tuning dataset and the weight update of the large language model after incremental fine-tuning. Perform a softmax operation on the predicted scores and use the loss function to calculate the difference between the probability distribution and the target distribution of the fine-tuning dataset to obtain the loss; Combine the loss and use the back propagation algorithm to update the magnitude vector and the weight update matrix; when updating the weight update matrix, a fixed learning rate ratio is set for the matrix; The updated amplitude vector is used to obtain a new quantized large language model, and the steps of incremental fine-tuning, calculating the cross entropy loss, and updating the amplitude vector and weight update matrix are repeated for training until the preset number of training rounds is reached or the stopping condition is met, thereby obtaining a fine-tuned large language model for the power field.

2. The large language model fine-tuning training method for plain text applications according to claim 1, characterized in that: The decomposing the pre-trained weight matrix into a directional matrix and an amplitude vector comprises: in, represents the pre-trained weight matrix, represents the magnitude vector, represents the directional matrix, express The norm across columns, Indicates the number of columns of the matrix.

3. The large language model fine-tuning training method for plain text applications according to claim 1, characterized in that: The quantization processing of the directivity matrix and the amplitude vector includes: The directional matrix Quantize the matrix to 8 bits of precision ; The magnitude vector Vector quantized to 4 bits of precision .

4. The large language model fine-tuning training method for plain text applications according to claim 3 is characterized in that: The weight update as follows: in, Express Incremental direction update, is the hyperparameter in the model training process, is the directional matrix The rank of the matrix and matrix Yes A low-rank decomposition of , and , Represents the number of rows of the original matrix or the dimension of the input features, Represents the number of columns of the original matrix or the dimension of the output features, Represents the norm of the matrix across its columns.

5. The large language model fine-tuning training method for plain text applications according to claim 4, characterized in that: The loss function is used to calculate the difference between the probability distribution and the target distribution of the fine-tuning dataset to obtain the loss, including: The loss function adopts the cross entropy loss function. For each time step, the cross entropy loss function The calculation is as follows: in, is the true distribution of the target word, is the model pair The predicted probability of is the size of the vocabulary.

6. The large language model fine-tuning training method for plain text applications according to claim 5, characterized in that: The method of updating the magnitude vector and the weight update matrix using the back propagation algorithm includes: (1) Calculate cross entropy loss About Matrix Gradient and about the matrix Gradient : in, represents the identity matrix, express The transposed matrix of and The matrices are and matrix The transposed matrix of is the cross entropy loss About weight update The gradient of (2) For the matrix The learning rate or matrix The learning rate Use the learning rate optimization algorithm to update the magnitude vector ,matrix and matrix : in, Express m Update, express about The gradient of , Express A Update, Express B The update of and matrix The setting has a fixed learning rate ratio.

7. The large language model fine-tuning training method for plain text applications according to claim 6, characterized in that: The steps of repeating incremental fine-tuning, calculating cross entropy loss, and updating the magnitude vector and weight update matrix for training are as follows: , , Then repeat the steps of incremental fine-tuning, calculating the cross entropy loss, and updating the magnitude vector and weight update matrix; The stopping condition is that the loss value during the model fine-tuning process drops below 0.

2.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises computer instructions, a processor of a computer device reads the computer instructions, and the processor of the computer device executes the computer instructions to implement the steps of the method according to any one of claims 1 to 7.

Images