Natural language to low rank adaptation model parameter optimization method

Abstract

The invention relates to the technical field of model parameter optimization, in particular to a natural language-to-low rank adaptation model parameter optimization method, which comprises the following steps of: S1, acquiring a natural language instruction P according to a text input by a user; s2, inputting a natural language instruction P into the large reasoning model to obtain an instruction understanding text U; s3, inputting the instruction understanding text U into the vectorization model to obtain a text vector E; s4, constructing a LoRA parameter generation model and a loss function based on an attention activation feedback weight mechanism; according to the natural language to low rank adaptive model parameter optimization method, by introducing an attention activation feedback weight mechanism, the defects that all LoRA weight parameters are regarded as equal importance and attention activation weight distribution characteristics are ignored in an existing method are avoided, training can be more focused on key model parameter learning, the over-fitting phenomenon is reduced, and the training efficiency is improved. And the generalization performance and the overall generation effect of the natural language-to-low rank adaptation model parameters are improved.

Description

Technical Field

[0001] This invention relates to the field of model parameter optimization technology, specifically to a method for optimizing parameters in a natural language to low-rank adaptation model. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) are increasingly widely used in natural language processing, intelligent interaction and other fields. However, the traditional LLM fine-tuning process faces significant bottlenecks: for example, to adapt to each new task, a large amount of computing resources and time are required for full parameter optimization, which makes LLM costly in multi-scenario, large-scale deployment and difficult to meet the needs of practical applications.

[0003] To address these issues, efficient parameter fine-tuning methods have emerged, with Low-Rank Adaptation (LoRA) being the most representative. LoRA simulates model weight updates by freezing the parameters of the LLM backbone network and training only the low-rank matrix, significantly reducing storage and computation costs. However, even so, LoRA still requires a separate optimization process for each new dataset or task. When faced with hundreds or even thousands of heterogeneous workloads, this "task-based training" mode is extremely inefficient, and deployment remains challenging.

[0004] Specifically, this manifests in the following ways: First, the training dataset has limited diversity, making it difficult to cover all complex task scenarios; second, the training methods have defects: for example, existing methods use the simple mean squared error (MSE) loss function, treating all LoRA weight parameters as equally important, ignoring the weight importance distribution characteristics determined by the attention activation process during model inference, failing to focus on learning key weights, easily leading to overfitting, and ultimately resulting in poor model adaptability in unseen tasks or complex scenarios.

[0005] Therefore, how to overcome the limitations of dataset diversity and training methods and improve the generalization performance of natural language processing model parameters has become a highly challenging and significant issue in the field of efficient deployment of LLM and optimization of intelligent interactive systems. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method for optimizing parameters of a natural language to low-rank adaptation model, which solves the technical problems of existing natural language to model parameter methods, such as difficulty in focusing on key weight learning, easy overfitting, and insufficient generalization ability.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for optimizing parameters of a natural language to low-rank adaptation model, the method comprising the following steps:

[0008] S1. Obtain natural language instructions P based on user input text;

[0009] S2. Input the natural language instruction P into the inference model to obtain the instruction understanding text U;

[0010] S3. Input the instruction understanding text U into the vectorization model to obtain the text vector E;

[0011] S4. Constructing the LoRA parameter generation model and the loss function based on the attention activation feedback weight mechanism. Create a LoRA fine-tuning training dataset and perform LoRA fine-tuning training to obtain the fine-tuned LoRA weights. Then, construct a model training dataset for training the LoRA model using the LoRA fine-tuning training dataset, the fine-tuned LoRA weights, and the corresponding instruction understanding text U.

[0012] S5. Input the text vector E into the trained LoRA parameter generation model to obtain the LoRA model parameters.

[0013] Preferably, in step S3, the tensor size of the text vector E is set to (B, N, L, C), where B is the batch size, N is the number of prompts, L is the sequence length, and C is the channel.

[0014] Preferably, in step S4, the specific implementation steps are as follows:

[0015] S41. Create the LoRA fine-tuning training dataset and the LoRA parameter generation model;

[0016] S42. Perform LoRA fine-tuning training using the LoRA fine-tuning training dataset to obtain the fine-tuned LoRA weights;

[0017] S43. Construct a model training dataset for training the LoRA model by fine-tuning the training dataset of LoRA, the fine-tuned LoRA weights and the corresponding instruction-understanding text U.

[0018] S44. Construct a loss function based on an attention activation feedback weight mechanism. ;

[0019] S45. Train the LoRA model using the model training dataset.

[0020] Preferably, in step S41, the specific steps are as follows:

[0021] S411. Select an open-source dataset that includes commonsense reasoning, coding, mathematical, and multimodal tasks;

[0022] S412. Perform semantic deduplication on the data in the open-source dataset to obtain the LoRA fine-tuning training dataset.

[0023] Preferably, in step S45, the specific steps are as follows;

[0024] S451. Input the tensor size (B, N, L, C) of the text vector E into the superconvolution decoder;

[0025] S452. Transpose the tensor size (B,N,L,C) of the text vector E to (B,C,L,N) using a hyperconvolution decoder, perform width convolution calculation, then transpose the tensor size (B,N,L,C) of the text vector E to (B,L,N,C) and perform height convolution calculation to obtain width-first convolution. Breadth-first convolution The calculation formula is as follows:

[0026]

[0027] in, For width convolution, High-level convolution;

[0028] S453. Transpose the tensor dimensions (B, N, L, C) of the text vector E to (B, L, N, C), perform a height convolution, then transpose the tensor dimensions (B, N, L, C) of the text vector E to (B, C, L, N), and perform a width convolution to obtain a height-first convolution. Height-first convolution The calculation formula is as follows:

[0029]

[0030] in, For width convolution, High-level convolution;

[0031] S454, perform width-first convolution. and height-first convolution Input inter-layer convolutions to obtain LoRA weight moments with tensor size (B, Nw, Lw, Cw). LoRA weight matrix The calculation formula is as follows:

[0032]

[0033] in, For inter-layer convolution, This is a learnable bias.

[0034] Preferably, in step S44, the loss function The expression is as follows:

[0035]

[0036]

[0037] in, This represents the attention activation feature weights, which are the weights of the model for the current input problem. The distribution of activation responses Mean square error, This is the activation function.

[0038] The present invention also provides a computer storage medium storing a computer program, wherein when the computer program is executed, it implements the steps of a method for optimizing parameters of a natural language to low-rank adaptation model.

[0039] By employing the above technical solution, the present invention provides a method for optimizing parameters of a natural language to low-rank adaptation model, which has at least the following beneficial effects:

[0040] 1. This method for optimizing parameters in a natural language to low-rank adaptation model introduces an attention activation feedback weight mechanism. By incorporating the parameter activation distribution during model inference into the mean squared error as a loss function in the form of a weighted product, it achieves dynamic weight allocation of model parameters. This avoids the shortcomings of existing methods that treat all LoRA weight parameters as equally important and ignore the characteristics of attention activation weight distribution. It allows training to focus more on learning key model parameters, reduces overfitting, and improves the generalization performance and overall generation effect of the natural language to low-rank adaptation model parameters.

[0041] 2. The method for optimizing parameters of the natural language to low-rank adaptation model involves two aspects in the dataset construction stage. On the one hand, it selects open-source datasets covering various types of tasks, including common sense reasoning, coding, mathematics, and multimodal tasks, as training data for LoRA fine-tuning. Semantic deduplication is performed using a large language model to ensure data diversity and effectiveness. On the other hand, in the parameter generation model training dataset, additional instruction-understanding text obtained through the large inference model is added to enrich the input expression dimension, further reduce the risk of overfitting, and enhance the model's adaptability to different types of tasks.

[0042] 3. This method for optimizing the parameters of the natural language to low-rank adaptation model can be directly applied to intelligent interaction systems. Its LoRA parameter generation model is based on pre-training and can quickly convert natural language instructions into LoRA weights for specific tasks. Unlike traditional LoRA technology, it does not require separate optimization for each new task, which significantly reduces computational resources and time costs. At the same time, the improved model generation effect and generalization ability enable intelligent interaction systems to more accurately understand diverse user instructions and provide high-quality responses, effectively improving the user interaction experience. Attached Figure Description

[0043] The accompanying drawings, which are provided to further illustrate the invention, constitute a part of this application:

[0044] Figure 1This is a flowchart of the natural language to low-rank adaptation model parameter optimization method of the present invention;

[0045] Figure 2 This is a flowchart illustrating the construction of the LoRA parameter generation model and the loss function based on the attention activation feedback weight mechanism of this invention.

[0046] Figure 3 This is a flowchart illustrating the creation of the LoRA fine-tuning training dataset for this invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] In recent years, zero-shot prompting to weights has introduced a paradigm shift, completely eliminating the need for gradient-based optimization. The core of this technique is building a parameter generator with conditionalized prompts, capable of directly synthesizing task-specific LoRA weights from a small number of unlabeled task prompts, without requiring separate adapter training for each task. For example, the "drag-and-drop LLM" scheme proposed by institutions such as the National University of Singapore in the paper "Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights" extracts prompt embeddings through a lightweight text encoder and converts them into a complete LoRA weight matrix using a cascaded hyperconvolutional decoder. This not only reduces the adaptation overhead by up to 12,000 times but also achieves an average performance improvement of 30% compared to traditional LoRA training in commonsense reasoning, mathematics, coding, and multimodal benchmarks. Despite these significant advantages, current solutions are limited by two issues, resulting in insufficient generalization ability.

[0049] Existing natural language processing (NLP) model parameter optimization methods treat all LoRA weights as equally important, ignoring the influence of the attention activation process on the weight importance distribution. This leads to a failure to focus on learning key weights, resulting in overfitting and limited generalization performance. To address this issue, this invention provides a method for optimizing NLP low-rank adaptive model parameters. This method avoids the shortcomings of treating all LoRA weights as equally important and ignoring the attention activation weight distribution characteristics. It allows training to focus more on learning key model parameters, reducing overfitting and improving the generalization performance and overall generation effect of the NLP low-rank adaptive model parameters. Figure 1 As shown, the method includes the following steps:

[0050] S1. Obtain natural language instructions P based on user input text. There are two main ways to obtain them: one is to search for public datasets on relevant topics based on the task scenario to be applied, and the other is to manually organize user input and reference answers that appear in actual business based on specific scenarios. By obtaining natural language instructions P through dual paths, it can cover diverse task scenarios such as mathematical reasoning, programming, and multimodal understanding, and also incorporate user input and reference answers from real business, ensuring the comprehensiveness and practicality of instruction data, and providing a high-quality input foundation for subsequent model training.

[0051] S2. Input the natural language instruction P into the inference model to obtain the instruction understanding text U. The inference model uses mainstream open source models such as QwQ, which can transform broad and fuzzy user input into accurate and structured task descriptions, reducing ambiguity in subsequent vectorization and parameter generation.

[0052] S3. Input the instruction understanding text U into the vectorization model to obtain the text vector E. The vectorization model uses mainstream open-source models such as jina-embeddings. Furthermore, the tensor size of the text vector E is set to (B, N, L, C), where B is the batch size, N is the number of prompts, L is the sequence length, and C is the number of channels. By using the vectorization model, the instruction understanding text U is transformed into a text vector E (B, N, L, C) with defined dimensions. This not only achieves the standardized representation of text information but also provides a standardized input structure for the multi-dimensional feature extraction of the subsequent hyperconvolutional decoder.

[0053] S4. Constructing the LoRA parameter generation model and the loss function based on the attention activation feedback weight mechanism. This paper describes a method for creating a LoRA fine-tuning training dataset and performing LoRA fine-tuning training to obtain fine-tuned LoRA weights. A training dataset for the LoRA model is then constructed using the LoRA fine-tuning training dataset, the fine-tuned LoRA weights, and the corresponding instruction-understanding text U. By introducing an attention activation feedback weight mechanism, the parameter activation distribution during model inference is incorporated into the mean squared error as a loss function in the form of a weighted product. This achieves dynamic weight allocation of model parameters, avoiding the shortcomings of existing methods that treat all LoRA weight parameters as equally important and ignore the characteristics of the attention activation weight distribution. This allows training to focus more on learning key model parameters and reduces overfitting. Figure 2 As shown, the specific implementation steps are as follows:

[0054] S41. Create the LoRA fine-tuning training dataset and the LoRA parameter generation model, such as Figure 3 As shown, the specific steps are as follows:

[0055] S411. Select open-source datasets that include commonsense reasoning, coding, mathematical and multimodal tasks; among them, open-source datasets include MATH, GSM8k, MathV360K, HumanEvalOBQA, PIQA, MathV360K, etc.

[0056] S412. Perform semantic deduplication on the data in the open source dataset to obtain the LoRA fine-tuning training dataset. In this process, open source / closed source large language models are used to perform semantic deduplication on the data in the open source dataset, which effectively ensures the diversity and non-redundancy of the LoRA fine-tuning training dataset.

[0057] S42. Perform LoRA fine-tuning training using the LoRA fine-tuning training dataset to obtain the fine-tuned LoRA weights. Use mainstream open-source models such as the Qwen2.5 series, whose fine-tuning methods are consistent with existing mainstream LoRA training methods, which will not be elaborated here, thereby obtaining a large number of fine-tuned LoRA weights.

[0058] S43. Construct a model training dataset for training the LoRA model by fine-tuning the LoRA training dataset, the fine-tuned LoRA weights, and the corresponding instruction understanding text U. In addition to the original input text in the LoRA fine-tuning training dataset, add the instruction understanding text U obtained through the inference large model, which can enrich the input expression content and reduce the risk of overfitting.

[0059] S44. Construct a loss function based on an attention activation feedback weight mechanism. Using the mean squared error of ESM calculated with attention activation feature weights as the loss function, it can accurately calculate the difference between the attention activation feature weights and the fine-tuned LoRA weights, making model parameter updates more targeted and focusing more on learning key model parameters during training, thereby improving model generation performance. The expression is as follows:

[0060]

[0061]

[0062] in, This represents the attention activation feature weights, which are the weights of the model for the current input problem. The distribution of activation responses Mean square error, For activation functions;

[0063] S45. Train the LoRA model using the model training dataset. The specific steps are as follows:

[0064] S451. Input the tensor dimensions (B, N, L, C) of the text vector E into the hyperconvolution decoder. The hyperconvolution decoder has a total of 5 convolutional layers, including 3 types of convolutional modules, which extract and fuse features for different dimensions: width convolution ( ): operates on the channel and sequence length dimensions; height convolution ( ): operates on sequence length and hint quantity dimensions; inter-layer convolution ( This approach operates on the dimensions of cue quantity and sequence length, using a kernel size of 5 and a stride of 1. Through multi-dimensional convolution and feature fusion using a hyperconvolutional decoder, it comprehensively captures text vector features from dimensions such as channel, sequence length, and cue quantity, generating a LoRA weight matrix for accurate matching tasks. The specific implementation steps are as follows:

[0065] S452. Transpose the tensor size (B,N,L,C) of the text vector E to (B,C,L,N) using a hyperconvolution decoder, perform width convolution calculation, then transpose the tensor size (B,N,L,C) of the text vector E to (B,L,N,C) and perform height convolution calculation to obtain width-first convolution. Breadth-first convolution The calculation formula is as follows:

[0066]

[0067] in, For width convolution, High-level convolution;

[0068] S453. Then, the tensor dimensions (B, N, L, C) of the text vector E are transposed to (B, L, N, C) using a hyperconvolution decoder, and height convolution is performed. Then, the tensor dimensions (B, N, L, C) of the text vector E are transposed to (B, C, L, N) and width convolution is performed to obtain height-first convolution. Height-first convolution The calculation formula is as follows:

[0069]

[0070] in, For width convolution, High-level convolution;

[0071] S454, Finally, the width-first convolution is converted using a superconvolution decoder. and height-first convolution Input inter-layer convolutions to obtain LoRA weight matrices with tensor sizes (B, Nw, Lw, Cw). LoRA weight matrix The calculation formula is as follows:

[0072]

[0073] in, For inter-layer convolution, This is a learnable bias.

[0074] S5. Input the text vector E into the trained LoRA parameter generation model to obtain the LoRA model parameters. This eliminates the need to perform LoRA fine-tuning separately for each new task, significantly reducing computational resources and time costs and improving model deployment efficiency. At the same time, the LoRA model parameters generated in this process can accurately adapt to the task requirements corresponding to user instructions and can be directly applied to intelligent interaction systems. This allows for rapid response to diverse user needs and significantly improves the user interaction experience.

[0075] The present invention also provides a computer storage medium storing a computer program, wherein when the computer program is executed, it implements the steps of a method for optimizing parameters of a natural language to low-rank adaptation model.

[0076] It should be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for optimizing parameters in a natural language to low-rank adaptation model, characterized in that, The method includes the following steps: S1. Obtain natural language instructions P based on user input text; S2. Input the natural language instruction P into the inference model to obtain the instruction understanding text U; S3. Input the instruction understanding text U into the vectorization model to obtain the text vector E; S4. Constructing the LoRA parameter generation model and the loss function based on the attention activation feedback weight mechanism. Create a LoRA fine-tuning training dataset and perform LoRA fine-tuning training to obtain the fine-tuned LoRA weights. Then, construct a model training dataset for training the LoRA model using the LoRA fine-tuning training dataset, the fine-tuned LoRA weights, and the corresponding instruction understanding text U. S5. Input the text vector E into the trained LoRA parameter generation model to obtain the LoRA model parameters.

2. The method for optimizing parameters of a natural language to low-rank adaptation model according to claim 1, characterized in that, In step S3, the tensor size of the text vector E is set to (B, N, L, C), where B is the batch size, N is the number of prompts, L is the sequence length, and C is the channel.

3. The method for optimizing parameters of a natural language to low-rank adaptation model according to claim 1, characterized in that, In step S4, the specific implementation steps are as follows: S41. Create the LoRA fine-tuning training dataset and the LoRA parameter generation model; S42. Perform LoRA fine-tuning training using the LoRA fine-tuning training dataset to obtain the fine-tuned LoRA weights; S43. Construct a model training dataset for training the LoRA model by fine-tuning the training dataset of LoRA, the fine-tuned LoRA weights and the corresponding instruction-understanding text U. S44. Construct a loss function based on an attention activation feedback weight mechanism. ; S45. Train the LoRA model using the model training dataset.

4. The method for optimizing parameters of a natural language to low-rank adaptation model according to claim 3, characterized in that, In step S41, the specific steps are as follows: S411. Select an open-source dataset that includes commonsense reasoning, coding, mathematical, and multimodal tasks; S412. Perform semantic deduplication on the data in the open-source dataset to obtain the LoRA fine-tuning training dataset.

5. The method for optimizing parameters of a natural language to low-rank adaptation model according to claim 3, characterized in that, In step S45, the specific steps are as follows; S451. Input the tensor size (B, N, L, C) of the text vector E into the superconvolution decoder; S452. Transpose the tensor size (B,N,L,C) of the text vector E to (B,C,L,N) using a hyperconvolution decoder, perform width convolution calculation, then transpose the tensor size (B,N,L,C) of the text vector E to (B,L,N,C) and perform height convolution calculation to obtain width-first convolution. Breadth-first convolution The calculation formula is as follows: in, For width convolution, High-level convolution; S453. Transpose the tensor dimensions (B, N, L, C) of the text vector E to (B, L, N, C), perform a height convolution, then transpose the tensor dimensions (B, N, L, C) of the text vector E to (B, C, L, N), and perform a width convolution to obtain a height-first convolution. Height-first convolution The calculation formula is as follows: in, For width convolution, High-level convolution; S454, perform width-first convolution. and height-first convolution Input inter-layer convolutions to obtain LoRA weight matrices with tensor sizes (B, Nw, Lw, Cw). LoRA weight matrix The calculation formula is as follows: in, For inter-layer convolution, This is a learnable bias.

6. The method for optimizing parameters of a natural language to low-rank adaptation model according to claim 3, characterized in that, In step S44, the loss function The expression is as follows: in, This represents the attention activation feature weights, which are the weights of the model for the current input problem. The distribution of activation responses Mean square error, This is the activation function.

7. A computer storage medium, characterized in that, The system contains a computer program that, when executed, implements the steps of the natural language to low-rank adaptation model parameter optimization method as described in any one of claims 1 to 6.

Images