A model fine-tuning method, system, terminal and medium for parameter update control in large language model fine-tuning process

Abstract

The application belongs to the technical field of model fine-tuning, and specifically discloses a model fine-tuning method, system, terminal and medium for parameter update control in a large language model fine-tuning process. A fine-tuning model is established based on a large language model to be fine-tuned, input data and corresponding expected output data are obtained after pre-processing of training data, feature encoding and feature extraction are performed on the input data through a word vector encoding layer and a backbone neural network, intermediate feature results are updated and fused layer by layer using a multi-layer encoding structure, and model output results are obtained. Training loss is calculated based on the deviation between the model output results and the expected output data, the weight parameters in the backbone neural network are evaluated for importance according to the training loss, the weight parameters with a higher contribution degree to the training loss are selected as a target parameter set, and a parameter update operation is performed on the target parameter set. The application can improve the model fine-tuning efficiency and stability, and enhance the adaptation ability of the model in specific task scenarios.

Description

Technical Field

[0001] This invention belongs to the field of large language model fine-tuning technology, specifically relating to a model fine-tuning method, system, terminal, and medium for parameter update control during the fine-tuning process of large language models. Background Technology

[0002] With the widespread application of large language models in natural language understanding, question answering, and text generation, the industry typically adopts a method of pre-training on large-scale general corpora and then fine-tuning under supervision using domain-specific data to adapt the model to specific business scenarios. During the fine-tuning of large language models, the model parameters are enormous and the network layers are deep, requiring fine-tuning training to be completed under conditions of limited computing resources and limited labeled data.

[0003] Existing fine-tuning methods for large language models mostly rely on a unified loss function to update the trainable parameters of the model as a whole or semi-whole. That is, during the fine-tuning process, a large number of model parameters participate in gradient updates simultaneously. However, in the scenario of fine-tuning large language models, the degree of influence of different parameters on the model output and training loss varies significantly. Some parameters have limited actual contribution to improving model performance during the fine-tuning process, but they still participate in parameter updates, which can easily introduce unnecessary parameter perturbations.

[0004] Furthermore, when fine-tuning a large language model, the overall parameter update method will significantly increase the computational load and memory usage, limiting the training efficiency and deployment flexibility of the model in actual business systems. It may also destroy the general language representation capabilities formed in the pre-training stage and affect the stability of the model output results. Summary of the Invention

[0005] This invention addresses the problems in the prior art by providing a model fine-tuning method, system, terminal, and medium for parameter update control during the fine-tuning of large language models. This solves the problem in the background art where, during the fine-tuning of large language models, due to the large scale of model parameters and significant differences in parameter contribution, existing fine-tuning methods struggle to effectively control the parameter update range, resulting in high computational resource consumption and insufficient model stability.

[0006] The technical solution adopted in this invention is as follows: Firstly, this application provides a model fine-tuning method for parameter update control during the fine-tuning of a large language model, the method comprising the following steps: Step S1: Establish a fine-tuning model based on the large language model to be fine-tuned and its weight parameters. The fine-tuning model includes a word vector encoding layer and a backbone neural network. The weight parameters in the backbone neural network are set to correspond one-to-one with the weight parameters in the large language model to be fine-tuned. The weight parameters in the backbone neural network are used as the parameter update objects in the fine-tuning process. Step S2: Obtain training data, preprocess the training data, and construct input data for fine-tuning and expected output data corresponding to the input data based on the training data. Step S3: Input the input data and the expected output data into the word vector encoding layer, perform feature encoding on the input data to obtain the feature representation of the input data, and perform feature encoding on the expected output data to obtain the feature representation of the expected output data. Step S4: Input the feature representation of the input data into the backbone neural network for feature extraction and output intermediate feature results; The backbone neural network is processed by stacking multiple layers, and the output result is obtained based on the intermediate feature results. Step S5: Calculate the training loss during the fine-tuning process based on the deviation between the output result and the feature representation of the expected output data; Step S6: Based on the training loss, evaluate the importance of the weight parameters in the backbone neural network, select the target parameter set to participate in the parameter update from the weight parameters, and perform the parameter update operation on the target parameter set.

[0007] Furthermore, in step S1, establishing the fine-tuning model based on the large language model to be fine-tuned and its weight parameters includes: Based on the network structure configuration of the large language model to be fine-tuned, the word vector encoding layer and the backbone neural network are constructed. In the fine-tuning model, weight parameters are set that correspond one-to-one with the weight parameters of each network layer in the large language model to be fine-tuned in terms of parameter type, parameter position, and parameter dimension. The corresponding weight parameters are independent of each other in terms of parameter value.

[0008] Furthermore, the preprocessing of the training data in step S2 includes: The original training text is length normalized, and the maximum length of the sentence is set to L. When the length of the sentence in the original training text is less than L, the sentence is padded with padding characters. The length-normalized training text is represented as a text vector C, which serves as the input data, where the dimension of the text vector C is... ; For the model fine-tuning task, the expected output statement S corresponding to the input data is obtained. The expected output statement S is then subjected to length normalization processing consistent with the training text, so that the expected output statement S can participate in the model fine-tuning training as expected output data.

[0009] Furthermore, in step S3, the text vector C, which serves as the input data, is input to the word vector encoding layer. The corresponding feature representation T of the input data is obtained through a word vector mapping operation, where the dimension of the feature representation T is... E represents the output dimension of the word vector encoding layer and is a positive integer; The expected output statement S, which serves as the expected output data, is input into the word vector encoding layer, and the corresponding feature representation label of the expected output data is obtained through word vector mapping operations.

[0010] Furthermore, step S4 includes: The feature representation T is input into the encoding structure of the backbone neural network using the Transformer architecture, and the encoding structure is the Encoder part of the Transformer; The encoder consists of a location embedding layer, an attention network layer, a residual connection layer, a layer normalization layer, and a feedforward computation layer; The feature representation T is encoded using an encoder to obtain the text feature vector Y as an intermediate feature result, where the dimension of the text feature vector Y is... ,satisfy ; The encoder structure used to obtain intermediate feature results is set up in multiple layers in the network depth direction, so that the intermediate feature results output by the previous encoder layer are used as the input of the next encoder layer. The intermediate feature results are updated and fused layer by layer to obtain the output vector O, which is the final output result of the model.

[0011] Furthermore, in step S5, the importance assessment of the weight parameters in the backbone neural network includes: Calculate the cross-entropy loss by comparing the output vector O with the feature representation label of the expected output data:

[0012] in, This represents the output result of the i-th training sample in the output vector O; Based on the changes in training loss, the importance of weight parameters in the backbone neural network is evaluated using the Fisher information matrix, and a parameter importance metric is constructed to characterize the contribution of each weight parameter to the training loss. The weight parameters in the backbone neural network are sorted based on parameter importance metrics, and the weight parameters that contribute more to the training loss are selected as the target parameter set for parameter updates.

[0013] Furthermore, in step S6, updating the weight parameters in the backbone neural network includes: For the attention network layer and feedforward computation layer in the backbone neural network, Fisher information values ​​are calculated for their corresponding weight matrices. Based on Fisher information values, weight matrices whose Fisher information values ​​are within a preset ratio range are selected from the weight matrices of the attention network layer and the feedforward computation layer as target weight matrices. During the parameter update process, a parameter update operation is performed on the target weight matrix, which includes the query weight matrix and numerical weight matrix in the attention network layer, as well as the weight matrix in the feedforward computation layer.

[0014] Secondly, this application provides a model fine-tuning system for parameter update control during large language model fine-tuning, used to implement the model fine-tuning method for parameter update control during large language model fine-tuning as described in the first aspect. The system includes: The model building module is used to build a fine-tuning model based on the large language model to be fine-tuned and its weight parameters. The fine-tuning model includes a word vector encoding layer and a backbone neural network, and the weight parameters in the backbone neural network correspond one-to-one with the weight parameters in the large language model to be fine-tuned. The data preprocessing module is used to acquire training data and preprocess the training data to construct input data for model fine-tuning and the expected output data corresponding to the input data. The feature encoding module is used to input the input data into the word vector encoding layer, perform feature encoding on the input data, and obtain the feature representation used for model calculation; The feature extraction module is used to input feature representations into the backbone neural network for feature extraction and output intermediate feature results; The multi-layer stacking module is used to perform multi-layer stacking processing on the backbone neural network, update and fuse intermediate feature results layer by layer, and obtain the model's output vector. The loss calculation module is used to calculate the training loss during model fine-tuning based on the deviation between the output vector and the expected output data. The parameter evaluation and update module is used to evaluate the importance of the weight parameters in the backbone neural network based on the training loss, select the target weight parameter set to participate in the parameter update, and perform parameter update operations only on the target weight parameter set.

[0015] Thirdly, this application provides a terminal, including: The memory is used to store the model fine-tuning program for parameter update control during the fine-tuning of the large language model; The processor is configured to implement the steps of the model fine-tuning method for parameter update control during large language model fine-tuning as described in the first aspect when executing the model fine-tuning device for parameter update control during large language model fine-tuning.

[0016] Fourthly, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the model fine-tuning method for parameter update control during the fine-tuning of a large language model as described in the first aspect.

[0017] As can be seen from the above technical solutions, the advantages of the present invention are: By evaluating the contribution of model parameter updates during the fine-tuning of large language models, update operations are only performed on parameters that contribute significantly to the training loss. This effectively reduces the invalid computation caused by low-contribution parameters participating in fine-tuning training, thereby reducing the computational load and memory usage during the fine-tuning of large language models.

[0018] By finely controlling the range of parameter updates, excessive perturbation of the pre-training parameter structure during the fine-tuning of large language models can be reduced. This helps maintain the consistency and stability of the model's output results before and after fine-tuning, and avoids destroying the general language representation capabilities formed during the pre-training stage.

[0019] By introducing a screening mechanism based on parameter update contribution during model fine-tuning, the importance of weight parameters in the backbone neural network is evaluated based on the training loss between the model output and the expected output data. Parameter update operations are only performed on weight parameters that contribute significantly to the training loss, thereby avoiding indiscriminate updates to a large number of low-contribution parameters and improving the targeting and effectiveness of the model fine-tuning process.

[0020] During the model fine-tuning process, the word vector encoding layer and the backbone neural network are modeled in a unified manner, and the final output of the model is obtained by stacking multiple encoders layer by layer and feature fusion. This allows the model to extract and integrate high-level semantic features layer by layer while maintaining its original pre-training capabilities, which is beneficial to improving the model's expressive power and prediction accuracy in specific fine-tuning tasks.

[0021] By clearly distinguishing the correspondence between input data, expected output data, and model output results, and by quantifying the importance of weight parameters based on changes in training loss, parameter update decisions can directly reflect the actual contribution of each weight parameter to the improvement of model performance, avoiding the blindness of updates caused by relying solely on uniform gradient information, thereby enhancing the stability of the model fine-tuning process.

[0022] In the parameter update phase, a weight parameter filtering and freezing strategy is introduced. Only the target weight parameter set obtained by filtering is updated, while the remaining weight parameters remain frozen during the fine-tuning process. This effectively reduces the computational complexity and parameter perturbation range during the fine-tuning process, which helps to reduce training resource consumption and improve model training efficiency.

[0023] By selectively updating weight parameters in the attention network layer and feedforward computation layer, the model can maintain overall structural stability while focusing on optimizing key parameters that have a significant impact on the model's prediction results. This helps to achieve better model fit under limited fine-tuning sample conditions. Attached Figure Description

[0024] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of the model fine-tuning method for parameter update control during the fine-tuning of a large language model according to the present invention; Figure 2 This is an architecture diagram of the model fine-tuning system for parameter update control during the fine-tuning of a large language model, as described in this invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see Figure 1 As shown, this application provides a model fine-tuning method for parameter update control during the fine-tuning of a large language model, including the following steps: Step S1: Establish a fine-tuning model based on the large language model to be fine-tuned and its weight parameters. The fine-tuning model includes a word vector encoding layer and a backbone neural network. The weight parameters in the backbone neural network are set to correspond one-to-one with the weight parameters in the large language model to be fine-tuned. The weight parameters in the backbone neural network are used as the parameter update objects in the fine-tuning process. Step S2: Obtain training data, preprocess the training data, and construct input data for fine-tuning and expected output data corresponding to the input data based on the training data. Step S3: Input the input data and the expected output data into the word vector encoding layer, perform feature encoding on the input data to obtain the feature representation of the input data, and perform feature encoding on the expected output data to obtain the feature representation of the expected output data. Step S4: Input the feature representation of the input data into the backbone neural network for feature extraction and output intermediate feature results; The backbone neural network is processed by stacking multiple layers, and the output result is obtained based on the intermediate feature results. Step S5: Calculate the training loss during the fine-tuning process based on the deviation between the output result and the feature representation of the expected output data; Step S6: Based on the training loss, evaluate the importance of the weight parameters in the backbone neural network, select the target parameter set to participate in the parameter update from the weight parameters, and perform the parameter update operation on the target parameter set.

[0028] In one specific embodiment, the model fine-tuning method of the present invention is applied to the supervised fine-tuning training process of a large language model. After pre-training, the large language model needs to be fine-tuned based on a specific domain text corpus to adapt to industry-specific knowledge question-answering tasks.

[0029] In this embodiment, if the existing fine-tuning method of updating overall parameters is used, a large number of parameters will be updated simultaneously, which can easily lead to excessive consumption of training resources and interfere with the model's existing language representation capabilities. By adopting the model fine-tuning method described in this invention, the contribution of parameter updates is evaluated based on the training loss during the fine-tuning process. Only the selected target parameter set is updated, while the remaining parameters remain frozen. This effectively reduces the fine-tuning training cost and improves the model's operational stability while ensuring the model's output performance.

[0030] In some embodiments, establishing a fine-tuning model based on the large language model to be fine-tuned and its weight parameters in step S1 includes: Based on the network structure configuration of the large language model to be fine-tuned, the word vector encoding layer and the backbone neural network are constructed. In the fine-tuning model, weight parameters are set that correspond one-to-one with the weight parameters of each network layer in the large language model to be fine-tuned in terms of parameter type, parameter position, and parameter dimension. The corresponding weight parameters are independent of each other in terms of parameter value.

[0031] In some embodiments, preprocessing the training data in step S2 includes: The original training text is length normalized, and the maximum length of the sentence is set to L. When the length of the sentence in the original training text is less than L, the sentence is padded with padding characters. The length-normalized training text is represented as a text vector C, which serves as the input data, where the dimension of the text vector C is... ; For the model fine-tuning task, the expected output statement S corresponding to the input data is obtained. The expected output statement S is then subjected to length normalization processing consistent with the training text, so that the expected output statement S can participate in the model fine-tuning training as expected output data.

[0032] In some embodiments, in step S3, the text vector C, which serves as input data, is input to the word vector encoding layer, and the feature representation T of the corresponding input data is obtained through a word vector mapping operation, wherein the dimension of the feature representation T of the input data is... E represents the output dimension of the word vector encoding layer and is a positive integer; The expected output statement S, which serves as the expected output data, is input into the word vector encoding layer, and the corresponding feature representation label of the expected output data is obtained through word vector mapping operations.

[0033] In some embodiments, step S4 includes: The feature representation T is input into the encoding structure of the backbone neural network using the Transformer architecture, and the encoding structure is the Encoder part of the Transformer; The encoder consists of a location embedding layer, an attention network layer, a residual connection layer, a layer normalization layer, and a feedforward computation layer; The feature representation T is encoded using an encoder to obtain the text feature vector Y as an intermediate feature result, where the dimension of the text feature vector Y is... ,satisfy ; The encoder structure used to obtain intermediate feature results is set up in multiple layers along the network depth direction, so that the intermediate feature results output by the previous encoder layer are used as the input of the next encoder layer. The intermediate feature results are updated and fused layer by layer to obtain the output vector O, which is the final output result of the model.

[0034] Will Input the location embedding layer to obtain a vector containing location information. ,in The location embedding method is selected in the following form.

[0035]

[0036] In the above formula Represents the characters in the text in the vector The specific value. and This represents the position of a character in the text, indicating even and odd positions respectively, and E is the output dimension of the word vector encoding layer.

[0037] Based on the above formula, we can... Add position information for each character, and output as a text position vector. :

[0038] vector Divide the space along the last dimension E into N equal subspaces, each subspace having a dimension of... Therefore, it can be concluded that:

[0039] in , The number of subspaces N selected must be divisible by E, i.e. The number of spaces is a positive integer. .

[0040] Text position vector The input is fed into a linear network layer, undergoes a linear transformation, and yields vectors Q, K, and V, all of which have the same dimension. .

[0041]

[0042]

[0043]

[0044] in , and This is the original weight matrix of the large model. and To initialize the weight matrix during fine-tuning, a random Gaussian initialization method is used, with all dimensions being equal to 1. .

[0045] , and Similarly, it can be split according to the last dimension E. ,in , N is a positive integer, where N is the number of subspaces divided, which is also the number of attention heads.

[0046] The attention weights are calculated for vectors Q, K, and V to obtain the output of the attention network layer. :

[0047] Attention network layer vectors The text feature vector Y is obtained by sequentially passing through the residual connection layer, the layer normalization layer, and the feed forward layer, and finally obtaining the output text feature vector Y of the Encode layer. The specific process is as follows: The residual connection layer and the layer normalization layer are conventional neural network layers. Vector A outputs as follows after passing through the residual connection layer and the layer normalization layer: :

[0048] The Feed Forward layer is a multi-layer fully connected neural network; assuming the number of fully connected neural network layers is 2, the output of vector G after passing through the Feed Forward layer is... :

[0049] The parameters of each fully connected network layer are ,in This is the original weight matrix of the large model. To initialize the weight matrix during fine-tuning, a random Gaussian initialization method is used, with all dimensions being equal to 1. .

[0050] The final output of the Encode layer still needs to pass through a residual connection layer and a layer normalization layer to obtain... .

[0051] In some embodiments, step S5, evaluating the importance of the weight parameters in the backbone neural network, includes: Calculate the cross-entropy loss by comparing the output vector O with the feature representation label of the expected output data:

[0052] in, This represents the output result of the i-th training sample in the output vector O; Based on the changes in training loss, the importance of weight parameters in the backbone neural network is evaluated using the Fisher information matrix, and a parameter importance metric is constructed to characterize the contribution of each weight parameter to the training loss. The weight parameters in the backbone neural network are sorted based on parameter importance metrics, and the weight parameters that contribute more to the training loss are selected as the target parameter set for parameter updates.

[0053] In some embodiments, in step S6, performing parameter updates on the weight parameters in the backbone neural network includes: For the attention network layer and feedforward computation layer in the backbone neural network, Fisher information values ​​are calculated for their corresponding weight matrices. Based on Fisher information values, weight matrices whose Fisher information values ​​are within a preset ratio range are selected from the weight matrices of the attention network layer and the feedforward computation layer as target weight matrices. During the parameter update process, a parameter update operation is performed on the target weight matrix, which includes the query weight matrix and numerical weight matrix in the attention network layer, as well as the weight matrix in the feedforward computation layer.

[0054] When efficiently fine-tuning large model parameters, only the weight matrix in the attention network layer is updated. and the weight matrix in the Feed Forward layer ,in The remaining parameters are frozen and will not be included in the parameter updates for fine-tuning.

[0055] Here For example, the parameter update method based on the Fisher information matrix is ​​shown:

[0056] N represents the number of attention heads. ,in , It is a positive integer.

[0057] calculate Corresponding Fisher information value: First calculate each gradient:

[0058] in These are the parameters that need to be updated when fine-tuning a large model. .

[0059] Note It is an asymmetric matrix, and Select The diagonal elements of a matrix are represented as follows:

[0060] Finally, calculate each Corresponding Fisher information value:

[0061] right = Calculate the respective Fisher information values:

[0062] right The values ​​are sorted from largest to smallest. In this embodiment, the top 50% of Fisher information values ​​are selected. Assume the indexes of the top 50% after filtering form a set. , , This indicates the rounding operation.

[0063] Therefore, when updating parameters of a large model, only update... The following weight matrix, ;

[0064]

[0065] for The corresponding gradient.

[0066] Therefore, for the weight matrix in a large model The update method is as follows:

[0067]

[0068] in To update the gradient, the value is when hour, ;when hour, . To optimize the learning rate for fine-tuning large models, stochastic gradient descent is chosen as the update method. For the weights of the attention network layers... and Feed Forward layer weight matrix ,in Update the parameters in the same way.

[0069] Please see Figure 2 As shown, in some embodiments, this application provides a model fine-tuning system for parameter update control during large language model fine-tuning, used to implement a model fine-tuning method such as the one for parameter update control during large language model fine-tuning. The system includes: The model building module is used to build a fine-tuning model based on the large language model to be fine-tuned and its weight parameters. The fine-tuning model includes a word vector encoding layer and a backbone neural network, and the weight parameters in the backbone neural network correspond one-to-one with the weight parameters in the large language model to be fine-tuned. The data preprocessing module is used to acquire training data and preprocess the training data to construct input data for model fine-tuning and the expected output data corresponding to the input data. The feature encoding module is used to input the input data into the word vector encoding layer, perform feature encoding on the input data, and obtain the feature representation used for model calculation; The feature extraction module is used to input feature representations into the backbone neural network for feature extraction and output intermediate feature results; The multi-layer stacking module is used to perform multi-layer stacking processing on the backbone neural network, update and fuse intermediate feature results layer by layer, and obtain the model's output vector. The loss calculation module is used to calculate the training loss during model fine-tuning based on the deviation between the output vector and the expected output data. The parameter evaluation and update module is used to evaluate the importance of the weight parameters in the backbone neural network based on the training loss, select the target weight parameter set to participate in the parameter update, and perform parameter update operations only on the target weight parameter set.

[0070] In some embodiments, this application provides a terminal, including: The memory is used to store the model fine-tuning program for parameter update control during the fine-tuning of the large language model; A processor is configured to implement the steps of the model fine-tuning method for parameter update control during the fine-tuning of a large language model when executing the model fine-tuning system for parameter update control during the fine-tuning of a large language model.

[0071] In some embodiments, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the model fine-tuning method for parameter update control during the fine-tuning of a large language model.

[0072] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A model fine-tuning method for parameter update control during large language model fine-tuning, characterized in that, Includes the following steps: Step S1: Establish a fine-tuning model based on the large language model to be fine-tuned and its weight parameters. The fine-tuning model includes a word vector encoding layer and a backbone neural network. The weight parameters in the backbone neural network are set to correspond one-to-one with the weight parameters in the large language model to be fine-tuned. The weight parameters in the backbone neural network are used as the parameter update objects in the fine-tuning process. Step S2: Obtain training data, preprocess the training data, and construct input data for fine-tuning and expected output data corresponding to the input data based on the training data. Step S3: Input the input data and the expected output data into the word vector encoding layer, perform feature encoding on the input data to obtain the feature representation of the input data, and perform feature encoding on the expected output data to obtain the feature representation of the expected output data. Step S4: Input the feature representation of the input data into the backbone neural network for feature extraction and output intermediate feature results; The backbone neural network is processed by stacking multiple layers, and the output result is obtained based on the intermediate feature results. Step S5: Calculate the training loss during the fine-tuning process based on the deviation between the output result and the feature representation of the expected output data; Step S6: Based on the training loss, evaluate the importance of the weight parameters in the backbone neural network, select the target parameter set to participate in the parameter update from the weight parameters, and perform the parameter update operation on the target parameter set.

2. The model fine-tuning method for parameter update control during large language model fine-tuning according to claim 1, characterized in that, Step S1, which establishes the fine-tuning model based on the large language model to be fine-tuned and its weight parameters, includes: Based on the network structure configuration of the large language model to be fine-tuned, the word vector encoding layer and the backbone neural network are constructed. In the fine-tuning model, weight parameters are set that correspond one-to-one with the weight parameters of each network layer in the large language model to be fine-tuned in terms of parameter type, parameter position, and parameter dimension. The corresponding weight parameters are independent of each other in terms of parameter value.

3. The model fine-tuning method for parameter update control during large language model fine-tuning according to claim 1, characterized in that, The preprocessing of the training data in step S2 includes: The original training text is length normalized, and the maximum length of the sentence is set to L. When the length of the sentence in the original training text is less than L, the sentence is padded with padding characters. The length-normalized training text is represented as a text vector C, which serves as the input data, where the dimension of the text vector C is... ; For the model fine-tuning task, the expected output statement S corresponding to the input data is obtained. The expected output statement S is then subjected to length normalization processing consistent with the training text, so that the expected output statement S can participate in the model fine-tuning training as expected output data.

4. The model fine-tuning method for parameter update control during large language model fine-tuning according to claim 3, characterized in that, In step S3, the text vector C, which serves as the input data, is fed into the word vector encoding layer. The feature representation T of the corresponding input data is obtained through a word vector mapping operation, where the dimension of the feature representation T is... E represents the output dimension of the word vector encoding layer and is a positive integer; The expected output statement S, which serves as the expected output data, is input into the word vector encoding layer, and the corresponding feature representation label of the expected output data is obtained through word vector mapping operations.

5. The model fine-tuning method for parameter update control during large language model fine-tuning according to claim 4, characterized in that, Step S4 includes: The feature representation T is input into the encoding structure of the backbone neural network using the Transformer architecture, and the encoding structure is the Encoder part of the Transformer; The encoder consists of a location embedding layer, an attention network layer, a residual connection layer, a layer normalization layer, and a feedforward computation layer; The feature representation T is encoded using an encoder to obtain the text feature vector Y as an intermediate feature result, where the dimension of the text feature vector Y is... ,satisfy ; The encoder structure used to obtain intermediate feature results is set up in multiple layers in the network depth direction, so that the intermediate feature results output by the previous encoder layer are used as the input of the next encoder layer. The intermediate feature results are updated and fused layer by layer to obtain the output vector O, which is the final output result of the model.

6. The model fine-tuning method for parameter update control during large language model fine-tuning according to claim 5, characterized in that, In step S5, the importance evaluation of the weight parameters in the backbone neural network includes: Calculate the cross-entropy loss by comparing the output vector O with the feature representation label of the expected output data: in, This represents the output result of the i-th training sample in the output vector O; Based on the changes in training loss, the importance of weight parameters in the backbone neural network is evaluated using the Fisher information matrix, and a parameter importance metric is constructed to characterize the contribution of each weight parameter to the training loss. The weight parameters in the backbone neural network are sorted based on parameter importance metrics, and the weight parameters that contribute more to the training loss are selected as the target parameter set for parameter updates.

7. The model fine-tuning method for parameter update control during large language model fine-tuning according to claim 6, characterized in that, In step S6, updating the weight parameters in the backbone neural network includes: For the attention network layer and feedforward computation layer in the backbone neural network, Fisher information values ​​are calculated for their corresponding weight matrices. Based on Fisher information values, weight matrices whose Fisher information values ​​are within a preset ratio range are selected from the weight matrices of the attention network layer and the feedforward computation layer as target weight matrices. During the parameter update process, a parameter update operation is performed on the target weight matrix, which includes the query weight matrix and numerical weight matrix in the attention network layer, as well as the weight matrix in the feedforward computation layer.

8. A model fine-tuning system for parameter update control during large language model fine-tuning, used to implement the model fine-tuning method for parameter update control during large language model fine-tuning as described in claim 1, characterized in that, The system includes: The model building module is used to build a fine-tuning model based on the large language model to be fine-tuned and its weight parameters. The fine-tuning model includes a word vector encoding layer and a backbone neural network, and the weight parameters in the backbone neural network correspond one-to-one with the weight parameters in the large language model to be fine-tuned. The data preprocessing module is used to acquire training data and preprocess the training data to construct input data for model fine-tuning and the expected output data corresponding to the input data. The feature encoding module is used to input the input data into the word vector encoding layer, perform feature encoding on the input data, and obtain the feature representation used for model calculation; The feature extraction module is used to input feature representations into the backbone neural network for feature extraction and output intermediate feature results. The multi-layer stacking module is used to perform multi-layer stacking processing on the backbone neural network, update and fuse intermediate feature results layer by layer, and obtain the model's output vector. The loss calculation module is used to calculate the training loss during model fine-tuning based on the deviation between the output vector and the expected output data. The parameter evaluation and update module is used to evaluate the importance of the weight parameters in the backbone neural network based on the training loss, select the target weight parameter set to participate in the parameter update, and perform parameter update operations only on the target weight parameter set.

9. A terminal, characterized in that, include: The memory is used to store the model fine-tuning program for parameter update control during the fine-tuning of the large language model; The processor, when executing the model fine-tuning device for parameter update control during large language model fine-tuning, implements the steps of the model fine-tuning method for parameter update control during large language model fine-tuning as described in claim 1.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. When the computer reads the computer instructions from the storage medium, the computer executes the model fine-tuning method for parameter update control during the fine-tuning of a large language model as described in claim 1.

Images