Lightweight high-efficiency parameter fine tuning processing method for large model

Abstract

The invention relates to a lightweight efficient parameter fine tuning processing method for a large model, which belongs to the technical field of natural language processing, and comprises the following steps of: converting information to be processed into a vector form through a coding layer of a pre-training model to obtain an input vector; introducing a dimension reduction matrix and a dimension raising matrix beside a query matrix and a value matrix of the pre-training model, wherein the dimension reduction matrix and the dimension raising matrix are both generated by outer accumulation of one-dimensional vectors; multiplying the input vector by the dimension reduction matrix and the dimension raising matrix in sequence to obtain a first feature vector; multiplying the input vector by an original weight matrix which is kept frozen in the pre-training model to obtain a second feature vector; according to the method, only the four groups of one-dimensional vector parameters for generating the dimension reduction matrix and the dimension raising matrix are finely adjusted, so that the problems of high calculation cost and difficulty in deployment in a resource-limited scene due to the fact that a mode of adjusting all parameters of a pre-training model is adopted in traditional full parameter fine adjustment are solved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a lightweight and efficient parameter fine-tuning method for large models. Background Technology

[0002] With the development of artificial intelligence (AI) technology, especially the breakthroughs in deep learning, large-scale pre-trained models have become a key force driving significant success in multiple fields such as natural language processing (NLP) and computer vision (CV).

[0003] These models, through unsupervised or supervised pre-training on large-scale datasets, are able to capture rich linguistic or visual features and demonstrate strong generalization capabilities across various downstream tasks. In the field of Traditional Chinese Medicine (TCM), such large-scale pre-trained models have also shown great potential. For example, applying large-scale pre-trained models to TCM syndrome differentiation and treatment can effectively improve diagnostic and treatment efficiency.

[0004] Traditional full parameter fine-tuning mostly involves adjusting all parameters of the pre-trained model. Due to the huge number of parameters, this results in high computational costs and makes it difficult to deploy in resource-constrained scenarios. Summary of the Invention

[0005] The main objective of this invention is to provide a lightweight and efficient parameter fine-tuning method for large models, aiming to improve the problem that traditional full parameter fine-tuning mostly adopts the method of adjusting all parameters of the pre-trained model. Due to the huge number of parameters, this results in high computational costs and is difficult to deploy in resource-constrained scenarios.

[0006] Based on the first main aspect of the present invention, a lightweight and efficient parameter fine-tuning method for large models is provided, comprising the following steps executed by a computer system:

[0007] The information to be processed is converted into vector form through the encoding layer of the pre-trained model to obtain the input vector;

[0008] A dimension reduction matrix and a dimension increase matrix are introduced next to the query matrix and value matrix of the pre-trained model. Both the dimension reduction matrix and the dimension increase matrix are generated by the aggregation of the outer product of one-dimensional vectors.

[0009] The first eigenvector is obtained by multiplying the input vector by the reduced-dimensional matrix and the increased-dimensional matrix in sequence.

[0010] The second feature vector is obtained by multiplying the input vector with the original weight matrix that remains frozen in the pre-trained model.

[0011] The first feature vector and the second feature vector are fused to obtain a deep feature representation;

[0012] After normalizing the deep feature representation, the predicted label is obtained. The loss is calculated using the predicted label and the true label. Based on the loss, the four sets of one-dimensional vector parameters of the generated dimensionality reduction matrix and dimensionality increase matrix are finely adjusted through backpropagation.

[0013] By adopting the above technical solution, the original weights of the pre-trained model are frozen, and only the parameters of the four sets of one-dimensional vectors that generate the reduced-dimensional matrix and the increased-dimensional matrix are fine-tuned. This significantly reduces the number of trainable parameters, lowers computational complexity and storage overhead, and improves the problem that traditional full parameter fine-tuning mostly adopts the method of adjusting all parameters of the pre-trained model. Due to the huge number of parameters, this results in high computational cost and is difficult to deploy in resource-constrained scenarios.

[0014] As a further preferred embodiment, in the aforementioned method, the information to be processed includes text information or image information, wherein the text information includes texts of traditional Chinese medicine classics and texts of traditional Chinese medicine prescriptions, and the image information includes images of Chinese medicinal materials and images of pathology and physical signs.

[0015] As a further preferred embodiment, in the aforementioned method, the dimensionality reduction matrix is ​​set as... The upgraded matrix is ;

[0016] The dimensionality reduction matrix From the first group of one-dimensional vectors With the second group of one-dimensional vectors Generated through outer product aggregation, satisfying ;

[0017] The upgraded matrix From the third group of one-dimensional directions With the fourth group of one-dimensional vectors Generated by L2 norm aggregation through outer product, satisfying .

[0018] As a further preferred embodiment, in the aforementioned method, the first feature vector is set as... It is obtained by successively multiplying the input vector with the reduced-dimensional matrix and the increased-dimensional matrix, satisfying the following conditions: ,in The input vector is obtained by converting the information to be processed into vector form through the encoding layer of a pre-trained model.

[0019] Let the second feature vector be It is obtained by multiplying the input vector with the original weight matrix of the pre-trained model, satisfying... ,in To maintain the original weight matrix frozen in the pre-trained model, and during backpropagation The parameters are not updated.

[0020] As a further preferred embodiment, in the aforementioned method, the depth feature is defined as... It is obtained by adding and fusing the first feature vector and the second feature vector, satisfying the following conditions. .

[0021] As a further preferred embodiment, in the aforementioned method, the normalization process of the deep feature representation is performed using the softmax function, and the predicted label is... ,satisfy The softmax function converts deep features into prediction results in the form of a probability distribution.

[0022] As a further preferred embodiment, in the aforementioned method, the loss is set to... The cross-entropy function is used for calculation, and its formula is as follows:

[0023]

[0024] in, For the first The true label of each sample For the first Predicted label for each sample, It is the natural logarithm function. The total number of samples.

[0025] As a further preferred option, in the aforementioned method, the process of fine-tuning only the four sets of one-dimensional vector parameters for generating the reduced-dimensional matrix and the increased-dimensional matrix through backpropagation is as follows:

[0026] For the third set of vectors used to generate the upscaling matrix Update to meet requirements ;

[0027] For the first set of vectors used to generate the dimensionality reduction matrix Update to meet requirements ;

[0028] For the fourth set of vectors used to generate the upscaling matrix Update to meet requirements ;

[0029] For the second set of vectors used to generate the dimensionality reduction matrix Update to meet requirements ;

[0030] in, For learning rate, Losses The partial derivative with respect to the corresponding vector.

[0031] As a further preferred embodiment, in the aforementioned method, the pre-trained model is a language pre-trained model or a visual pre-trained model; the language pre-trained model is used for natural language understanding tasks, and the visual pre-trained model is used for image analysis tasks.

[0032] As a further preferred embodiment, in the aforementioned method, the encoding layer of the pre-trained model is the encoding layer of a language pre-trained model or the encoding layer of a visual pre-trained model; the encoding layer of the language pre-trained model is used to convert text information into vectors, and the encoding layer of the visual pre-trained model is used to convert image information into vectors.

[0033] According to a second key aspect of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0034] The memory stores a computer program that, when executed by the processor, causes the processor to perform the aforementioned lightweight and efficient parameter fine-tuning method for the large model.

[0035] Based on a third key aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, implements the aforementioned lightweight and efficient parameter fine-tuning method for large models.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. In this invention, by freezing the original weights of the pre-trained model, only the four sets of one-dimensional vector parameters that generate the dimension reduction matrix and the dimension increase matrix are fine-tuned, thereby significantly reducing the number of trainable parameters, reducing computational complexity and storage overhead. This improves the problem that traditional full parameter fine-tuning mostly adopts the method of adjusting all parameters of the pre-trained model, which results in high computational cost and difficulty in deployment in resource-constrained scenarios due to the huge number of parameters.

[0038] 2. In this invention, by fine-tuning only a small number of vector parameters, the generalization ability of the pre-trained model is preserved, thereby effectively avoiding overfitting on a dataset with limited training data. This improves the problem that traditional full parameter fine-tuning often involves adjusting a large number of parameters, which can easily lead to overlearning of data details on small datasets, resulting in poor model performance in actual deployment.

[0039] 3. In this invention, by using vector outer product aggregation to generate dimensionality-reduced and dimensionality-increasing matrices, the calculation logic of parameter adjustment is simplified, thereby reducing the complexity of model training and the time required for training. This improves the problem that traditional low-rank approximation and matrix factorization techniques mostly use complex matrix factorization operations, which increase the complexity of the training process and thus cause long model training time.

[0040] 4. In this invention, by reducing the number of trainable parameters and storage overhead, the demand for hardware computing power is reduced, thereby enabling the model to run efficiently in resource-constrained environments such as consumer-grade graphics processing units and edge computing. This improves the problem that traditional parameter fine-tuning techniques, which mostly require high computing power, are difficult to meet the computing needs in resource-constrained hardware environments, thus making it difficult to support real-time application scenarios.

[0041] 5. In this invention, by only needing to fine-tune four sets of one-dimensional vector parameters, the number of hyperparameters and the difficulty of adjustment are reduced, thereby reducing the requirements for the user's professional knowledge and making the fine-tuning process simpler. This improves the problem that traditional parameter fine-tuning often requires complex hyperparameter selection and optimization, which is not user-friendly for non-expert users and causes great difficulty in fine-tuning large models. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0043] Figure 1 A schematic diagram of the overall model architecture of a lightweight and efficient parameter fine-tuning method for a large model is shown in one embodiment of the present invention.

[0044] Figure 2 The following is an execution flowchart of a lightweight and efficient parameter fine-tuning method for a large model according to one embodiment of the present invention.

[0045] Figure 3 This illustrates how the number of parameters of the AVLoRA and LoRA models varies with rank on the cola dataset in one embodiment of the present invention.

[0046] Figure 4 This illustrates how the number of parameters of the AVLoRA and LoRA models varies with rank on the RTE dataset in one embodiment of the present invention.

[0047] Figure 5This illustrates the performance metrics of the two models as a function of rank on the Cola dataset in one embodiment of the present invention.

[0048] Figure 6 This illustrates the performance metrics of the two models as a function of rank on the RTE dataset in one embodiment of the present invention.

[0049] Figure 7 This invention illustrates a visualization of the parameters and fractional stability of the LoRA scheme at different ranks in one embodiment of the invention.

[0050] Figure 8 A visualization of the parameter and fractional stability of the AVLoRA scheme at different ranks is shown in one embodiment of the present invention. Detailed Implementation

[0051] The preferred embodiments of the present invention will be described in detail below to provide a clearer understanding of the purpose, features, and advantages of the invention. It should be understood that the following embodiments are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the technical solution of the invention.

[0052] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known techniques associated with the invention may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0053] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0054] like Figure 2 As shown, in one embodiment of the present invention, a lightweight and efficient parameter fine-tuning method for large models includes the following steps S110-S160 executed by a computer system:

[0055] S110, the information to be processed is converted into vector form through the encoding layer of the pre-trained model to obtain the input vector;

[0056] S120, introduce a dimension reduction matrix and a dimension increase matrix next to the query matrix and value matrix of the pre-trained model. Both the dimension reduction matrix and the dimension increase matrix are generated by the aggregation of the outer product of one-dimensional vectors.

[0057] S130, Multiply the input vector with the reduced-dimensional matrix and the increased-dimensional matrix in sequence to obtain the first eigenvector;

[0058] S140, multiply the input vector with the original weight matrix that is kept frozen in the pre-trained model to obtain the second feature vector;

[0059] S150, the first feature vector and the second feature vector are fused to obtain a deep feature representation;

[0060] S160 normalizes the deep feature representation and then predicts the predicted label. The loss is calculated using the predicted label and the true label. Based on the loss, backpropagation is used to fine-tune the four sets of one-dimensional vector parameters that generate the dimensionality reduction matrix and the dimensionality increase matrix.

[0061] In one possible implementation, the specific execution steps of S110 include: converting the information x to be processed into vector form through the Encoder layer of the pre-trained model, and obtaining the vector. , represented as , where Encoder represents the encoding layer of the pre-trained model;

[0062] In one possible implementation, the specific execution steps of S120 include:

[0063] A dimension reduction matrix A and an increased dimension matrix B are introduced alongside the Q and V matrices in the pre-trained model to ensure that the dimensions of the input data are aligned with the dimensions of the model weight matrix. The corresponding dimension reduction matrix A and increased dimension matrix B are obtained by aggregating the outer products of vectors, and their matrix representations are as follows:

[0064] ,

[0065] ,in and It is a one-dimensional vector;

[0066] In one possible implementation, the specific execution steps of S130 include:

[0067] vector Multiplying the matrix by the reduced-dimensional matrix A and the increased-dimensional matrix B yields V. out :

[0068] ,in ;

[0069] In one possible implementation, the specific execution steps of S140 include:

[0070] input vector The input modal feature representation is obtained by multiplying the original weights of the pre-trained model, which remain frozen in place. :

[0071]

[0072] in: These are the initial weights of the pre-trained model; the original weights will not be updated during subsequent backpropagation.

[0073] In one possible implementation, the specific execution steps of S150 include:

[0074] V obtained from S130 out And obtained from S140 By fusing and adding the features, we obtain the deep feature mode representation. : ;

[0075] In one possible implementation, the specific execution steps of S160 include:

[0076] Deep feature mode representation After normalization using the softmax function, the predicted label is obtained. Use the real labels from the dataset The loss is calculated using cross-entropy with the predicted labels, and then backpropagation is performed to fine-tune the four sets of vectors. , , , parameter.

[0077]

[0078]

[0079] Backpropagation process:

[0080]

[0081]

[0082] , Similarly, It is the learning rate.

[0083] The above describes a lightweight parameter fine-tuning method for large models, where information x is text information or image information.

[0084] The following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effectiveness of the lightweight parameter fine-tuning method for large models proposed in this invention:

[0085] Example 1:

[0086] See Figure 1 The overall architecture of the model shown in this embodiment, a lightweight and efficient parameter fine-tuning method for a large-scale intelligent diagnostic model of traditional Chinese medicine, includes the following steps:

[0087] S110: The TCM syndrome differentiation and treatment text information x to be processed is converted into vector form through the Encoder layer of the pre-trained language model to obtain the text information vector. , represented as , where Encoder represents the encoding layer of the pre-trained language model;

[0088] S120 introduces a dimension reduction matrix A and a dimension increase matrix B next to the Q and V matrices in the pre-trained language model to ensure the dimension alignment between the input data and the model weight matrix. The corresponding dimension reduction matrix A and dimension increase matrix B are obtained by aggregating the outer products of vectors, and their matrix representations are as follows:

[0089] ,

[0090] ,in and It is a one-dimensional vector;

[0091] S130, based on the reduced dimension matrix A and the increased dimension matrix B, the input text information... A linear transformation is performed to obtain the transformed output information. This information is then integrated with the textual information to ultimately extract the structural features of the input text that contain richer semantic layers. That is, text information vector Multiplying the matrix by the reduced-dimensional matrix A and the increased-dimensional matrix B yields V. out :

[0092] ,in ;

[0093] S140, input vector The structural feature representation of the input text is obtained by multiplying it with the original weights of the pre-trained language model, which remain frozen in place. :

[0094]

[0095] in: These are the initial weights of the pre-trained language model; the original weights will not be updated during subsequent backpropagation.

[0096] S150, the V obtained from S130 out And obtained from S140 By fusing and adding the features, we obtain the text feature h: ;

[0097] S160: After normalizing the text features using the softmax function, the labels are predicted to obtain the predicted labels. Use the real labels from the dataset The loss is calculated using cross-entropy with the predicted labels, and then backpropagation is performed to fine-tune the four sets of vectors. , , , parameter.

[0098]

[0099]

[0100] Backpropagation process:

[0101]

[0102]

[0103] , Similarly, It is the learning rate.

[0104] Example 2:

[0105] The present invention provides a lightweight and efficient parameter fine-tuning method for a large-scale image classification model of traditional Chinese medicine pathology and physical signs, comprising the following steps:

[0106] S110: The pathological and physical sign image information x to be processed is converted into vector form through the Encoder layer of the pre-trained model VIT to obtain the image feature vector. , represented as , where Encoder represents the encoding layer of the pre-trained model VIT.

[0107] The remaining steps are the same as in Example 1.

[0108] Experimental Example 1:

[0109] To verify the effectiveness of Example 1, several different pre-trained large models were selected. These models were fine-tuned using the training set from the Natural Language Understanding dataset, following the steps in Example 1 (hereinafter referred to as AVLoRA). Subsequently, they were validated using the test set from the Natural Language Understanding dataset. The results are shown in Table 1.

[0110] Table 1. Experimental results of the present invention on natural language understanding tasks.

[0111]

[0112] The results of various methods on GLUE are shown in Table 1. As can be seen, Example 1 achieves more competitive and advantageous performance compared to LoRA and VeRA.

[0113] Example 1 reduces the number of model parameters without significantly sacrificing model accuracy, making it more parameter-efficient. For example, when using the DeBERTa model as the base model, Example 1 uses only 300,000 trainable parameters, while LoRA requires 4.7 million parameters.

[0114] Compared to LoRA, Example 1 reduces the number of parameters by 4.4 million, a reduction of 13 times, while maintaining model accuracy. Example 1 uses one-dimensional vectors for training, making it more lightweight and robust. Furthermore, Example 1 achieves excellent performance on other datasets (such as MNLI, SST-2, QNLI), demonstrating the stability and lightweight nature of the invention.

[0115] Experimental Example 2:

[0116] The GPU memory usage and number of trainable parameters were evaluated using the Qwen-VL-Chat model for comparison with existing methods. Specifically, during fine-tuning, the values ​​of the dimensionality reduction and expansion matrices were set from 1 to 64, and the trainable parameters and GPU memory usage of the model were statistically analyzed during the fine-tuning process. The results are as follows: Figure 3-6 As shown.

[0117] This result verifies the effectiveness of the present invention in improving resource efficiency. Figure 3-4 As shown, the number of trainable parameters remains relatively stable regardless of whether the rank is set to 1 or 126. When dealing with a model with 7 billion (7B) parameters, this invention is able to keep GPU memory usage within a manageable range.

[0118] Specifically, when the matrix rank of this invention increases from 1 to 64, the GPU memory usage increases only slightly, from 34.81 GB to 34.82 GB, a mere 0.028%. In contrast, under similar conditions, the LoRA method leads to a more significant increase in GPU memory usage, from 34.77 GB to 36.95 GB, an increase of 6.26%. This characteristic highlights the superior efficiency and portability of this invention, especially when fine-tuning large models on consumer-grade GPUs.

[0119] Experimental Example 3:

[0120] To verify the performance and lightweight nature of this invention, the ColA and RTE datasets from the Natural Language Understanding datasets were selected, and the DeBERTa model was fine-tuned. The fine-tuning process strictly followed the steps in Example 1. During the fine-tuning process, the values ​​of the dimensionality reduction matrix and the dimensionality increase matrix were set to 4, 8, 16, and 32, respectively. The invention was validated using test sets from the ColA and RTE datasets to evaluate its performance and lightweight nature. The results are shown below. Figure 5-6 .

[0121] The experimental results of this invention on natural language understanding tasks are as follows: Figure 7-8 As shown in the diagram, experiments on the CoLA dataset revealed that as the r value increases, the model performance initially improves and then stabilizes. This phenomenon suggests that, with sufficient training data, higher rank and a larger number of trainable parameters can help improve model performance, but they also come at the cost of higher training.

[0122] Furthermore, the experiments showed that the present invention outperformed LoRA under all r-value settings without significant accuracy loss. This indicates that, under the same conditions, the present invention is more lightweight and better suited for deployment in edge computing environments. Example 4:

[0123] Using the Base and Large variants of the Vision Transformer, fine-tuning tests were conducted on CIFAR100, Food101, and Flower102, following the same fine-tuning process as in Example 2. Several different fine-tuning methods were employed and compared with experimental methods for verification. The results are shown in Table 2.

[0124] Table 2 shows the experimental results of the present invention on image tasks.

[0125]

[0126] Table 2 shows that Example 2 reduces the number of trainable parameters by 4 times in the VIT-Large model and outperforms LoRA on the CIFAR100 and FLOWER datasets using the Base model.

[0127] Specifically, under the VIT-Base model, this invention not only significantly reduces parameter load but also exhibits stronger generalization ability. On the CIFAR100 dataset, this invention achieves an accuracy of 91.29%, while LoRA achieves 87.76%, demonstrating a clear performance advantage. On the FLOWER dataset, this advantage is even more pronounced, with an accuracy of 98.89%, compared to LoRA's 98.48%.

[0128] The technical terms, principles, or means related to the technical solutions of the present invention mentioned in the above embodiments, which are not described in detail above, are all well-known technologies or common practices that are known to those skilled in the art.

[0129] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A lightweight efficient parameter fine-tuning processing method of a large model, characterized by, The method comprises the following steps implemented by a computer system: transforming the information to be processed into a vector form through an encoding layer of a pre-trained model to obtain an input vector; introducing a dimension reduction matrix and a dimension increase matrix beside a query matrix and a value matrix of the pre-trained model, wherein the dimension reduction matrix and the dimension increase matrix are both generated by an outer product aggregation of one-dimensional vectors; multiplying the input vector with the dimension reduction matrix and the dimension increase matrix in sequence to obtain a first feature vector; multiplying the input vector with an original weight matrix in the pre-trained model which remains frozen to obtain a second feature vector; fusing the first feature vector with the second feature vector to obtain a deep feature representation; performing normalization processing on the deep feature representation to obtain a predicted label, calculating a loss by using the predicted label and a real label, and only fine-tuning four groups of one-dimensional vector parameters of the dimension reduction matrix and the dimension increase matrix through back propagation based on the loss.

2. The method of claim 1, wherein, The information to be processed includes text information or image information, wherein the text information includes traditional Chinese medicine classic text and traditional Chinese medicine prescription text, and the image information includes Chinese medicinal material image and pathology and sign image.

3. The method of claim 1, wherein the method further comprises: The dimension reduction matrix is set as , and the dimension increasing matrix is ; the dimensionality reduction matrix from the first set of one-dimensional vectors with the second set of one-dimensional vectors generated by the outer product aggregation, satisfying ; the elevation matrix from the third set of one-dimensional vectors with the fourth set of one-dimensional vectors generated by L2 norm aggregation of outer products, satisfying .

4. The method of claim 3, wherein the method further comprises: The first feature vector is set as , obtained by sequentially multiplying an input vector with a dimension reduction matrix and a dimension increase matrix, satisfying , wherein is an input vector obtained after converting the information to be processed into a vector form through an encoding layer of a pre-trained model; The second feature vector is set as , obtained by multiplying the input vector with the original weight matrix of the pre-trained model, satisfying , wherein is the original weight matrix that remains frozen in the pre-trained model, and the parameters of are not updated during backpropagation.

5. The method of claim 4, wherein, The deep feature representation is expressed as , and is obtained by adding and fusing the first feature vector and the second feature vector, satisfying .

6. The method of claim 5, wherein the method further comprises: The normalization processing on the deep feature representation adopts a softmax function, and a prediction label is , satisfies , and the softmax function converts the deep feature into a prediction result in the form of a probability distribution.

7. The method of claim 1, wherein the method further comprises: The pre-trained model is a language pre-trained model or a visual pre-trained model, wherein the language pre-trained model is used for a natural language understanding task, and the visual pre-trained model is used for an image analysis task.

8. The method of claim 1, wherein, The encoding layer of the pre-trained model is an encoding layer of a language pre-trained model or an encoding layer of a visual pre-trained model, wherein the encoding layer of the language pre-trained model is used for converting text information into a vector, and the encoding layer of the visual pre-trained model is used for converting image information into a vector.

9. An electronic device, comprising: The method comprises the following steps: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; the memory stores a computer program, and when the program is executed by the processor, the processor executes the light-weight and efficient parameter fine-tuning processing method of the large model according to any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed to implement the light-weight and efficient parameter fine-tuning processing method of the large model according to any one of claims 1-8.

Images