Model fine tuning method and device, storage medium and electronic equipment

Abstract

The invention discloses a fine adjustment method and device of a model, a storage medium and electronic equipment. The method comprises the steps that firstly, a weight matrix of a sparse model is obtained, then an adapter matrix and a sparse mask matrix are generated based on the weight matrix, the former enables the model to adapt to a new task through parameter adjustment, and the latter keeps a sparse structure of the model. Multiple rounds of training are carried out based on the matrix, in the training process, adapter matrix parameters are controlled to be adjusted based on gradients, weight matrix parameters are kept unchanged, a target sparse model adaptive to a target task is finally determined, video memory use is optimized, and training cost is reduced. According to the method and the device, the technical problem of relatively low model fine tuning efficiency caused by excessive model parameters due to model fine tuning in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a method and apparatus for fine-tuning a model, a storage medium, and an electronic device. Background Technology

[0002] Currently, large language models contain a large number of weight matrices, so training or fine-tuning these models often requires a lot of computational resources. Therefore, existing research has proposed a series of methods to save the resource cost of training models. For example, LoRA (Low-Rank Adaptation of Large Language Models) reduces the resource cost of model training by freezing the initial weight matrix and inserting a trainable low-rank matrix. However, when LoRA is applied to sparse large language models, it causes the sparse model to degenerate into a dense model, resulting in a geometric increase in model parameters, making it difficult to guarantee the efficiency of model fine-tuning.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a model fine-tuning method and apparatus, storage medium and electronic device to at least solve the technical problem in the related art that model fine-tuning can lead to excessive model parameters, resulting in low model fine-tuning efficiency.

[0005] According to one aspect of the embodiments of this application, a method for fine-tuning a model is provided, comprising: obtaining a weight matrix used by a sparse model; generating an adapter matrix based on the weight matrix, wherein the adapter matrix is ​​used to fine-tune the sparse model to adapt it to a target task; sharing the memory of the weight matrix with the adapter matrix, keeping the weight matrix unchanged, and in each round of training, regenerating a sparse mask matrix based on the weight matrix, maintaining the sparsity of the sparse model based on the sparse mask matrix, and training the model using the weight matrix and the adapter matrix until an adapter matrix that meets certain conditions is obtained; merging the adapter matrix that meets certain conditions into the sparse model to obtain a fine-tuned target sparse model, wherein the target sparse model represents a sparse model adapted to the target task.

[0006] According to another aspect of the embodiments of this application, a model fine-tuning apparatus is also provided, comprising: an acquisition module for acquiring a weight matrix used by a sparse model; a first generation module for generating an adapter matrix based on the weight matrix, wherein the adapter matrix is ​​used to fine-tune the sparse model to adapt it to a target task; a second generation module for sharing the memory of the weight matrix with the adapter matrix, keeping the weight matrix unchanged, regenerating a sparse mask matrix based on the weight matrix in each round of training, maintaining the sparsity of the sparse model based on the sparse mask matrix, and training the model using the weight matrix and the adapter matrix until an adapter matrix that meets the conditions is obtained; and a determination module for merging the adapter matrix that meets the conditions into the sparse model to obtain a fine-tuned target sparse model, wherein the target sparse model represents a sparse model adapted to the target task.

[0007] Optionally, the apparatus is configured to regenerate the sparse mask matrix based on the weight matrix by: obtaining the positions of non-zero elements in the weight matrix; generating the sparse mask matrix based on the number of rows and columns of the weight matrix and the positions of the non-zero elements, wherein the size of the sparse mask matrix is ​​the same as the size of the weight matrix.

[0008] Optionally, the apparatus is used to generate the sparse mask matrix based on the number of rows and columns of the weight matrix and the positions of the non-zero elements in the following manner: calculating the weight matrix using a pre-defined conditional expression and the number of rows and columns of the weight matrix to determine a Boolean matrix, wherein the positions of the non-zero elements are represented by the Boolean matrix; and performing a floating-point conversion operation on the Boolean matrix to obtain the sparse mask matrix, wherein the sparse mask matrix is ​​in the form of a floating-point matrix.

[0009] Optionally, the apparatus is configured to generate an adapter matrix based on the weight matrix in the following manner: obtaining the rank corresponding to the adapter matrix, where the rank is r; decomposing the weight matrix based on the rank to obtain a first adapter matrix and a second adapter matrix, wherein the size of the weight matrix is ​​N×M, the size of the first adapter matrix is ​​N×r, and the size of the second adapter matrix is ​​r×M, where r, M, and N are all positive integers, and r is less than M or N.

[0010] Optionally, the apparatus is used to merge the adapter matrices that meet the conditions into the sparse model to obtain a fine-tuned target sparse model by: acquiring training samples associated with the sparse model; generating a target weight matrix based on the weight matrix, the adapter matrix, and the sparse mask matrix, wherein the target weight matrix is ​​calculated using the memory corresponding to the weight matrix; determining the model output value through forward propagation based on the training samples and the target weight matrix; performing backpropagation based on the model output value and a preset loss function to determine the target gradient; and adjusting the parameters of the adapter matrix using the target gradient until the model output value meets the preset loss condition, thereby determining the target sparse model.

[0011] Optionally, the apparatus is configured to determine the model output value through forward propagation based on the training samples, the weight matrix, the adapter matrix, and the sparse mask matrix in the following manner: performing matrix multiplication on the first adapter matrix and the second adapter matrix to determine a first intermediate weight matrix, wherein the first intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix, and the adapter matrix includes the first adapter matrix and the second adapter matrix; performing element-wise multiplication on the first intermediate weight matrix and the sparse mask matrix to determine a second intermediate weight matrix, wherein the second intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix; determining a third intermediate weight matrix based on the matrix sum of the second intermediate weight matrix and the weight matrix, wherein the third intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix; and determining the model output value based on the model input value determined from the training samples and the third intermediate weight matrix, wherein the model output value and the sparse mask matrix are stored in target video memory, the target video memory including video memory allocated for the model output value and the sparse mask matrix respectively, and the sparse mask matrix is ​​allowed to be released after use.

[0012] Optionally, the apparatus is further configured to: generate a target computation graph based on the weight matrix, the adapter matrix, and the sparse mask matrix, wherein the target computation graph is used to indicate the computation process in each round of training; record the parameters associated with the model input values ​​in the target computation graph in a first video memory, and release them after the end of the round of training, wherein the target video memory includes the first video memory.

[0013] Optionally, the apparatus is configured to determine the target gradient by performing backpropagation based on the model output value and a preset loss function in the following manner: when the sparse mask matrix is ​​released, regenerate the sparse mask matrix based on the weight matrix; determine a first gradient based on the model output value, the preset loss function, and the target weight matrix, wherein the first gradient corresponds to the model output value, and the target gradient includes the first gradient; determine a second gradient based on the model output value, the model input value, and the sparse mask matrix, wherein the second gradient corresponds to the target weight matrix, and the target gradient includes the second gradient; determine a third gradient and a fourth gradient based on the second gradient, the first adapter matrix, and the second adapter matrix, wherein the third gradient corresponds to the first adapter matrix, the fourth gradient corresponds to the second adapter matrix, and the target gradient includes the third gradient and the fourth gradient.

[0014] Optionally, the apparatus is further configured to: after backpropagating based on the model output value and a preset loss function to determine the target gradient, in response to the determination of the target gradient, adjust the parameters of the adapter matrix using the target gradient; restore the target weight matrix to the weight matrix, perform the next round of training, until the model output value satisfies the preset loss condition, and determine the target sparse model.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which is configured to execute the fine-tuning method of the above-described model at runtime.

[0016] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the fine-tuning method as described above.

[0017] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the fine-tuning method of the model described above through the computer program.

[0018] This application's embodiments employ a method for obtaining the weight matrix used in a sparse model. By generating an adapter matrix and a sparse mask matrix based on the weight matrix, the goal of maintaining model sparsity while adapting the model to the target task is achieved, thus realizing the technical effects of parameter efficiency and computational resource conservation. First, obtaining the weight matrix provides the foundation for subsequent sparsification processing and adapter matrix generation. This step ensures that the model has the required sparse structure from the outset, laying the foundation for lightweight model design. Next, the adapter matrix is ​​generated based on the weight matrix. The adapter matrix's design allows the model to adapt to a specific target task through parameter fine-tuning without retraining the entire model's weights. This parameter adjustment method enables the model to quickly adapt to new tasks while maintaining its sparsity. Then, a sparse mask matrix is ​​generated based on the weight matrix. The sparse mask matrix maintains the model's sparsity during training, ensuring that non-critical weights are not activated, thereby maintaining the model's computational efficiency. Finally, the target sparse model is determined through multiple rounds of training based on the weight matrix, adapter matrix, and sparse mask matrix. In this process, the adapter matrix is ​​stored using the memory corresponding to the weight matrix, thus avoiding additional memory overhead. During training, the model parameters corresponding to the weight matrix remain unchanged, while the model parameters corresponding to the adapter matrix are adjusted based on the gradients generated in each training round. This training strategy allows the target sparse model to maintain its sparsity while adapting to the requirements of the target task, ultimately achieving rapid adaptation and performance optimization for new tasks while maintaining the sparse structure. This also solves the technical problem in related technologies where model fine-tuning leads to excessive model parameters, resulting in low fine-tuning efficiency. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a schematic diagram of the application environment for an optional model fine-tuning method according to an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating an optional model fine-tuning method according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of an optional model fine-tuning method according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0030] Figure 11 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0031] Figure 12 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0032] Figure 13 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0033] Figure 14 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0034] Figure 15 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0035] Figure 16 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0036] Figure 17 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0037] Figure 18 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0038] Figure 19 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0039] Figure 20 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0040] Figure 21 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0041] Figure 22 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0042] Figure 23 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0043] Figure 24 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0044] Figure 25 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0045] Figure 26 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0046] Figure 27 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application;

[0047] Figure 28 This is a schematic diagram of the structure of a fine-tuning device for an optional model according to an embodiment of this application;

[0048] Figure 29 This is a structural schematic diagram of a fine-tuned product of an optional model according to an embodiment of this application;

[0049] Figure 30 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0053] SP-LoRA: This application proposes a low-rank fine-tuning method for sparse large language models.

[0054] LoRA: A low-rank fine-tuning method for large language models.

[0055] Embedding: Language model embedding layer.

[0056] Decoder: Language model decoding layer.

[0057] Fine-tuning involves using a pre-trained model to customize the training for specific tasks and adjusting the network accordingly. The sparse model in this embodiment is a pre-trained model. After parameter tuning using the methods described in this embodiment, the target sparse model can be obtained to achieve the relevant tasks.

[0058] In the embodiments of this application, the fine-tuning process of the above model can be carried out through a pre-trained sparse model. The part of adjusting the model parameters involves artificial intelligence (AI), machine learning (ML) technology and natural language processing, and is designed based on feature extraction, machine learning and natural language processing in artificial intelligence.

[0059] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use computers or computer-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0060] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large models or foundational models, can be fine-tuned and widely applied to downstream tasks across various AI fields. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0061] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and pre-trained learning. Pre-trained models are the latest development in deep learning, integrating all of these techniques.

[0062] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP involves natural language—the language people use in daily life—and is closely related to linguistics; it also involves computer science and mathematics, and is a crucial technique for model training in artificial intelligence. Pre-trained models, which evolved from Large Language Models (LLMs) in NLP, can be widely applied to downstream tasks after fine-tuning. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0063] Specifically, a pretrained model (PTM) refers to a deep neural network (DNN) with a large number of parameters, trained on massive amounts of unlabeled data. Leveraging the function approximation capabilities of the DNN, the PTM extracts common features from the data. Through fine-tuning and Parameter Efficient Fine Tuning (PEFT) techniques, it is suitable for downstream tasks. Pretrained models are important tools for outputting Artificial Intelligence Generated Content (AIGC) and can also serve as a universal interface connecting multiple task-specific models.

[0064] The present application will be described below with reference to embodiments:

[0065] According to one aspect of the embodiments of this application, a model fine-tuning method is provided. Optionally, in this embodiment, the above-described model fine-tuning method can be applied to, for example... Figure 1 The hardware environment shown consists of server 101 and terminal device 103. For example... Figure 1 As shown, server 101 is connected to terminal 103 via a network and can be used to provide services to terminal devices or applications installed on terminal devices. The applications can be video applications, instant messaging applications, browser applications, educational applications, game applications, etc. Database 105 can be set up on the server or independently of the server to provide data storage services for server 101, such as a game data storage server. The network mentioned above can include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks, metropolitan area networks, and wide area networks. The wireless network includes Bluetooth, WIFI, and other networks that enable wireless communication. Terminal device 103 can be a terminal configured with an application, and can include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MID (Mobile Internet Devices), PADs, desktop computers, smart TVs, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, virtual reality (VR) terminals, augmented reality (AR) terminals, mixed reality (MR) terminals, and other computer devices. The server mentioned above can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0066] Combination Figure 1As shown, the fine-tuning method of the above model can be executed by an electronic device, which can be a terminal device or a server. The fine-tuning method of the above model can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0067] The above is merely an example, and this embodiment does not impose any specific limitations.

[0068] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the fine-tuning methods for the above model include:

[0069] S202, obtain the weight matrix used by the sparse model;

[0070] Optionally, in this embodiment, "obtaining" refers to acquiring the weight matrix of the sparse model through a method or process, including but not limited to obtaining it by training, loading a pre-trained model, or using random initialization. The sparse model refers to a model that has undergone sparsification, including but not limited to neural networks or language models that reduce model complexity by removing some weight connections. The weight matrix refers to the weight matrix possessed by the model after sparsification.

[0071] For example, in the field of deep learning, especially when dealing with large language models (LLMs), sparsification is often required to reduce the number of model parameters and computational resource consumption.

[0072] First, a pre-trained model is needed. This model can be randomly initialized or trained on a large amount of data. In practical applications, a pre-trained model is usually chosen because it has already demonstrated good performance on a specific task, and such a model is more likely to maintain high performance after sparsification.

[0073] Next, the weight matrix of this pre-trained model is analyzed to determine which weights can be sparsified. This typically involves sparsification techniques such as weight-based threshold pruning, Hessian matrix-based feature selection, or using regularization methods (e.g., L1 regularization) to push weights towards zero. These techniques can help identify weights that have a smaller impact on model performance, thereby reducing the number of model parameters without significantly impairing model performance.

[0074] After determining the weights that can be sparsified, a sparse weight matrix is ​​created by setting these weights to zero. This sparse weight matrix is ​​the weight matrix of the sparse model. In practical applications, this means that the model's storage and computational requirements will be significantly reduced while maintaining the model's core performance.

[0075] Finally, this sparse model can be used in various applications, such as natural language processing and image recognition, where model efficiency and performance are key considerations. This approach not only optimizes the model's resource consumption but also potentially improves its generalization ability by reducing model complexity.

[0076] In one exemplary embodiment, Figure 3 This is a schematic diagram of an optional model fine-tuning method according to an embodiment of this application, such as... Figure 3 As shown, taking a Natural Language Processing (NLP) application scenario as an example, especially in sentiment analysis tasks, it is necessary to build a model capable of identifying sentiment tendencies from text. The steps to obtain the weight matrix used by the sparse model in this process are as follows:

[0077] S1, Selection of pre-trained model: First, select a pre-trained BERT model that has been trained on a large-scale text corpus and has good language understanding capabilities.

[0078] S2 extracts the weight matrix from the BERT model. This matrix contains all the parameters of the model and is the basis for sparsification.

[0079] S3, Sparsification: By applying sparsification techniques such as L1 regularization, we identify and set the weights that have little impact on the sentiment analysis task to zero, thereby obtaining the weight matrix of the sparse model.

[0080] S4. Model Evaluation and Tuning: After sparsification, the model is evaluated to ensure that its performance on the sentiment analysis task has not significantly decreased. If necessary, the sparsification strategy is adjusted to balance model performance and sparsity.

[0081] In another exemplary embodiment, Figure 4 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 4 As shown, a lightweight convolutional neural network (CNN) model is constructed in the field of image recognition, especially in large-scale image classification tasks. The steps to obtain the weight matrix used by the sparse model include:

[0082] S1, Selection of pre-trained model: Select a ResNet model pre-trained on the ImageNet dataset, which has learned rich image features.

[0083] S2, Obtaining the weight matrix: Extract the weight matrix from the ResNet model. These weights are the basis for the model to recognize different image categories.

[0084] S3, Application of sparsity strategy: By using pruning techniques, such as feature selection of the Hessian matrix, remove those weights that have little impact on image classification to form a sparse weight matrix.

[0085] S4, Performance Validation and Optimization: After sparsification, the model undergoes rigorous performance testing to ensure its performance on the image classification task still meets requirements. Based on the test results, the degree of sparsification may need to be fine-tuned to achieve the optimal balance between performance and sparsity.

[0086] This embodiment allows us to obtain the weight matrix used in the sparse model. Through a carefully designed sparsification strategy, we can significantly reduce the computation and storage requirements of the model while maintaining its performance.

[0087] S204, Generate an adapter matrix based on the weight matrix, where the adapter matrix is ​​used to fine-tune the sparse model to adapt it to the target task;

[0088] Optionally, in this embodiment, generating an adapter matrix based on the weight matrix refers to using the weight matrix as a foundation to create additional parameter matrices to adjust and optimize the model, including but not limited to adding additional trainable layers to the neural network or introducing additional trainable parameters into existing layers. The adapter model refers to a model component designed to better adapt a sparse model to a specific target task, including but not limited to small network structures or parameter modules inserted into the model. These modules can be side networks attached to existing layers or parallel small networks. Their purpose is to adjust the model's behavior by fine-tuning these parameters to make it more suitable for new tasks, rather than retraining the entire model.

[0089] Optionally, in the embodiments of this application, the above-mentioned parameter adjustment refers to the process of optimizing the parameters in the adapter model, including but not limited to updating the weights in the adapter matrix through algorithms such as gradient descent, so as to adjust the output of the model to better meet the needs of the target task.

[0090] For example, in a deep learning project, there might be a sparse model that has been trained for a specific task. The weight matrix of this model contains a large number of parameters, but due to sparsification, some parameters have non-zero values. As project requirements change, this model needs to be adapted to new target tasks, such as shifting from image recognition to natural language processing.

[0091] To achieve this goal, an adapter matrix is ​​introduced. An adapter matrix is ​​a small, trainable parameter matrix designed to adjust the behavior of a sparse model by fine-tuning its parameters, adapting it to new tasks without altering the sparse model itself. This process is analogous to adding a highly flexible "adaptation layer" to the model.

[0092] Specifically, the generation and application of the adapter matrix can be divided into the following steps:

[0093] S1. Design the structure of the adapter matrix based on the characteristics of the new task (target task). For example, if the new task is text classification, an adapter matrix might be designed that contains small network structures capable of capturing text features.

[0094] S2. After the adapter matrix is ​​designed, it needs to be initialized. Typically, the parameters of the adapter matrix are initialized to small random values ​​to break the symmetry and provide a starting point for training.

[0095] S3. During training, the parameters of the adapter matrix are adjusted based on the data from the new task. This process involves calculating the loss function, then using backpropagation to compute the gradients of the adapter matrix parameters and updating these parameters.

[0096] S4. Parameter tuning of the adapter matrix is ​​an iterative process, typically requiring multiple forward and backward propagations for gradual optimization. In each iteration, the parameters of the adapter matrix are updated based on the gradients until the model achieves satisfactory performance on the new task.

[0097] S5, after the adapter matrix parameters are adjusted, it merges with the sparse model to form a new model. This new model retains the sparsity of the initial model while gaining the ability to adapt to new tasks through the adapter matrix.

[0098] S6. Finally, the new model will be evaluated on a validation set to ensure its performance meets expectations. If the performance is insufficient, it may be necessary to further adjust the parameters of the adapter matrix or redesign its structure.

[0099] Through the above process, a sparse model trained for a specific task can be effectively transformed into a model that can adapt to new tasks by introducing an adapter matrix and adjusting parameters. This saves resources that would otherwise be needed to retrain the entire model and improves the model's adaptability and flexibility.

[0100] S206: Share the memory of the weight matrix with the adapter matrix, keep the weight matrix unchanged, and regenerate the sparse mask matrix based on the weight matrix in each training round. Maintain the sparsity of the sparse model based on the sparse mask matrix. Train the model using the weight matrix and the adapter matrix until an adapter matrix that meets the conditions is obtained.

[0101] Optionally, in this embodiment, generating a sparse mask matrix based on the weight matrix refers to creating a new matrix from an existing weight matrix. This new matrix, the sparse mask matrix, is used to identify and maintain the positions of non-zero elements in the sparse model, including but not limited to determining which weights should be retained and which should be set to zero during weight pruning in a neural network. The sparse mask matrix is ​​a matrix with the same size as the weight matrix, and its elements are typically 0 or 1. It is used to control and maintain the sparse structure of the model, including but not limited to preventing some weight values ​​that have been set to zero from becoming non-zero again during weight updates.

[0102] For example, in a machine learning project, it might be necessary to sparsify a large neural network model to reduce its computational complexity and improve its operational efficiency. The following is the implementation flow based on an application scenario:

[0103] In a deep learning application, such as image recognition, it may be necessary to sparsify a convolutional neural network (CNN) model to reduce computational resource consumption. The following is the implementation process for this application scenario:

[0104] S1, First, start with a pre-trained CNN model that performs well on image recognition tasks, whose weight matrix contains a large number of parameters.

[0105] S2 determines the sparsity level of the model based on its performance and resource constraints. For example, it might be decided to maintain 80% sparsity, meaning only 20% of the weights are non-zero.

[0106] S3. Based on the determined sparsity level, determine the sparse mask matrix. This matrix has the same shape as the weight matrix, and its elements are 0 or 1, indicating which weights in the weight matrix should remain non-zero and which should be set to zero.

[0107] S4 uses a sparsification technique, such as weight pruning, to remove the weights that have the least impact on model performance and set the corresponding positions of these weights in the sparse mask matrix to 0.

[0108] S5. During model training and inference, a sparse mask matrix is ​​used to preserve sparsity. Only weights marked as 1 in the sparse mask matrix are included in the calculation, while those marked as 0 are excluded, thus maintaining the sparsity of the model.

[0109] In S6, the sparse mask matrix is ​​continuously applied throughout multiple training rounds to ensure that the sparse structure of the model is not destroyed. The non-zero weights of the model are updated using gradient descent, while the zero weights remain unchanged.

[0110] S7. During training, continuously evaluate the model's performance to ensure that sparsity processing does not lead to a significant performance degradation. If performance is affected, it may be necessary to adjust the sparsity level or retrain the model.

[0111] S8. Based on the evaluation results, it may be necessary to fine-tune the sparsity strategy, such as adjusting the sparse mask matrix, to find the optimal balance between sparsity and performance.

[0112] Once the model achieves the required sparsity while maintaining good performance, it can be deployed to target devices, such as mobile devices or embedded systems, to take advantage of their reduced computational requirements.

[0113] This process effectively reduces the number of model parameters, lowers the model's computational and storage requirements, while maintaining the model's high performance in image recognition tasks.

[0114] S208 merges the adapter matrices that meet the conditions into the sparse model to obtain the fine-tuned target sparse model, which represents the sparse model adapted to the target task.

[0115] Optionally, in the embodiments of this application, the aforementioned multi-round training based on the weight matrix, adapter matrix, and sparse mask matrix refers to utilizing these three matrices together in the model training process. This includes, but is not limited to, using iterative optimization algorithms within a deep learning framework to gradually adjust the parameters of the adapter matrix to achieve better model performance. The sparse mask matrix can control which parameters in the model participate in the calculation. For example, in the weight pruning of neural networks, the sparse mask matrix can identify those weights that have a small impact on the model output and set them to zero, thereby maintaining the sparsity of the model.

[0116] Optionally, in this embodiment, the aforementioned targeted sparse model refers to a model that, after training and optimization, can adapt to a specific task while maintaining a sparse structure. This includes, but is not limited to, natural language processing or image recognition tasks. Targeted sparse models can reduce the number of model parameters while maintaining high accuracy, thereby improving the model's operating efficiency. During multiple rounds of training, setting the model parameters corresponding to the weight matrix to remain unchanged means that the parameter values ​​in the weight matrix will not change during training. This includes, but is not limited to, in transfer learning scenarios, where the weight matrix may originate from a model pre-trained on a related task. These parameters are considered to be instructive for the new task and are therefore kept fixed during training.

[0117] It should be noted that the model parameters corresponding to the adapter matrix mentioned above are set to be adjusted based on the gradients generated in each training round. This means that during the training process, the parameters of the adapter matrix will be updated according to the gradients of these parameters based on the loss function, including but not limited to using gradient descent to optimize the parameters of the adapter matrix so that it can better adjust the model to adapt to new tasks.

[0118] Optionally, in the embodiments of this application, the above-mentioned target sparse model means that the sparse model adapted to the target task refers to the model that is finally trained and can not only complete the specific task, but also maintain sparsity. The target sparse model can reduce the consumption of computing resources while maintaining high recognition accuracy.

[0119] For example, in the field of natural language processing, especially in text classification tasks, first, a sparse large language model that has been trained on a large-scale dataset and performs well is prepared. The weight matrix of this model is the weight matrix. An adapter matrix is ​​designed to adjust the model to adapt to a new specific task, such as sentiment analysis or topic classification. The adapter matrix is ​​initialized and stored in the same memory space as the weight matrix to save resources. A sparse mask matrix is ​​created to identify which weights should remain unchanged (0) or can be updated (1) during training. This helps to maintain the sparsity of the model.

[0120] Then, Figure 5 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 5 As shown, the training process for a target sparse model can be implemented through the following steps:

[0121] S1, Start multi-round training: During training, the parameters of the weight matrix remain fixed, while the parameters of the adapter matrix are adjusted based on the gradients generated in each training round. In this way, the model can adapt to new tasks by fine-tuning the adapter matrix while keeping most of the parameters unchanged.

[0122] S2, Applying a sparse mask matrix: In each training round, a sparse mask matrix is ​​used to control the updates of the weights. Only the weights marked as 1 in the sparse mask matrix are updated according to the gradient, while the weights marked as 0 remain unchanged, which helps to maintain the sparse structure of the model.

[0123] S3, Calculate gradients and update adapter matrix: After each forward propagation and loss calculation, backpropagation is performed to calculate the gradients of the adapter matrix parameters, and the parameters of the adapter matrix are updated based on these gradients.

[0124] S4, Evaluate Model Performance: At each stage of training, evaluate the model's performance using a validation set to ensure that the model performs as expected on new tasks.

[0125] S5, Adjust training strategy: If the model performance does not meet expectations, it may be necessary to adjust the size of the adapter matrix, the learning rate, or the sparsity strategy of the sparse mask matrix, and continue training.

[0126] S6, Determining the Target Sparse Model: After multiple rounds of training and evaluation, the target sparse model is obtained when the model achieves satisfactory performance on the new task. This model not only adapts to the new task but also maintains sparsity, reducing the number of parameters and computational requirements.

[0127] In an exemplary embodiment, taking machine translation as an example, we first start with a pre-trained Transformer model, which has been trained on a large-scale bilingual corpus and possesses basic translation capabilities. The weight matrix of this model is the weight matrix itself. To adapt the Transformer model to a specific translation domain, such as medical literature translation, an adapter matrix is ​​introduced. The adapter matrix is ​​a small network structure attached to the self-attention layer of the Transformer model; it is designed to adjust the model's semantic understanding capabilities to better suit the needs of the new task.

[0128] Then, during training, the parameters of the Transformer model's weight matrix remain fixed, while the parameters of the adapter matrix are adjusted based on the gradients generated in each training round. A sparse mask matrix is ​​used to control weight updates; only weights marked as 1 in the sparse mask matrix are updated based on the gradient. To conserve memory, the adapter matrix is ​​designed to use the same memory space as the weight matrix, meaning that updating the adapter matrix's parameters does not consume additional memory at any time. During multiple training rounds, the model parameters corresponding to the weight matrix are kept constant, which helps maintain the model's stability on the translation task. The adapter matrix's parameters are adjusted based on the gradients generated in each training round, which helps the model quickly adapt to new translation domains. After multiple rounds of training and evaluation, when the model achieves satisfactory performance on the new task, the target sparse model is obtained. This model not only adapts to the new task but also maintains sparsity, reducing the number of parameters and computational requirements.

[0129] In another exemplary embodiment, taking facial expression recognition as an example in the field of image recognition, we first start with a pre-trained convolutional neural network (CNN) model, which has been trained on a large-scale face dataset and possesses basic facial feature recognition capabilities. The weight matrix of this model is the weight matrix of the CNN. To adapt the CNN model to the specific facial expression recognition task, an adapter matrix is ​​introduced. The adapter matrix is ​​a small network structure attached to the fully connected layers of the CNN model; it is designed to adjust the model's feature fusion capabilities to better suit the requirements of the new task. During training, the parameters of the CNN model's weight matrix remain fixed, while the parameters of the adapter matrix are adjusted based on the gradients generated in each training round. A sparse mask matrix is ​​used to control the updates of the weights; only weights marked as 1 in the sparse mask matrix are updated according to the gradient. To save memory resources, the adapter matrix is ​​designed to use the same memory space as the weight matrix, meaning that updating the adapter matrix parameters does not consume additional memory at any time. During multiple training rounds, the model parameters corresponding to the weight matrix are set to remain constant, which helps maintain the stability of the model on the facial expression recognition task. The parameters of the adapter matrix are adjusted based on the gradients generated in each training round, which helps the model quickly adapt to new facial expression recognition tasks. After multiple rounds of training and evaluation, when the model achieves satisfactory performance on the new task, the target sparse model is obtained. This model not only adapts to the new task but also maintains sparsity, reducing the number of parameters and computational requirements.

[0130] Through these two application scenarios, it can be seen that, whether in text classification or image recognition tasks, by introducing adapter matrices and sparse mask matrices, and combining them with weight matrices for multiple rounds of training, the model can be effectively adjusted and optimized to adapt to specific task requirements, while maintaining the sparsity and computational efficiency of the model.

[0131] In yet another exemplary embodiment, it can be applied to the training and fine-tuning of sparse models. Figure 6 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 6 As shown, in related technologies, full training of sparse models requires a large amount of computing resources. However, in the SP-LoRA-based sparse model training process, by using a training adapter, the computing resource requirements for producing sparse models can be significantly reduced compared to full training.

[0132] This application's embodiments employ a method for obtaining the weight matrix used in a sparse model. By generating an adapter matrix and a sparse mask matrix based on the weight matrix, the goal of maintaining model sparsity while adapting the model to the target task is achieved, thus realizing the technical effects of parameter efficiency and computational resource conservation. First, obtaining the weight matrix provides the foundation for subsequent sparsification processing and adapter matrix generation. This step ensures that the model has the required sparse structure from the outset, laying the foundation for lightweight model design. Next, the adapter matrix is ​​generated based on the weight matrix. The adapter matrix's design allows the model to adapt to a specific target task through parameter fine-tuning without retraining the entire model's weights. This parameter adjustment method enables the model to quickly adapt to new tasks while maintaining its sparsity. Then, a sparse mask matrix is ​​generated based on the weight matrix. The sparse mask matrix maintains the model's sparsity during training, ensuring that non-critical weights are not activated, thereby maintaining the model's computational efficiency. Finally, the target sparse model is determined through multiple rounds of training based on the weight matrix, adapter matrix, and sparse mask matrix. In this process, the adapter matrix is ​​stored using the memory corresponding to the weight matrix, thus avoiding additional memory overhead. During training, the model parameters corresponding to the weight matrix remain unchanged, while the model parameters corresponding to the adapter matrix are adjusted based on the gradients generated in each training round. This training strategy allows the target sparse model to maintain its sparsity while adapting to the requirements of the target task, ultimately achieving rapid adaptation and performance optimization for new tasks while maintaining the sparse structure. This also solves the technical problem in related technologies where model fine-tuning leads to excessive model parameters, resulting in low fine-tuning efficiency.

[0133] As an optional approach, a sparse mask matrix is ​​regenerated based on the weight matrix, including: obtaining the positions of non-zero elements in the weight matrix; generating a sparse mask matrix based on the number of rows and columns of the weight matrix and the positions of non-zero elements, wherein the size of the sparse mask matrix is ​​the same as the size of the weight matrix.

[0134] Optionally, in the embodiments of this application, obtaining the position of non-zero elements in the weight matrix refers to determining the specific position of non-zero values ​​in the weight matrix, including but not limited to finding the row and column coordinates of all non-zero weights in the parameter matrix of the neural network.

[0135] It should be noted that a sparse mask matrix is ​​a matrix with the same size as the weight matrix, used to mark which positions in the weight matrix are non-zero, so that the positions of these non-zero weights remain unchanged during subsequent training.

[0136] For example, Figure 7 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 7 As shown, the steps for generating a sparse mask matrix based on the weight matrix include, but are not limited to:

[0137] S1, Determine the weight matrix: Starting with a pre-trained neural network model, which may have been processed using some sparsification technique so that only a portion of the elements in its weight matrix are non-zero, this weight matrix is ​​called the weight matrix.

[0138] S2, Obtain the positions of non-zero elements: By traversing the weight matrix, record the row and column indices of all non-zero elements. This positional information will be used to generate the sparse mask matrix later.

[0139] S3, Initialize the sparse mask matrix: Create a new matrix with the same number of rows and columns as the weight matrix. This new matrix is ​​the sparse mask matrix. Initially, all elements of the sparse mask matrix are set to 0.

[0140] S4, Fill the sparse mask matrix: Based on the non-zero element position information obtained in step S2, set the corresponding element in the sparse mask matrix to 1. Thus, a position of 1 in the sparse mask matrix indicates that the corresponding weight in the weight matrix is ​​non-zero.

[0141] S5, Application of Sparse Mask Matrix: During model training and inference, the sparse mask matrix is ​​used to control the update of weights. Only the weights corresponding to the positions marked as 1 in the sparse mask matrix are included in the calculation and update, while the weights corresponding to the positions marked as 0 are ignored and remain at zero.

[0142] S6, Multi-round Training: In subsequent training iterations, the sparse mask matrix is ​​continuously applied to maintain the model's sparsity. In each iteration, only weights with a value of 1 in the sparse mask matrix are updated using optimization algorithms such as gradient descent. During training, the model's performance is periodically evaluated to ensure that sparsification does not negatively impact performance. If necessary, the sparse mask matrix can be adjusted, for example, by changing the sparsity level, to optimize the balance between performance and sparsity. After multiple rounds of training and evaluation, when the model achieves satisfactory performance on the new task, the target sparse model is obtained. This model not only adapts to the new task but also maintains sparsity, reducing the number of parameters and computational requirements.

[0143] By following the steps above, we can reduce the number of model parameters and computational complexity while maintaining model performance, making the model more suitable for deployment in resource-constrained environments.

[0144] As an alternative approach, a sparse mask matrix is ​​generated based on the number of rows and columns of the weight matrix and the positions of non-zero elements, including:

[0145] Using pre-defined conditional expressions and the number of rows and columns of the weight matrix, the weight matrix is ​​calculated to determine the Boolean matrix, where the positions of non-zero elements are represented by the Boolean matrix.

[0146] Performing a floating-point conversion on a Boolean matrix yields a sparse mask matrix, which is in the form of a floating-point matrix.

[0147] Optionally, in the embodiments of this application, the above-mentioned use of a pre-set conditional expression and the number of rows and columns of the weight matrix refers to evaluating the elements in the weight matrix by using a specific algorithm or rule, combined with the dimensional information of the weight matrix, including but not limited to using threshold pruning techniques to set a threshold conditional expression to determine which weight elements are important.

[0148] It should be noted that a Boolean matrix is ​​a matrix with only true values ​​(1) and false values ​​(0) used to represent the position of non-zero elements in a weight matrix. Including but not limited to the representation of sparse matrices, a Boolean matrix can accurately identify the row and column positions of all non-zero elements. For example, in the weight pruning of a neural network, a Boolean matrix can show which connections are activated.

[0149] The floating-point conversion operation on the Boolean matrix refers to converting the true and false values ​​in the Boolean matrix into floating-point numbers to obtain a sparse mask matrix. This includes, but is not limited to, converting the Boolean value 1 to the floating-point number 1.0 and the Boolean value 0 to the floating-point number 0.0. Such operations allow the sparse mask matrix to be used in subsequent mathematical operations and model training. For example, in deep learning frameworks, the sparse mask matrix can be used for masking operations during weight updates, ensuring that only the weights at positions with a value of 1.0 in the sparse mask matrix are updated.

[0150] For example, Figure 8 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 8 As shown, the steps for generating the Boolean matrix and sparse mask matrix based on the weight matrix include, but are not limited to, the following:

[0151] S1, Determine the weight matrix: Starting with a pre-trained neural network model, which may have been processed using some sparsification technique so that only a portion of the elements in its weight matrix are non-zero, this weight matrix is ​​called the weight matrix.

[0152] S2, Define the conditional expression: Define a conditional expression to evaluate each element in the weight matrix. This conditional expression can be based on the magnitude of the weight, activation value, or other relevant metrics. For example, a threshold can be set, and only weights greater than this threshold are considered important.

[0153] S3, Calculate the Boolean matrix: Evaluate each element in the weight matrix according to the conditional expression. If an element satisfies the condition, its corresponding position in the Boolean matrix is ​​marked as true (1); otherwise, it is marked as false (0). In this way, the Boolean matrix represents the position of the non-zero element in the weight matrix.

[0154] S4, Floating-point conversion operation: Convert the true values ​​(1) and false values ​​(0) in the Boolean matrix to floating-point numbers. Specifically, convert all true values ​​to the floating-point number 1.0 and all false values ​​to the floating-point number 0.0. In this way, a sparse mask matrix is ​​obtained, which is a floating-point matrix.

[0155] S5, Application of Sparse Mask Matrix: During model training and inference, the sparse mask matrix is ​​used to control the update of weights. Only the weights corresponding to positions with a value of 1.0 in the sparse mask matrix are included in the calculation and update, while the weights corresponding to positions with a value of 0.0 are ignored and remain at zero.

[0156] S6, Multi-round Training: In subsequent training iterations, the sparse mask matrix is ​​continuously applied to maintain the model's sparsity. In each iteration, only weights in the sparse mask matrix with a value of 1.0 are updated using optimization algorithms such as gradient descent. During training, the model's performance is periodically evaluated to ensure that sparsification does not negatively impact performance. If necessary, the conditional expressions can be adjusted, such as changing the threshold, to optimize the balance between performance and sparsity. After multiple rounds of training and evaluation, the target sparse model is obtained when it achieves satisfactory performance on the new task. This model not only adapts to the new task but also maintains sparsity, reducing the number of parameters and computational requirements.

[0157] It should be noted that when calculating the weight matrix using pre-defined conditional expressions, these expressions can be based on various standards or rules, such as the absolute value of the weights, their activation level, or their importance in a specific layer. The design of these conditional expressions can be customized according to the needs and performance objectives of different models, and this application does not impose any limitations on them. The number of rows and columns of the weight matrix determines the size of the Boolean matrix, and each element in the Boolean matrix corresponds to whether the weight at the corresponding position in the weight matrix is ​​non-zero. This representation can be flexibly applied to weight matrices of different sizes and types, and this application does not impose any limitations on it.

[0158] When performing floating-point conversion operations on a Boolean matrix to obtain a sparse mask matrix, the conversion rules can vary. For example, True can be converted to 1.0 and False to 0.0, or other numerical representations can be used, such as converting True to -1.0 and False to 0.0. The sparse mask matrix is ​​in the form of a floating-point matrix, meaning it can be easily used in mathematical operations, such as multiplication and addition, with other floating-point weight matrices in the model, thus playing a role in model training and inference. The specific application of the sparse mask matrix, such as whether it participates in gradient calculation or is used for weight updates, can be determined based on the specific needs of the model and the optimization objective; this application does not impose any limitations on this.

[0159] Through the embodiments of this application, the number of model parameters and computational complexity can be reduced while maintaining model performance, making the model more suitable for deployment in resource-constrained environments.

[0160] As an optional approach, the adapter matrix is ​​generated based on the weight matrix, including: obtaining the rank corresponding to the adapter matrix, where the rank is r; decomposing the weight matrix based on the rank to obtain a first adapter matrix and a second adapter matrix, where the size of the weight matrix is ​​N×M, the size of the first adapter matrix is ​​N×r, and the size of the second adapter matrix is ​​r×M, where r, M, and N are all positive integers, and r is less than M or N.

[0161] Optionally, in the embodiments of this application, obtaining the rank corresponding to the adapter matrix refers to determining the rank value of the adapter matrix, including but not limited to determining the minimum dimension number in the matrix decomposition process that a matrix can be decomposed into the product of two smaller matrices, and this dimension number is the rank.

[0162] Among them, rank-pair weight matrix decomposition refers to decomposing the weight matrix into two smaller matrices. This process includes, but is not limited to, using singular value decomposition (SVD) or other matrix decomposition techniques to decompose an N×M matrix into an N×r first adapter matrix and an r×M second adapter matrix, where r, M, and N are all positive integers, and r is less than M or N. Such decomposition helps to reduce the parameter complexity of the model while maintaining model performance.

[0163] For example, in the application scenarios of compression and acceleration of deep learning models, Figure 9 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 9 As shown, the steps for weight matrix decomposition based on the rank of the adapter matrix include, but are not limited to:

[0164] S1, Determine the rank of the adapter matrix: In the process of model optimization, the rank of the adapter matrix, denoted as r, needs to be determined first. This rank value represents how many singular values ​​are desired to approximate the original weight matrix in low-rank approximation. The choice of r directly affects the compression degree and performance of the model.

[0165] S2, Decompose the weight matrix: Given a weight matrix of size N×M, based on a determined rank r, use Singular Value Decomposition (SVD) or other matrix decomposition techniques to decompose the weight matrix into two smaller matrices. This process yields a first adapter matrix of size N×r and a second adapter matrix of size r×M.

[0166] S3, Applying the decomposition results: During model training and inference, these two adapter matrices are used instead of the original weight matrices. This decomposition can significantly reduce the number of model parameters, thereby reducing the model's storage requirements and computational complexity.

[0167] S4, Model Training Adjustment: During model training, the parameters of the first and second adapter matrices are updated based on the training data. Since r is much smaller than M or N, this reduces the number of model parameters, making the training process more efficient.

[0168] S5, Performance Evaluation: During training, periodically evaluate the model's performance to ensure that the decomposed model still achieves the expected accuracy. If performance degrades, it may be necessary to adjust the value of the rank r or redesign the adapter matrix structure.

[0169] S6, Model Deployment: Once the model achieves satisfactory performance on a new task, the compressed model can be deployed to resource-constrained environments, such as mobile devices or embedded systems, to take advantage of its reduced computational resource consumption.

[0170] It should be noted that when obtaining the rank of the adapter matrix, the choice of rank can be determined based on the specific needs and performance goals of the model. Different models and application scenarios may require different rank values, and this application does not impose any restrictions on this. The rank can be a parameter based on the model compression ratio, or it can be a parameter dynamically adjusted based on the model's performance feedback on a specific task; this application does not impose any restrictions on this either.

[0171] It should be noted that when decomposing the weight matrix based on the rank pair, the decomposition method can be singular value decomposition (SVD) or other matrix decomposition techniques, such as QR decomposition and LU decomposition. This application does not limit the choice of these decomposition techniques. The choice of these decomposition techniques may depend on the characteristics of the weight matrix, such as symmetry, sparsity, or other mathematical properties. This application does not limit the choice of these techniques. The size of the weight matrix N×M and the size of the adapter matrix N×r and r×M are determined based on the rank r and the model structure. The specific values ​​of these sizes can be flexibly varied according to different models and task requirements. This application does not limit the choice of these sizes. r, M, and N are all positive integers, and r is less than M or N. This condition ensures the effectiveness of the decomposition and the feasibility of model compression. This application does not limit the choice of these sizes.

[0172] The embodiments of this application demonstrate that while maintaining model performance, the number of model parameters and computational complexity can be reduced, making the model more suitable for deployment in resource-constrained environments. This approach is particularly suitable for application scenarios that have limited computing resources but require high-performance models.

[0173] As an alternative approach, the adapter matrices that meet the conditions are merged into the sparse model to obtain the fine-tuned target sparse model, including:

[0174] Obtain training samples associated with the sparse model;

[0175] The target weight matrix is ​​generated based on the weight matrix, adapter matrix, and sparse mask matrix. The target weight matrix is ​​calculated using the memory corresponding to the weight matrix.

[0176] The model output value is determined through forward propagation based on the training samples and the target weight matrix.

[0177] Backpropagation is performed based on the model output value and a preset loss function to determine the target gradient;

[0178] The parameters of the adapter matrix are adjusted using the target gradient until the model output value meets the preset loss condition, thus determining the target sparse model.

[0179] Optionally, in the embodiments of this application, obtaining training samples associated with sparse models refers to collecting and preparing a dataset for training sparse models, including but not limited to using a series of labeled images and their corresponding category labels in image recognition tasks.

[0180] Generating the target weight matrix based on the weight matrix, adapter matrix, and sparse mask matrix refers to combining the information from these three matrices to form a new weight matrix. This new matrix will be used in the forward and backward propagation processes of the model, including but not limited to adjusting the weight matrix through the adapter matrix in neural networks while maintaining sparsity using the sparse mask matrix, thereby generating the target weight matrix.

[0181] On the other hand, the above-mentioned determination of the model output value through forward propagation based on training samples and target weight matrix refers to using training samples as input to the model and using the target weight matrix for calculation to generate the model's predicted output for each input sample. This includes, but is not limited to, in natural language processing tasks, where the model generates the predicted output of the sequence based on the input text sequence and the target weight matrix.

[0182] Among them, backpropagation based on model output value and preset loss function to determine target gradient refers to calculating the error through loss function based on the difference between model's predicted output and true label, and determining the gradient of each parameter in target weight matrix through backpropagation algorithm, including but not limited to using cross-entropy loss function to calculate error gradient between model output and true class label in classification tasks.

[0183] On the other hand, the above-mentioned use of the target gradient to adjust the parameters of the adapter matrix until the model output value meets the preset loss condition, and the determination of the target sparse model refers to updating the parameters in the adapter matrix according to the calculated gradient, iterating this process until the model output value in the loss function reaches the preset optimization condition, including but not limited to continuously adjusting the adapter matrix parameters to minimize the loss function value during the training process, and finally determining a sparse model with satisfactory performance.

[0184] For example, in text classification tasks within the field of natural language processing, Figure 10 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 10 As shown, the training steps for the above-mentioned target sparse model include, but are not limited to:

[0185] S1, Obtain training samples associated with the sparse model: Extract a sample set from the dataset for training the sparse model. These samples include a series of text documents and their corresponding classification labels.

[0186] S2, Generate the target weight matrix based on the weight matrix, adapter matrix, and sparse mask matrix: The target weight matrix is ​​calculated by combining the information from the weight matrix, adapter matrix, and sparse mask matrix. In this step, the adapter matrix is ​​used to adjust the weight matrix, while the sparse mask matrix ensures that sparsity is maintained during the adjustment process. The calculation of the target weight matrix directly reuses the memory space of the weight matrix.

[0187] S3, determine the model output value through forward propagation based on training samples and target weight matrix: input the training samples into the model, use the target weight matrix to perform forward propagation calculation, and obtain the model's predicted output for each input sample, i.e., the classification result.

[0188] S4. Backpropagation is performed based on the model output value and the preset loss function to determine the target gradient: Based on the difference between the model's predicted output and the true label, the loss value is calculated using the preset loss function (e.g., cross-entropy loss), and the gradient of each parameter in the target weight matrix is ​​determined through the backpropagation algorithm.

[0189] S5, Adjust the parameters of the adapter matrix using the target gradient: Update the parameters in the adapter matrix based on the calculated target gradient. This step is repeated iteratively until the model's output value in the loss function reaches the preset optimization condition.

[0190] S6, Determine the target sparse model: Once the model output values ​​meet the preset loss conditions, the final determined target sparse model maintains sparsity while achieving satisfactory performance on the specific task. This model can be used for subsequent inference or further fine-tuning.

[0191] It should be noted that, firstly, when obtaining training samples for sparse model association, these samples can come from different datasets, including but not limited to public datasets, private datasets, or those obtained through specific experiments and observations; this application does not limit this. The types of samples can also be diverse; for example, in image recognition, they can be labeled images, and in speech recognition, they can be audio files with transcribed text.

[0192] Secondly, in the process of generating the target weight matrix based on the weight matrix, adapter matrix, and sparse mask matrix, the combination of these matrices can vary. For example, the adapter matrix can be inserted into the weight matrix in different ways, or the application strategy of the sparse mask matrix can be different; this application does not limit this. The calculation of the target weight matrix can also employ different algorithms or techniques, such as matrix factorization and gradient descent; this application does not limit this either.

[0193] Then, when determining the model output value through forward propagation based on the training samples and the target weight matrix, the specific implementation of forward propagation can adopt different neural network architectures, such as convolutional neural networks, recurrent neural networks, or transformer models; this application does not limit this. The type of model output value can also vary depending on the task requirements; for example, it may be a class probability distribution in classification tasks or a continuous value prediction in regression tasks.

[0194] Furthermore, when determining the target gradient through backpropagation based on the model output value and a preset loss function, the choice of loss function can be varied, including but not limited to mean squared error, cross-entropy loss, or a custom loss function; this application does not impose any limitations on this. The specific algorithm for backpropagation can also vary depending on the complexity of the model and the efficiency requirements of training; for example, standard gradient descent or more advanced optimization algorithms such as Adam or RMSprop can be used.

[0195] Finally, the parameters of the adapter matrix are adjusted using the target gradient until the model output value meets the preset loss condition. In determining the target sparse model, there are various strategies for parameter adjustment, including the selection of the learning rate, the frequency of parameter updates, or the use of an early stopping strategy; this application does not limit these. The preset loss condition can also be flexibly set according to the needs of the actual application, such as a specific loss threshold or a performance indicator on the validation set; this application does not limit these either.

[0196] In one exemplary embodiment, taking an image recognition application scenario as an example, Figure 11 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 11 As shown, the steps to implement a target sparse model in image recognition applications include, but are not limited to:

[0197] S1, Obtaining Training Samples for the Sparse Model: In image recognition tasks, training samples are a set of labeled image data, including images of various categories, such as cats, dogs, and cars. These images are used to train the sparse model so that the model can learn to distinguish different image categories.

[0198] S2 generates the target weight matrix based on the weight matrix, adapter matrix, and sparse mask matrix: the weight matrix is ​​obtained from the pre-trained model, the adapter matrix is ​​a newly added parameter matrix used to adjust the model to adapt to the new image recognition task, and the sparse mask matrix is ​​used to maintain the sparsity of the model. Combining these three matrices, the target weight matrix is ​​calculated, which will be used in the forward and backward propagation processes of the model.

[0199] S3, determine the model output value through forward propagation based on training samples and target weight matrix: input training samples into the model, use the target weight matrix to perform forward propagation calculation, and obtain the model's predicted output for each input image, that is, the predicted image category.

[0200] S4. Backpropagation is performed based on the model output value and the preset loss function to determine the target gradient: Based on the difference between the model's predicted output and the true label, the loss value is calculated using a loss function (such as cross-entropy loss), and the gradient of each parameter in the target weight matrix is ​​determined through the backpropagation algorithm.

[0201] S5, Adjust the parameters of the adapter matrix using the target gradient: Update the parameters in the adapter matrix based on the calculated target gradient. This step is repeated iteratively until the model's output value in the loss function reaches the preset optimization condition.

[0202] S6, Determine the target sparse model: When the model output value meets the preset loss condition, the final determined target sparse model maintains sparsity and achieves satisfactory performance in the image recognition task.

[0203] In another exemplary embodiment, taking the application scenario of speech recognition as an example, Figure 12 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 12 As shown, the steps to implement a target sparse model in speech recognition applications include, but are not limited to:

[0204] S1, Obtaining training samples for the sparse model: In speech recognition tasks, training samples are a series of audio clips with text transcriptions. These audio clips are used to train the sparse model so that the model can learn to recognize the corresponding text information from the audio signal.

[0205] S2 generates the target weight matrix based on the weight matrix, adapter matrix, and sparse mask matrix: the weight matrix comes from the pre-trained speech recognition model, the adapter matrix is ​​used to adjust the model to adapt to a specific dialect or accent, and the sparse mask matrix is ​​used to maintain the sparsity of the model. Combining these matrices, the target weight matrix is ​​calculated and used in the model's forward and backward propagation processes.

[0206] S3, determine the model output value through forward propagation based on training samples and target weight matrix: input the training samples into the model, use the target weight matrix to perform forward propagation calculation, and obtain the model's predicted output for each input audio segment, that is, the predicted text transcription.

[0207] S4. Backpropagation is performed based on the model output value and the preset loss function to determine the target gradient: Based on the difference between the model's predicted output and the real text, the loss value is calculated using a loss function (such as CTC loss), and the gradient of each parameter in the target weight matrix is ​​determined through the backpropagation algorithm.

[0208] S5, Adjust the parameters of the adapter matrix using the target gradient: Update the parameters in the adapter matrix based on the calculated target gradient. This step is repeated iteratively until the model's output value in the loss function reaches the preset optimization condition.

[0209] S6, Determine the target sparse model: When the model output value meets the preset loss condition, the final determined target sparse model maintains sparsity and achieves satisfactory performance in the speech recognition task.

[0210] The embodiments of this application demonstrate that while maintaining model performance, the number of model parameters and computational complexity can be reduced, making the model more suitable for deployment in resource-constrained environments, such as mobile devices or embedded systems. This approach is particularly suitable for application scenarios that have limited computing resources but require high-performance models.

[0211] As an optional approach, the model output value is determined through forward propagation based on training samples, weight matrix, adapter matrix, and sparse mask matrix, including:

[0212] Multiply the first adapter matrix and the second adapter matrix to determine the first intermediate weight matrix, wherein the first intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix, and the adapter matrix includes the first adapter matrix and the second adapter matrix.

[0213] The first intermediate weight matrix and the sparse mask matrix are multiplied element by element to determine the second intermediate weight matrix, which is stored in the memory corresponding to the weight matrix.

[0214] The third intermediate weight matrix is ​​determined based on the second intermediate weight matrix and the matrix sum of the weight matrices, wherein the third intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix.

[0215] The model output value is determined based on the model input value determined by the training samples and the third intermediate weight matrix. The model output value and the sparse mask matrix are stored in the target memory, which includes the memory allocated for the model output value and the sparse mask matrix respectively. The sparse mask matrix can be released after it has been used.

[0216] Optionally, in the embodiments of this application, the matrix multiplication of the first adapter matrix and the second adapter matrix refers to performing a multiplication operation between the two matrices to determine the first intermediate weight matrix, including but not limited to, in a deep learning model, calculating a temporary weight matrix, which is the result of adjusting the adapter matrix parameters.

[0217] Optionally, in the embodiments of this application, the first intermediate weight matrix refers to the result matrix obtained by multiplying the two adapter matrices, which is stored in the memory corresponding to the weight matrix. This includes, but is not limited to, saving the calculation result of the adapter matrix in the original weight storage space during the model parameter optimization process to reduce additional memory allocation.

[0218] It should be noted that the element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix mentioned above refers to multiplying corresponding elements of the two matrices to determine the second intermediate weight matrix. This includes, but is not limited to, applying the sparse mask matrix through element-wise multiplication in the sparsification process of the model to maintain the sparse structure of the model. For example, in the weight pruning of a neural network, the first intermediate weight matrix may contain some unnecessary connections, while the sparse mask matrix is ​​used to identify these connections. The second intermediate weight matrix obtained after element-wise multiplication retains only the important connections.

[0219] Optionally, in the embodiments of this application, the aforementioned second intermediate weight matrix refers to the result matrix after element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix, which is stored in the memory corresponding to the weight matrix. This includes, but is not limited to, saving the result after sparsification in the original weight storage space during the parameter update process of the model, so as to facilitate subsequent calculation and optimization.

[0220] It should be noted that the above-mentioned matrix based on the second intermediate weight matrix and the weight matrix and the determination of the third intermediate weight matrix refers to adding the second intermediate weight matrix and the weight matrix to obtain the third intermediate weight matrix. This includes, but is not limited to, combining the adjusted parameters of the adapter matrix and sparse mask matrix with the original model parameters in model training to generate the final weight matrix used for model inference.

[0221] Optionally, in the embodiments of this application, the aforementioned third intermediate weight matrix refers to the result matrix obtained by adding the second intermediate weight matrix and the weight matrix, which is stored in the memory corresponding to the weight matrix. This includes, but is not limited to, combining the adjusted parameters with the original parameters during the parameter update and inference process of the model to generate the final model parameters.

[0222] It should be noted that the above-mentioned model input value determined based on training samples and the model output value determined by the third intermediate weight matrix refers to using training samples as model input, combining the third intermediate weight matrix for forward propagation calculation, and obtaining the model's predicted output. This includes, but is not limited to, in the training process of deep learning, input data is used to obtain the prediction result through model calculation. For example, in image classification tasks, the input image is used to obtain the category prediction through model calculation.

[0223] Optionally, in this embodiment, storing the model output value and sparse mask matrix in the target GPU memory refers to storing the model's predicted output and sparse mask matrix in a specific storage area. The target GPU memory includes GPU memory allocated for the model output value and sparse mask matrix respectively, including but not limited to storing the model's output and the sparse mask matrix used for auxiliary calculation in the GPU's GPU memory during the inference process of the deep learning model to improve computational efficiency. Allowing the sparse mask matrix to be released after use means that after the calculation is completed, the sparse mask matrix can be released and recalculated when it is needed again to save storage resources.

[0224] For example, in the application scenario of deep learning model optimization, Figure 13 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 13 As shown, the steps for optimizing model weights based on the adapter matrix and sparse mask matrix described above may include, but are not limited to:

[0225] S1, perform matrix multiplication on the first adapter matrix and the second adapter matrix: During model optimization, the product of the first adapter matrix (size N×r) and the second adapter matrix (size r×M) is first calculated to obtain the first intermediate weight matrix (size N×M). This step is actually performing low-rank decomposition to reduce the number of model parameters.

[0226] S2, Determine the first intermediate weight matrix: Store the product result obtained in step S1 in the memory location corresponding to the weight matrix. This can effectively reuse memory space and reduce additional memory allocation.

[0227] S3, perform element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix: Multiply the first intermediate weight matrix and the sparse mask matrix element-wise to obtain the second intermediate weight matrix. The sparse mask matrix is ​​used to identify non-zero elements, preserving the sparsity of the model.

[0228] S4, Determine the second intermediate weight matrix: Store the result of element-wise multiplication obtained in step S3 in the memory location corresponding to the weight matrix, so that the second intermediate weight matrix inherits the memory space of the first intermediate weight matrix.

[0229] S5, Determine the third intermediate weight matrix based on the second intermediate weight matrix and the weight matrix: Add the second intermediate weight matrix to the weight matrix to obtain the third intermediate weight matrix. This step combines the adjustment of the adapter matrix and the information from the weight matrix to generate the final weight matrix used for model forward propagation.

[0230] S6, Determine the third intermediate weight matrix: Store the matrix and result obtained in step S5 in the memory location corresponding to the weight matrix. In this way, the third intermediate weight matrix becomes the actual weight matrix for the forward propagation of the model.

[0231] S7. Determine the model output value based on the model input value determined by the training samples and the third intermediate weight matrix: Use the training samples as input, combine them with the third intermediate weight matrix to perform forward propagation calculation, and obtain the model's predicted output.

[0232] S8, Model output values ​​and sparse mask matrix are stored in target memory: The model's predicted output and sparse mask matrix are stored in target memory, which is a separate memory area allocated for the model output values ​​and the sparse mask matrix. This improves data access speed and optimizes the model's computational performance.

[0233] S9. The sparse mask matrix can be released after it has been used: Once the sparse mask matrix has completed its role in the forward propagation process and its data has been used to calculate the model output value, the GPU memory it occupies can be released to make room for other computing tasks.

[0234] In an exemplary embodiment, taking text classification in natural language processing as an example, the steps include, but are not limited to, the following:

[0235] S1, perform matrix multiplication of the first adapter matrix and the second adapter matrix: In text classification tasks, the adapter matrix is ​​used to adjust the pre-trained word embedding matrix to adapt it to a specific classification task. The first adapter matrix may contain topic-related weights, while the second adapter matrix may contain context-related weights. Multiplying these two matrices yields the first intermediate weight matrix, which is stored in the memory corresponding to the weight matrix.

[0236] S2, perform element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix: multiplying the first intermediate weight matrix with the sparse mask matrix achieves sparsity of the model, retaining important feature connections and removing unimportant connections. The resulting second intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix.

[0237] S3. Determine the third intermediate weight matrix based on the second intermediate weight matrix and the weight matrix: Add the second intermediate weight matrix to the weight matrix to obtain the third intermediate weight matrix. This matrix combines the adjustment of the adapter matrix and the original model weights and is stored in the memory corresponding to the weight matrix.

[0238] S4. Determine the model output value based on the model input value determined by the training samples and the third intermediate weight matrix: Use the training samples (such as a series of labeled text data) as input, combine them with the third intermediate weight matrix for forward propagation, and obtain the model's predicted output value. The model output value and the sparse mask matrix are stored in the target memory.

[0239] S5. The sparse mask matrix can be released after it has been used: Once the sparse mask matrix has participated in the forward propagation calculation and the model output value has been determined, the GPU memory occupied by the sparse mask matrix can be released to make room for subsequent calculations.

[0240] In another exemplary embodiment, taking image recognition application scenarios in computer vision as an example, the steps include, but are not limited to, the following:

[0241] S1, perform matrix multiplication on the first adapter matrix and the second adapter matrix: In image recognition tasks, the adapter matrix is ​​used to adjust the weights of the convolutional neural network to adapt to the new recognition task. The first adapter matrix may contain weights related to feature extraction, while the second adapter matrix may contain weights related to the classifier. Multiplying these two matrices yields a first intermediate weight matrix, which is stored in the memory corresponding to the weight matrix.

[0242] S2, perform element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix: multiplying the first intermediate weight matrix with the sparse mask matrix achieves sparsity of the model, retaining important feature connections and removing unimportant connections. The resulting second intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix.

[0243] S3. Determine the third intermediate weight matrix based on the second intermediate weight matrix and the weight matrix: Add the second intermediate weight matrix to the weight matrix to obtain the third intermediate weight matrix. This matrix combines the adjustment of the adapter matrix and the original model weights and is stored in the memory corresponding to the weight matrix.

[0244] S4. Determine the model output value based on the model input value determined by the training samples and the third intermediate weight matrix: Use the training samples (such as a series of labeled image data) as input, combine them with the third intermediate weight matrix for forward propagation, and obtain the model's predicted output value. The model output value and the sparse mask matrix are stored in the target memory.

[0245] S5. The sparse mask matrix can be released after it has been used: Once the sparse mask matrix has participated in the forward propagation calculation and the model output value has been determined, the GPU memory occupied by the sparse mask matrix can be released to make room for subsequent calculations.

[0246] It should be noted that the application of the adapter matrix and the matrix multiplication operation can vary depending on the structure of different models and optimization objectives. For example, the adapter matrix can be used between different neural network layers or in different types of machine learning models, and this application does not limit this. The calculation of the first intermediate weight matrix is ​​not limited to the product of two adapter matrices, but can also be a combination of the adapter matrix and the weight matrix or other types of matrices, and this application does not limit this.

[0247] It should also be noted that during the element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix, the design and application of the sparse mask matrix can be diverse. For example, the sparse mask matrix can have different sparsity patterns or be used for different parameter filtering strategies, and this application does not limit this. The storage location of the second intermediate weight matrix can also be adjusted according to actual memory management and optimization needs, and is not necessarily limited to the memory corresponding to the weight matrix; this application does not limit this.

[0248] Furthermore, when determining the third intermediate weight matrix based on the second intermediate weight matrix and the weight matrix, the combination of these two matrices can vary depending on the specific needs and performance objectives of the model. For example, information can be fused through weighted summation or other mathematical operations, and this application does not impose any limitations on this. The storage of the third intermediate weight matrix can also be optimized based on actual computing resources and memory layout, and this application does not impose any limitations on this either.

[0249] Furthermore, in the process of determining the model output value based on the model input value determined by the training samples and the third intermediate weight matrix, the model input value can come from various types of data, such as images, text, or audio, and the calculation of the model output value can adopt different forward propagation strategies, which this application does not limit. During the storage of the model output value and the sparse mask matrix in the target GPU memory, the allocation and release strategy of the target GPU memory can be adjusted according to the specific computing platform and performance requirements. For example, the specific implementation of GPU memory management may differ in different hardware architectures or operating systems, which this application does not limit. The sparse mask matrix can be released after use to save GPU memory space. This strategy can be flexibly adjusted according to the actual GPU memory usage and the model's computational needs, which this application does not limit.

[0250] The embodiments of this application demonstrate that while maintaining model performance, the number of model parameters and computational complexity can be reduced, making the model more suitable for deployment in resource-constrained environments. This approach is particularly suitable for application scenarios that have limited computing resources but require high-performance models.

[0251] As an optional approach, the above method also includes:

[0252] A target computation graph is generated based on the weight matrix, adapter matrix, and sparse mask matrix, where the target computation graph is used to indicate the computation process in each round of training.

[0253] The parameters associated with the model input values ​​in the target computation graph are recorded in the first GPU memory and released after the training round ends. The target GPU memory includes the first GPU memory.

[0254] Optionally, in the embodiments of this application, the above-mentioned generation of the target computation graph based on the weight matrix, adapter matrix and sparse mask matrix refers to creating a detailed computation flowchart. This computation graph is used to indicate the computation steps and operations in each round of training, including but not limited to the mathematical operations and data flow defined by these matrices in deep learning training. For example, when training a neural network, the target computation graph will describe in detail all computation steps from the input layer to the output layer.

[0255] It should be noted that the target computation graph mentioned above is used to indicate the computation process in each round of training, including but not limited to how data flows through network layers and how gradients are calculated based on the loss function during forward and backward propagation in model training. For example, when training an image recognition model, the target computation graph will provide detailed instructions on how to process the input image data, how to apply convolutional layers, pooling layers, and fully connected layers, and how to calculate the loss based on the prediction results and the actual labels.

[0256] The above-mentioned recording of parameters associated with the model input values ​​in the target computation graph in the first video memory refers to storing parameters or intermediate calculation results directly related to the input data in a specific video memory area during model training. This video memory area is called the first video memory. This includes, but is not limited to, temporarily storing the preprocessing results or feature maps of the input data in the GPU's video memory when processing image or video data, so as to facilitate fast access and processing.

[0257] Furthermore, the aforementioned release after this training round refers to releasing unused GPU memory resources after the forward and backward propagation of each training round. This frees up space for the next training round or other computational tasks. For example, after completing an iteration, if some intermediate feature maps or parameters are no longer needed, the GPU memory they occupy can be released, thereby optimizing GPU memory usage efficiency. The target GPU memory includes the first GPU memory, which refers to all GPU memory resources required throughout the entire model training process, including the first GPU memory and other possible GPU memory regions. These GPU memory regions collectively support the model's training and inference computations.

[0258] For example, during the training of a deep learning model, the following is the implementation process for generating the target computation graph based on the weight matrix, adapter matrix, and sparse mask matrix, and managing GPU memory:

[0259] S1, Generating the Target Computation Graph: During the model training preparation phase, a target computation graph is constructed based on the weight matrix, adapter matrix, and sparse mask matrix. This computation graph details the mathematical operations and data flow during training, including the forward and backward propagation paths.

[0260] S2, Training Initialization: Before each training round, the parameters associated with the model input values, such as the input data and possible augmentation results, are loaded into the first memory. The first memory is part of the target memory and is specifically set up to store data directly related to the model input.

[0261] S3, Forward Propagation: Using the path indicated by the target computation graph, the input data is propagated forward through the model to calculate the model's predicted output. Intermediate results and parameters generated during these calculations are also temporarily stored in the first memory.

[0262] S4, Backpropagation and Parameter Update: Based on the difference between the predicted output and the true label, backpropagation is performed through the target computation graph to calculate the gradient and update the model parameters. The adapter matrix and sparse mask matrix play a role in adjusting and filtering the parameters in this step.

[0263] S5, Release GPU memory resources: After this round of training, parameters and intermediate results that are no longer needed are released from the first GPU memory to free up memory space for the next round of training or other computational tasks. This includes releasing parameters associated with the model input values, as well as any intermediate data generated during forward propagation.

[0264] S6, Iterative Loop: Repeat steps S3 to S5 for multiple rounds of training until the model converges or reaches the preset training period. After each round, the first memory is cleaned up and released to ensure efficient use of memory resources.

[0265] It should be noted that in the process of generating the target computation graph based on the weight matrix, adapter matrix, and sparse mask matrix, the source and specific form of these matrices can be diverse. For example, the weight matrix can be the parameters of a pre-trained model, the adapter matrix can be an additional parameter matrix added to adapt to a new task, and the sparse mask matrix can be a parameter matrix used to achieve model sparsity. This application does not limit this. The specific structure and complexity of the target computation graph can also vary depending on the model. It can contain different layers, activation functions, and connection patterns. This application does not limit this.

[0266] In the process of recording the parameters associated with the model input values ​​in the target computation graph in the first video memory, the specific implementation and technical details of the first video memory can vary. It can be GPU video memory, CPU memory, or other high-speed storage media, and this application does not limit it. The operation of releasing video memory resources after this round of training can also vary depending on the actual training framework and hardware environment. For example, in some cases, a delayed release strategy may be adopted to optimize performance, and this application does not limit it. The target video memory includes the first video memory, which means that in addition to the first video memory, there may be other video memory areas involved in computation and storage. Their configuration and management can also be adjusted according to actual needs, and this application does not limit them.

[0267] Through the embodiments of this application, the use of GPU memory resources can be optimized while ensuring model training efficiency. This method of managing GPU memory is particularly important when dealing with large-scale datasets or complex models.

[0268] As an optional approach, backpropagation is performed based on the model output value and a preset loss function to determine the target gradient, including:

[0269] When the sparse mask matrix is ​​released, the sparse mask matrix is ​​regenerated based on the weight matrix.

[0270] The first gradient is determined based on the model output value, the preset loss function, and the target weight matrix. The first gradient corresponds to the model output value, and the target gradient includes the first gradient.

[0271] The second gradient is determined based on the model output value, the model input value, and the sparse mask matrix. The second gradient corresponds to the target weight matrix, and the target gradient includes the second gradient.

[0272] The third and fourth gradients are determined based on the second gradient, the first adapter matrix, and the second adapter matrix, wherein the third gradient corresponds to the first adapter matrix, the fourth gradient corresponds to the second adapter matrix, and the target gradient includes the third gradient and the fourth gradient.

[0273] Optionally, in the embodiments of this application, the sparse mask matrix refers to a matrix with the same size as the weight matrix, whose elements are 0 or 1, used to identify which weights in the weight matrix should be activated or kept at zero value, including but not limited to the use of the sparse mask matrix to control the update of weights in the sparsification process of neural networks to maintain the sparse structure of the model.

[0274] It should be noted that the above determination of the first gradient based on the model output value, the preset loss function, and the target weight matrix refers to determining the gradient directly related to the model output by calculating the difference between the model output value and the actual value, and combining it with the preset loss function. This includes, but is not limited to, gradients calculated using the cross-entropy loss function in classification tasks. These gradients reflect the influence of the model output on the loss function.

[0275] Furthermore, the second gradient refers to the gradient corresponding to the target weight matrix. These are determined by considering the model output, model input, and sparse mask matrix, including but not limited to gradients calculated during backpropagation of the neural network based on the sensitivity of the loss function to the weights. These gradients are used to adjust the weights to optimize model performance. The third and fourth gradients refer to the gradients corresponding to the first and second adapter matrices. They are determined based on the second gradient, the first adapter matrix, and the second adapter matrix. These gradients include, but are not limited to, gradients calculated in multi-layer adapter structures based on the interactions between adapters and their contributions to the model output. These gradients guide the updating of adapter parameters to further optimize the model's adaptability and performance.

[0276] For example, in the training and optimization process of a deep learning model, Figure 14 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 14 As shown, the implementation process for gradient calculation and updating based on the sparse mask matrix and adapter matrix can include, but is not limited to, the following:

[0277] S1, Regenerate the sparse mask matrix: After the sparse mask matrix is ​​released (e.g., after one forward propagation iteration), it needs to be regenerated based on the weight matrix. This step is to maintain sparsity during model training. The generation of the sparse mask matrix can be based on the importance of the weights or other predetermined rules to determine which weights should be retained or set to zero.

[0278] S2, Determine the first gradient: Calculate the first gradient using the model's output, the pre-defined loss function, and the target weight matrix. The first gradient reflects the impact of the model's output on the loss function and is the first step in the backpropagation algorithm, used to adjust the model weights to reduce prediction error.

[0279] S3, Determine the second gradient: Calculate the second gradient by combining the model output value, model input value, and sparse mask matrix. The second gradient corresponds to the target weight matrix. It takes into account the impact of input data and model sparsity on weight updates, helping to further optimize the sparse structure of the model while maintaining model performance.

[0280] S4, Determine the third and fourth gradients: Based on the second gradient, the first adapter matrix, and the second adapter matrix, calculate the third and fourth gradients. The third gradient corresponds to the first adapter matrix, while the fourth gradient corresponds to the second adapter matrix. These two types of gradient calculations are used to update the parameters of the adapter matrices, enabling the model to adapt more effectively to the training data while maintaining or improving the model's sparsity.

[0281] It should be noted that the regeneration of the sparse mask matrix can be based on different sparsification strategies, such as the weight size, the output of the activation function, or other predetermined sparsity criteria; this application does not limit this. The calculation of the first gradient can vary depending on different loss functions and optimization objectives. For example, cross-entropy loss may be used in classification problems, while mean squared error loss may be used in regression problems; this application does not limit this. The determination of the second gradient can also be adjusted according to different model structures and input features. For example, the calculation method of the second gradient may differ between different neural network layers or in different feature maps; this application does not limit this. The determination of the third and fourth gradients can be based on the design and configuration of various adapter matrices. These adapter matrices can have different dimensions and parameter initialization methods, as well as different update rules and learning rates; this application does not limit this. The integration and use of the target gradient can be adjusted according to the requirements of different training stages and optimization algorithms. For example, in some cases, gradient normalization or the application of momentum terms may be performed; this application does not limit this.

[0282] The embodiments of this application allow for precise adjustment of model parameters in each iteration, optimizing model performance while maintaining model sparsity, which is particularly important for handling large-scale datasets and complex models. This approach is especially suitable for applications with limited computing resources but requiring high-performance models.

[0283] As an optional approach, after backpropagation based on the model output value and the preset loss function to determine the target gradient, the above method also includes: adjusting the parameters of the adapter matrix using the target gradient in response to the determination of the target gradient; restoring the target weight matrix to the weight matrix and performing the next round of training until the model output value meets the preset loss condition, thus determining the target sparse model.

[0284] Optionally, in the embodiments of this application, the above-mentioned response to the determination of the target gradient refers to the process of taking corresponding measures to update the model parameters after the target gradient is calculated, including but not limited to adjusting the parameters of the adapter matrix according to the gradient value calculated by the backpropagation algorithm in deep learning to optimize the performance of the model.

[0285] The above-mentioned use of the target gradient to adjust the parameters of the adapter matrix refers to updating the parameters in the adapter matrix according to the specific value of the target gradient. This includes, but is not limited to, adjusting the weights in the adapter matrix through gradient descent or its variants (such as the Adam optimizer) during the training of a neural network to reduce the loss function value of the model.

[0286] For example, restoring the target weight matrix to its original state as described above refers to resetting the target weight matrix, which has been adjusted by the adapter matrix, to its initial state after each round of training. This includes, but is not limited to, restoring the weight matrix to its state before training begins during the iteration of model training, so as to proceed with the next round of training.

[0287] It should be noted that conducting the next round of training refers to starting a new round of training with the updated model parameters after the current round of training is completed. This includes, but is not limited to, repeating the forward and backward propagation processes during model training until the model output value meets the preset loss condition.

[0288] Furthermore, the phrase "until the model output value meets the preset loss condition" refers to continuously training the model and updating the parameters until the model's output value in the loss function reaches a preset threshold or condition. This includes, but is not limited to, in classification tasks, until the model's accuracy reaches a certain standard, or in regression tasks, until the mean squared error between the predicted value and the actual value is lower than a certain set limit.

[0289] For example, in the training and optimization process of a deep learning model, Figure 15This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 15 As shown, including but not limited to the following steps:

[0290] S1, Determine the target gradient: During model training, the target gradient is calculated using the backpropagation algorithm. These gradients reflect the influence of the model parameters on the loss function. The calculation of the target gradient is based on the model output value, the preset loss function, and the current target weight matrix.

[0291] S2, Adjusting the Parameters of the Adapter Matrix Using the Target Gradient: Once the target gradient is determined, the next step is to use these gradients to update the parameters of the adapter matrix. The adapter matrix consists of trainable parameters introduced into the model to tune and optimize its performance. This process typically involves gradient descent or its variants, such as Adam or the RMSprop optimizer, to adjust the parameters in the adapter matrix and reduce the model's loss.

[0292] S3, Restore the Target Weight Matrix to the Target Weight Matrix: After each training round, in order to proceed with the next training round, the target weight matrix needs to be restored to the state of the target weight matrix. This step ensures that the model starts from the same parameter base at the beginning of each training round, allowing the model to learn new knowledge in each iteration.

[0293] S4, Proceed to the next round of training: Using the recovered weight matrix, begin a new round of training. This includes forward propagation to compute the model output, then calculating the loss function, followed by backpropagation to determine the new gradients and updating the parameters of the adapter matrix.

[0294] S5, Repeat training until the preset loss condition is met: This process will be repeated continuously until the model output value meets the preset loss condition. These conditions may be that the loss value drops below a specific threshold, or that the performance on the validation set reaches the expected target.

[0295] S6, Determine the target sparse model: When the model output values ​​meet the preset loss conditions, the final determined target sparse model maintains sparsity while achieving satisfactory performance on the specific task. This model can be used in practical applications such as image recognition, natural language processing, or other machine learning tasks.

[0296] By following the steps described above, the number of model parameters and computational complexity can be reduced while maintaining model performance, making the model more suitable for deployment in resource-constrained environments. This approach is particularly suitable for application scenarios that require high-performance models despite limited computing resources.

[0297] The following specific examples will further explain this application:

[0298] This application proposes a low-rank fine-tuning method, SP-LoRA, for sparse large language models. It can maintain the sparsity of the model while fine-tuning it. Based on this, this application further proposes a memory optimization method that combines gradient checkpointing and memory reuse to reduce the memory overhead of SP-LoRA.

[0299] This application is used for training and fine-tuning sparse models. Figure 16 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 16 As shown, the training process of a sparse model includes two steps: model sparsification and full training. Full training requires a large amount of computing resources, and after training, the sparse model may degenerate into a dense model. In contrast, the training process of a sparse model based on SP-LoRA requires training an adapter, which can significantly reduce the computing resource requirements for producing a sparse model compared to full training. At the same time, the trained model still maintains its sparsity.

[0300] It should be noted that SP-LoRA is a technique for efficiently fine-tuning large sparse language models. Its main purpose is to reduce the required computational resources and training time while maintaining model performance and sparsity.

[0301] First, let's review the computational process of large language models. Currently, large language models typically use the Transformer architecture. To process the input text, the model first segments the input text into words and then embeds the text into a vector space through an embedding layer.

[0302] x0 = Embedding(text) ∈ R L×d ,

[0303] Where L represents the sequence length, d represents the model feature dimension, and Embedding represents the language model embedding layer.

[0304] The embedded text then passes through a series of decoder layers to obtain the final hidden state x. N And through the final classification layer (LM) head The model output y is obtained:

[0305] x i =Decoder i (x i-1 ), i∈{1, 2,…,N}, y=LM head (x N )

[0306] Each decoder layer contains two modules: an Attention module and an FFN module. Assuming the model's input is x and its output is y, then:

[0307] y attn =x attn +Attention(RMSNorm(x attn )),

[0308] x ffn =y attn ,

[0309] y = x ffn +FFN(RMSNorm(x ffn ))

[0310] Where Attention represents the attention module of the language model. Figure 17 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 17 The diagram illustrates the computational flow of the Attention module in the decoder. FFN represents the multilayer perceptron module of the language model. Figure 18 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 18 The diagram illustrates the computation flow of the FFN module in the decoding layer, where RMSNorm represents the normalization layer. Figure 17 and Figure 18 The computational flow of the decoder layers is illustrated, where each decoder layer contains 7 weight matrices W. q W k W v W o W up W gate W down These weight matrices contain the vast majority of the parameters in the model. Figure 19 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 19 As shown, to reduce training overhead, related techniques employ the LoRA method, which freezes the weight matrix during training and trains the inserted adapter matrix. Since the adapter matrix contains far fewer parameters than the weight matrix, and SP-LoRA also updates the adapter matrix of the inserted model, SP-LoRA additionally records a mask to maintain the sparsity of the model.

[0311] It should be noted that SP-LoRA includes two parts: forward propagation and backward propagation. Consider an input X∈R to a certain weight matrix. C×L Then the forward propagation process can be expressed as the following expression: Y = {W + M⊙(A)} (t) ×B (t) )}X,

[0312] Among them, A(t) Let B represent the first adapter matrix mentioned above. (t) Let M represent the second adapter matrix, M represent the sparse mask matrix, and W represent the weight matrix.

[0313] The expression corresponds to the following calculation steps:

[0314] M = [W ≠ 0];

[0315] in These are the intermediate weight matrices generated during the calculation process, corresponding to the aforementioned first, second, and third intermediate weight matrices, respectively. The backpropagation process corresponding to this expression is as follows:

[0316]

[0317] in, They can be combined into the first gradient described above, where dX represents the second gradient described above, and dA... (t) This represents the third gradient mentioned above, dB. (t) This represents the fourth gradient mentioned above.

[0318] As can be seen from the above expression, during the forward propagation process, SP-LoRA needs to be an intermediate weight M. The output Y is allocated to video memory and contains 4RC+RL parameters. Simultaneously, the input activation X, intermediate weights M, and... It needs to be stored in the computation graph for backpropagation, containing CL+2RC parameters. Finally, during backpropagation, SP-LoRA needs to be used for the gradient. dX, dA (t) and dB (t) The video memory is allocated, which includes a total of 2RC+CL+rR+rC parameters.

[0319] Therefore, although SP-LoRA only requires training a small number of parameters, it still incurs certain memory overhead. To address this, a hybrid gradient checkpointing and memory reuse technique is proposed to optimize the memory consumption of SP-LoRA. Note that the intermediate weights M... and Although it consumes a significant amount of video memory, it involves only a small amount of computation. Therefore, instead of allocating additional video memory to store these intermediate weights, existing storage space can be used to store these intermediate variables, or these weights can be recalculated during backpropagation through recomputation. Figure 20 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 20 As shown, Algorithm 1 presents the pseudocode for the optimized SP-LoRA forward propagation. Figure 21This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 21 As shown, Algorithm 2 presents the pseudocode for the optimized SP-LoRA backpropagation.

[0320] Specifically, during the forward propagation process, the weight matrix W is first calculated. (t) And reuse the memory of the weight matrix W to store the result, such as Figure 20 As shown in the second row, during the backpropagation stage, the mask M is recalculated, as follows: Figure 21 The first line shown, followed by the calculation of adapter A. (t) B (t) And the gradient of the input activation X, such as Figure 21 Rows 2 through 5 are shown. Finally, from the weight matrix W... (t) Recover the initial weight matrix W, such as Figure 21 The sixth line shown. This method can effectively reduce the memory allocation during forward propagation to RC+RL parameters, and reduce the number of parameters recorded in the computation graph to CL.

[0321] Figure 22 This is a schematic diagram of another optional model fine-tuning method according to an embodiment of this application, such as... Figure 22 The diagram illustrates the optimized computational flow of SP-LoRA, corresponding to Algorithms 1 and 2. To evaluate the effectiveness of this patent across various models, several language models of different sizes were selected for testing, including Llama-2-7B, Llama-2-13B, and Llama-3-8B. For each model, the SparseGPT and Wanda pruning methods were used to obtain the sparse model corresponding to the dense model. Three datasets were considered for model fine-tuning: a pre-training dataset, an instruction-based fine-tuning dataset, and a domain dataset. The model was evaluated using the ARC-C, ARC-e, BoolQ, Hellaswag, OBQA, RTE, Winogrande, MT-Bench, GSM-8k, and Human-Eval datasets.

[0322] To demonstrate the advantages of this patent in terms of reliability and efficiency, several different and competitive baseline algorithms were selected for comparison, including LoRA and SPP.

[0323] First, the model was trained using a subset of the SlimPajama dataset containing 0 and 5B tokens. The experimental results are as follows. Figure 23 As shown.

[0324] The model was then trained using the instruction fine-tuning dataset Stanford Alpaca, and the experimental results are as follows. Figure 24As shown.

[0325] Next, the model was first trained using the pre-training dataset FineWe-Edu, then fine-tuned using Stanford Alpaca, and finally trained using a domain dataset. The experimental results are as follows. Figure 25 , 26 As shown.

[0326] To demonstrate the efficiency of the proposed method, tests were conducted on the memory usage and training speed of different methods. The experimental results are as follows: Figure 27 As shown, GC represents gradient checkpoints, and NO indicates that memory optimization is not used.

[0327] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0328] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0329] According to another aspect of the embodiments of this application, a fine-tuning apparatus for a model implementing the fine-tuning method of the above-described model is also provided. For example... Figure 28 As shown, the device includes:

[0330] Module 2802 is used to obtain the weight matrix used by the sparse model.

[0331] The first generation module 2804 is used to generate an adapter matrix based on the weight matrix, wherein the adapter matrix is ​​used to fine-tune the sparse model to adapt it to the target task.

[0332] The second generation module 2806 is used to share the memory of the weight matrix with the adapter matrix, keep the weight matrix unchanged, regenerate the sparse mask matrix according to the weight matrix in each round of training, keep the sparsity of the sparse model based on the sparse mask matrix, and train the model through the weight matrix and the adapter matrix until an adapter matrix that meets the conditions is obtained.

[0333] The determination module 2808 is used to merge the adapter matrices that meet the conditions into the sparse model to obtain the fine-tuned target sparse model, wherein the target sparse model represents a sparse model adapted to the target task.

[0334] As an alternative, the above-mentioned apparatus is used to regenerate the sparse mask matrix based on the weight matrix in the following manner: obtaining the positions of non-zero elements in the weight matrix; generating the sparse mask matrix based on the number of rows and columns of the weight matrix and the positions of the non-zero elements, wherein the size of the sparse mask matrix is ​​the same as the size of the weight matrix.

[0335] As an optional approach, the aforementioned apparatus is used to generate a sparse mask matrix based on the number of rows and columns of the weight matrix and the positions of non-zero elements in the following manner: the weight matrix is ​​calculated using a pre-defined conditional expression and the number of rows and columns of the weight matrix to determine a Boolean matrix, wherein the positions of non-zero elements are represented by the Boolean matrix; a floating-point conversion operation is performed on the Boolean matrix to obtain the sparse mask matrix, wherein the sparse mask matrix is ​​in the form of a floating-point matrix.

[0336] As an optional approach, the above-mentioned device is used to generate an adapter matrix based on a weight matrix in the following manner: obtaining the rank corresponding to the adapter matrix, where the rank is r; decomposing the weight matrix based on the rank to obtain a first adapter matrix and a second adapter matrix, where the size of the weight matrix is ​​N×M, the size of the first adapter matrix is ​​N×r, and the size of the second adapter matrix is ​​r×M, where r, M, and N are all positive integers, and r is less than M or N.

[0337] As an optional approach, the above-mentioned apparatus is used to merge the adapter matrices that meet the conditions into the sparse model to obtain the fine-tuned target sparse model in the following manner: obtaining training samples associated with the sparse model; generating a target weight matrix based on the weight matrix, adapter matrix, and sparse mask matrix, wherein the target weight matrix is ​​calculated using the memory corresponding to the weight matrix; determining the model output value through forward propagation based on the training samples and the target weight matrix; performing backpropagation based on the model output value and a preset loss function to determine the target gradient; adjusting the parameters of the adapter matrix using the target gradient until the model output value meets the preset loss condition, thereby determining the target sparse model.

[0338] As an optional approach, the aforementioned apparatus is used to determine the model output value through forward propagation based on training samples, a weight matrix, an adapter matrix, and a sparse mask matrix in the following manner: matrix multiplication of the first adapter matrix and the second adapter matrix is ​​performed to determine a first intermediate weight matrix, wherein the first intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix, and the adapter matrix includes the first adapter matrix and the second adapter matrix; element-wise multiplication of the first intermediate weight matrix and the sparse mask matrix is ​​performed to determine a second intermediate weight matrix, wherein the second intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix; a third intermediate weight matrix is ​​determined based on the matrix sum of the second intermediate weight matrix and the weight matrix, wherein the third intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix; and the model output value is determined based on the model input value determined from the training samples and the third intermediate weight matrix, wherein the model output value and the sparse mask matrix are stored in target GPU memory, the target GPU memory including GPU memory allocated for the model output value and the sparse mask matrix respectively, and the sparse mask matrix is ​​allowed to be released after use.

[0339] As an optional approach, the above-mentioned apparatus is also used to: generate a target computation graph based on the weight matrix, the adapter matrix, and the sparse mask matrix, wherein the target computation graph is used to indicate the computation process in each round of training; record the parameters associated with the model input values ​​in the target computation graph in a first video memory, and release them after the end of the round of training, wherein the target video memory includes the first video memory.

[0340] As an optional approach, the aforementioned apparatus is used to determine the target gradient by performing backpropagation based on the model output value and a preset loss function in the following manner: When the sparse mask matrix is ​​released, the sparse mask matrix is ​​regenerated based on the weight matrix; a first gradient is determined based on the model output value, the preset loss function, and the target weight matrix, wherein the first gradient corresponds to the model output value, and the target gradient includes the first gradient; a second gradient is determined based on the model output value, the model input value, and the sparse mask matrix, wherein the second gradient corresponds to the target weight matrix, and the target gradient includes the second gradient; a third and fourth gradient are determined based on the second gradient, the first adapter matrix, and the second adapter matrix, wherein the third gradient corresponds to the first adapter matrix, the fourth gradient corresponds to the second adapter matrix, and the target gradient includes both the third and fourth gradients.

[0341] As an optional approach, the above-mentioned device is also used for: performing backpropagation based on the model output value and a preset loss function to determine the target gradient; adjusting the parameters of the adapter matrix using the target gradient in response to the determination of the target gradient; restoring the target weight matrix to the weight matrix; performing the next round of training until the model output value meets the preset loss condition, thereby determining the target sparse model.

[0342] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0343] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0344] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program.

[0345] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0346] Figure 29 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0347] It should be noted that, Figure 29 The computer system 2900 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0348] like Figure 29 As shown, the computer system 2900 includes a central processing unit (CPU) 2901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 2902 or programs loaded from storage section 2908 into random access memory (RAM) 2903. The RAM 2903 also stores various programs and data required for system operation. The CPU 2901, ROM 2902, and RAM 2903 are interconnected via a bus 2904. An input / output interface 2905 (I / O interface) is also connected to the bus 2904.

[0349] The following components are connected to the input / output interface 2905: an input section 2906 including a keyboard, mouse, etc.; an output section 2907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 2908 including a hard disk, etc.; and a communication section 2909 including a network interface card such as a local area network card, modem, etc. The communication section 2909 performs communication processing via a network such as the Internet. A drive 2910 is also connected to the input / output interface 2905 as needed. Removable media 2911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on the drive 2910 as needed so that computer programs read from them can be installed into the storage section 2908 as needed.

[0350] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 2909, and / or installed from removable medium 2911. When the computer program is executed by central processing unit 2901, it performs various functions defined in the system of this application.

[0351] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 2909, and / or installed from removable media 2911. When the computer program is executed by central processing unit 2901, it performs various functions provided in the embodiments of this application.

[0352] According to another aspect of the embodiments of this application, an electronic device for implementing the fine-tuning method of the above-described model is also provided, the electronic device being... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 30 As shown, the electronic device includes a memory 3002 and a processor 3004. The memory 3002 stores a computer program, and the processor 3004 is configured to execute the steps in any of the above method embodiments via the computer program.

[0353] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0354] Optionally, in this embodiment, the processor may be configured to execute the methods in the embodiments of this application via a computer program.

[0355] Alternatively, as those skilled in the art will understand, Figure 30 The structure shown is for illustrative purposes only. Figure 30 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 30 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 30 The different configurations shown.

[0356] The memory 3002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the model fine-tuning method and apparatus in this embodiment. The processor 3004 executes various functional applications and data processing by running the software programs and modules stored in the memory 3002, thereby implementing the aforementioned model fine-tuning method. The memory 3002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 3002 may further include memory remotely located relative to the processor 3004, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 3002 may be used, but is not limited to, to store information such as weight matrices. As an example, such as... Figure 30 As shown, the memory 3002 may include, but is not limited to, the acquisition module 2802, the first generation module 2804, the second generation module 2806, and the determination module 2808 in the fine-tuning device of the model described above. Furthermore, it may include, but is not limited to, other module units in the fine-tuning device of the model described above, which will not be elaborated upon in this example.

[0357] Optionally, the transmission device 3006 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 3006 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 3006 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0358] In addition, the aforementioned electronic device also includes: a display 3008 for displaying the aforementioned weight matrix; and a connection bus 3010 for connecting the various module components in the aforementioned electronic device.

[0359] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0360] According to one aspect of this application, a computer-readable storage medium is provided, from which a processor of an electronic device reads computer instructions, and the processor executes the computer instructions, causing the electronic device to perform a fine-tuning method of the model provided in various alternative implementations of the fine-tuning aspect of the above-described model.

[0361] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store methods for performing the embodiments of this application.

[0362] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0363] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0364] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0365] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0366] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0367] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0368] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0369] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for fine-tuning a model, characterized in that, include: Obtain the weight matrix used by the sparse model; An adapter matrix is ​​generated based on the weight matrix, wherein the adapter matrix is ​​used to fine-tune the sparse model to adapt it to the target task; The memory of the weight matrix is ​​shared with the adapter matrix, while the weight matrix remains unchanged. In each round of training, a sparse mask matrix is ​​regenerated based on the weight matrix. The sparsity of the sparse model is maintained based on the sparse mask matrix. The model is trained using the weight matrix and the adapter matrix until an adapter matrix that meets the conditions is obtained. The adapter matrices that meet the conditions are merged into the sparse model to obtain the fine-tuned target sparse model, which represents a sparse model adapted to the target task.

2. The method according to claim 1, characterized in that, Regenerating the sparse mask matrix based on the weight matrix includes: Obtain the positions of the non-zero elements in the weight matrix; The sparse mask matrix is ​​generated based on the number of rows and columns of the weight matrix and the positions of the non-zero elements, wherein the size of the sparse mask matrix is ​​the same as the size of the weight matrix.

3. The method according to claim 2, characterized in that, The process of generating the sparse mask matrix based on the number of rows and columns of the weight matrix and the positions of the non-zero elements includes: Using a pre-defined conditional expression and the number of rows and columns of the weight matrix, the weight matrix is ​​calculated to determine a Boolean matrix, wherein the positions of the non-zero elements are represented by the Boolean matrix; The Boolean matrix is ​​converted to a floating-point number to obtain the sparse mask matrix, wherein the sparse mask matrix is ​​in the form of a floating-point matrix.

4. The method according to claim 1, characterized in that, The process of generating the adapter matrix based on the weight matrix includes: Obtain the rank corresponding to the adapter matrix, where the rank is r; The weight matrix is ​​decomposed based on the rank to obtain a first adapter matrix and a second adapter matrix, wherein the weight matrix has a size of N×M, the first adapter matrix has a size of N×r, and the second adapter matrix has a size of r×M, where r, M, and N are all positive integers, and r is less than M or N.

5. The method according to claim 1, characterized in that, The step of merging the adapter matrices that meet the conditions into the sparse model to obtain the fine-tuned target sparse model includes: Obtain the training samples associated with the sparse model; A target weight matrix is ​​generated based on the weight matrix, the adapter matrix, and the sparse mask matrix, wherein the target weight matrix is ​​calculated using the memory corresponding to the weight matrix; The model output value is determined through forward propagation based on the training samples and the target weight matrix. Backpropagation is performed based on the model output value and the preset loss function to determine the target gradient; The parameters of the adapter matrix are adjusted using the target gradient until the model output value meets the preset loss condition, thereby determining the target sparse model.

6. The method according to claim 5, characterized in that, The process of determining the model output value through forward propagation based on the training samples, the weight matrix, the adapter matrix, and the sparse mask matrix includes: Multiply the first adapter matrix and the second adapter matrix to determine the first intermediate weight matrix, wherein the first intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix, and the adapter matrix includes the first adapter matrix and the second adapter matrix; The first intermediate weight matrix and the sparse mask matrix are multiplied element-wise to determine the second intermediate weight matrix, wherein the second intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix; A third intermediate weight matrix is ​​determined based on the second intermediate weight matrix and the matrix sum of the weight matrices, wherein the third intermediate weight matrix is ​​stored in the memory corresponding to the weight matrix; The model output value is determined based on the model input value determined by the training samples and the third intermediate weight matrix, wherein the model output value and the sparse mask matrix are stored in the target video memory, the target video memory includes video memory allocated for the model output value and the sparse mask matrix respectively, and the sparse mask matrix is ​​allowed to be released after being used.

7. The method according to claim 6, characterized in that, The method further includes: A target computation graph is generated based on the weight matrix, the adapter matrix, and the sparse mask matrix, wherein the target computation graph is used to indicate the computation process in each round of training; The parameters associated with the model input values ​​in the target computation graph are recorded in the first GPU memory and released after the training round ends, wherein the target GPU memory includes the first GPU memory.

8. The method according to claim 5, characterized in that, The backpropagation based on the model output value and the preset loss function to determine the target gradient includes: If the sparse mask matrix is ​​released, the sparse mask matrix is ​​regenerated based on the weight matrix; A first gradient is determined based on the model output value, the preset loss function, and the target weight matrix, wherein the first gradient corresponds to the model output value, and the target gradient includes the first gradient. The second gradient is determined based on the model output value, the model input value, and the sparse mask matrix, wherein the second gradient corresponds to the target weight matrix, and the target gradient includes the second gradient; The third and fourth gradients are determined based on the second gradient, the first adapter matrix, and the second adapter matrix, wherein the third gradient corresponds to the first adapter matrix, the fourth gradient corresponds to the second adapter matrix, and the target gradient includes the third gradient and the fourth gradient.

9. The method according to claim 5, characterized in that, After determining the target gradient through backpropagation based on the model output value and a preset loss function, the method further includes: In response to the determination of the target gradient, the parameters of the adapter matrix are adjusted using the target gradient; The target weight matrix is ​​restored to the original weight matrix, and the next round of training is performed until the model output value meets the preset loss condition, thus determining the target sparse model.

10. A fine-tuning device for a model, characterized in that, include: The acquisition module is used to obtain the weight matrix used by the sparse model. The first generation module is used to generate an adapter matrix based on the weight matrix, wherein the adapter matrix is ​​used to fine-tune the sparse model to adapt it to the target task. The second generation module is used to share the memory of the weight matrix with the adapter matrix, keep the weight matrix unchanged, regenerate the sparse mask matrix according to the weight matrix in each round of training, keep the sparsity of the sparse model based on the sparse mask matrix, and train the model through the weight matrix and the adapter matrix until an adapter matrix that meets the conditions is obtained. A determination module is used to merge the adapter matrices that meet the conditions into the sparse model to obtain a fine-tuned target sparse model, wherein the target sparse model represents a sparse model adapted to the target task.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 9.

13. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 9 through the computer program.

Images