Method, device and equipment for fine tuning of model, medium and product

Abstract

The embodiment of the invention provides a method, device and equipment for fine tuning of a model, a storage medium and a computer program product. The method includes determining a first gradient set for a first coefficient set in a low-rank adaptation table of a model according to an object sample set and a corresponding recommendation resource set, the low-rank adaptation table indicating fine tuning of basic coefficients associated with object features in the model, recommendation resources in the recommendation resource set are recommended to object samples in the object sample set. The method further includes resizing a low rank adaptation table based on the first set of gradients. The method further includes updating a low rank adaptation table based on the first set of gradients and the adjusted size. According to the method disclosed by the embodiment of the invention, the size of the low-rank adaptation table can be dynamically adjusted based on the object sample used for training, so that the dynamically adjusted low-rank adaptation table not only can reserve enough update information, but also can reduce redundant data.

Description

Technical Field

[0001] This disclosure generally relates to the field of computers, and more specifically to methods, apparatus, devices, computer-readable storage media, and computer program products for fine-tuning models. Background Technology

[0002] With the rapid evolution of machine learning technology, foundational models (i.e., large-scale machine learning models pre-trained on massive datasets) have become the core supporting architecture for many fields such as natural language processing, computer vision, and speech recognition. These models typically have a large number of parameters and can be fine-tuned to adapt to various downstream tasks, thereby achieving customized requirements and meeting application standards for specific scenarios.

[0003] Low-Rank Adaptation (LoRA) is based on the redundancy of model parameters. It has been found that when adapting to a specific task, a low-dimensional subspace exists in the parameter space that captures task features; efficient adaptation can be achieved by adjusting the parameters of this subspace. In LoRA, the basic coefficients of the pre-trained model can be frozen to prevent the destruction of general knowledge. A bypass structure consisting of low-rank matrices A (dimension d×r, e.g., r=16, d=4096) and B (dimension r×d) is injected alongside the basic coefficients, where the rank r is much smaller than the hidden layer dimension d. During training, only the parameters of these two low-rank matrices are updated. After training, B·A is obtained through matrix multiplication and superimposed with the basic coefficients to form fine-tuned coefficients, thus reducing the size of the coefficients. Summary of the Invention

[0004] According to exemplary embodiments of this disclosure, a method, apparatus, device, computer storage medium, and computer program product for fine-tuning a model are provided.

[0005] In a first aspect of this disclosure, a method for fine-tuning a model is provided. The method includes determining a first gradient set for a first set of coefficients in a low-rank fitness table of the model, based on a set of object samples and a corresponding set of recommended resources. The low-rank fitness table indicates fine-tuning of the underlying coefficients in the model associated with object features. Recommended resources in the recommended resource set are recommended to object samples in the object sample set. The method further includes adjusting the size of the low-rank fitness table based on the first gradient set. The method also includes updating the low-rank fitness table based on the first gradient set and the adjusted size.

[0006] In a second aspect of this disclosure, an apparatus for fine-tuning a model is provided. The apparatus includes a determining module configured to determine a first gradient set for a first set of coefficients in a low-rank fitness table of the model, based on a set of object samples and a corresponding set of recommended resources. The low-rank fitness table indicates fine-tuning of the underlying coefficients in the model associated with object features, and recommended resources in the recommended resource set are recommended to object samples in the object sample set. The apparatus also includes an adjusting module configured to adjust the size of the low-rank fitness table based on the first gradient set. The apparatus further includes an updating module configured to update the low-rank fitness table based on the first gradient set and the adjusted size.

[0007] In a third aspect of this disclosure, an electronic device is provided, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method described in the first aspect of this disclosure when executed by the at least one processing unit.

[0008] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform the method described in the first aspect of this disclosure.

[0009] In a fifth aspect of this disclosure, a computer program product is provided, including computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method described in the first aspect of this disclosure.

[0010] The summary section is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0012] Figure 2 A flowchart of a method for fine-tuning a model according to an embodiment of the present disclosure is shown;

[0013] Figure 3 A schematic diagram of an inference path and a training path according to an embodiment of the present disclosure is shown;

[0014] Figure 4A A schematic diagram illustrating the adjustment of the length of a low-rank adaptation table according to an embodiment of the present disclosure is shown;

[0015] Figure 4B A schematic diagram illustrating the adjustment of the width of a low-rank adaptation table according to an embodiment of the present disclosure is shown;

[0016] Figure 5 A schematic diagram illustrating the adjustment of the size of a low-rank adaptation table according to an embodiment of the present disclosure is shown;

[0017] Figure 6 A schematic diagram of an adjustment processing unit according to an embodiment of the present disclosure is shown;

[0018] Figure 7 A schematic block diagram of an example apparatus according to some embodiments of the present disclosure is shown; and

[0019] Figure 8 A block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0020] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0021] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information. It is understood that before using the technical solutions disclosed in the embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0022] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message. As an optional but non-limiting implementation, the prompt message can be sent to the user in the form of a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0026] In related techniques, if low-rank adaptation is used to fine-tune the model, the ranks of the low-rank adaptation matrices A and B in the low-rank adaptation table need to be manually set. After setting, the fine-tuning process will maintain the rank and adjust the values ​​of the low-rank adaptation matrices according to the loss. However, when the model size is large and there are many parameters, the low-rank adaptation table still occupies a large space and requires a lot of computational resources, making the model fine-tuning process very time-consuming, which in turn leads to problems such as low update frequency and poor accuracy of the low-rank adaptation table.

[0027] To address this, this disclosure proposes a method for fine-tuning a model. This method determines a first gradient set based on a set of object samples and corresponding recommendation resources. Before updating the coefficients in the low-rank fitness table, the method adjusts the size of the low-rank fitness table, for example, by adjusting its length and / or width, based on this first gradient set. This method combines the first gradient set with the adjusted size of the low-rank fitness table to update the low-rank fitness table, thereby achieving model fine-tuning. This method can dynamically adjust the size of the low-rank fitness table based on the object samples used for training, ensuring that the dynamically adjusted low-rank fitness table retains sufficient update information while reducing redundant data.

[0028] The embodiments of this disclosure will now be described in further detail with reference to the accompanying drawings, wherein... Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Example environment 100 includes a server, which may be deployed with a model 120 (e.g., a multimodal model capable of processing multimodal data, a deep recommendation model). In this embodiment, the server performs the methods of embodiments of the present disclosure.

[0029] In some embodiments, the server can determine a first gradient set based on the first coefficient set in the low-rank fitness table of the model, according to the object sample set and the corresponding recommendation resource set. The low-rank fitness table indicates fine-tuning of the base coefficients associated with object features in the model, and the recommendation resources in the recommendation resource set are recommended to the object samples in the object sample set. Taking object sample 110 in the object sample set as an example, during training, the server can input object sample 110 into model 120, where it is encoded as an initial feature. In some embodiments, the embedding layer 122 of model 120 includes base coefficients 124 and low-rank fitness coefficients 126. The initial feature is subjected to a matrix operation with the base coefficients 124 to obtain a first value, and the initial feature is subjected to a matrix operation with the low-rank fitness coefficients 126 to obtain a second value. The first value and the second value are added together to obtain the object embedding feature of object sample 110 in model 120. In some embodiments, this object embedding feature can be passed to other network layers, such as fusion layers, interaction layers, prediction layers 128, etc.

[0030] To facilitate the description of the core principles, this embodiment does not introduce too many network layers, but those skilled in the art can apply the method of the embodiments of this disclosure to various models with different architectures. In this embodiment, the object embedding feature is passed to the prediction layer 128 of model 120. The prediction layer 128 makes predictions based on the object embedding feature, outputs the recommendation probability of each resource through an activation function, and determines the predicted resource 130 from multiple resources according to the probability sorting of object samples 110.

[0031] In some embodiments, the server can calculate the loss based on the predicted resource 130 and the real resource 132 (i.e., the corresponding label), and calculate the gradient 134 with respect to the low-rank fitness coefficient 126 based on the loss. The loss function can be cross-entropy loss, mean squared error loss, etc. Other gradients can be obtained in a similar manner for other object samples in the object sample set, and these gradients constitute the first gradient set.

[0032] In some embodiments, the server can adjust the size of the low-rank fitness table based on the first gradient set. The size of the low-rank fitness table involves both length and width. The length refers to the dynamic rank of the coefficients, and the width refers to the number of coefficients. As described above, the coefficients of the low-rank fitness table are used to reflect the fine-tuning of the model's base coefficients, i.e., A·B≈△W. Here, A and B are both low-rank fitness matrices, W is the corresponding base coefficient, and △W is the increment of W. Assuming the dimension of W is 10000×10000, then the dimension of △W is also 10000 (i.e., the number of rows d) × 10000 (i.e., the number of columns d). Through rank decomposition, △W can be represented as the matrix product of A and B. The dimension of A can be 10000 (i.e., number of rows d) × 10 (i.e., dynamic rank r), and the dimension of B can be 10 (i.e., dynamic rank r) × 10000 (i.e., number of columns d). Thus, a 10000 × 10000 matrix can be replaced by a 10000 × 10 matrix and a 10 × 10000 matrix, thereby reducing the data volume. Clearly, if the dynamic rank r is set larger (e.g., close to the rank of ΔW), then coefficients A and B can summarize more information and more accurately reflect the changes in the corresponding basic coefficients. However, if the dynamic rank r is set too large (e.g., exceeding the rank of ΔW), these dimensions become wasteful, generating unnecessary redundant information. In this embodiment, adjusting the length of the low-rank adaptation table based on the first gradient set ensures that the dimensions of coefficients A and B accurately summarize the fine-tuning of the basic coefficients without generating redundant information.

[0033] In some embodiments, the server can update the low-rank fitness table based on the first gradient set and an adjusted size. Taking the low-rank fitness coefficient 126 in the low-rank fitness table as an example, the low-rank fitness coefficient 126 before the update can be [1, 1, 1, 1, 1], and the low-rank fitness coefficient 126 after the update can be [1, 2, 5]. It can be seen that its length is shortened, thereby effectively reducing the size of the low-rank fitness table and the computational cost of the fine-tuning process while ensuring the contribution of the coefficients to the model prediction.

[0034] It should be understood that a server instance can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Servers can be connected directly or indirectly through wired or wireless communication methods, and this application does not impose any restrictions.

[0035] It should be understood that the architecture and functionality in the example environment are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Embodiments of this disclosure can also be applied to other environments with different structures and / or functionalities.

[0036] The processes according to embodiments of this disclosure will be described in detail below with reference to other accompanying drawings. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It will be understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0037] Figure 2 A flowchart of a method 200 for fine-tuning a model according to certain embodiments of the present disclosure is shown. In this embodiment, it can be performed by... Figure 1 The method is executed on a server in an embodiment. In block 202, a first gradient set is determined for a first set of coefficients in a low-rank fitness table of the model, based on an object sample set and a corresponding set of recommended resources. The low-rank fitness table indicates fine-tuning of the underlying coefficients in the model associated with object features. Recommended resources in the recommended resource set are recommended to object samples in the object sample set. The object may be, for example, a user of a content recommendation platform, and the recommended resources in the recommended resource set are those resources (e.g., content items) recommended to the user, which can act as tags. For example, recommended resources are resources that the object actually interacts with. The model may be, for example, a recommendation model used to predict resources that an object might be interested in based on object samples. The model can predict resources that an object might be interested in or pay attention to based on object samples. The object sample set includes one or more object samples, which may include information related to the object, such as object attribute information, public behavior information, etc. Obtaining user samples requires the explicit consent and permission of the user. In some embodiments, object samples include information related to recommended resources. The recommended resource set includes one or more recommended resources.

[0038] The low-rank fitness table stores one or more coefficients, which can be represented as vectors or matrices. These coefficients are also called low-rank fitness matrices (e.g., low-rank fitness matrices A and B, or their operational results). Low-rank fitness matrices A and B can be transposed, multiplied, and other operations to generate new matrices, which represent the fine-tuning (or increment) of the underlying coefficients corresponding to low-rank fitness matrices A and B. The first gradient set includes one or more gradients, where each gradient corresponds one-to-one with an object sample in the object sample set.

[0039] In box 204, the size of the low-rank adaptation table is adjusted based on the first gradient set. The size of the low-rank adaptation table involves both length and width, which together determine the overall data scale of the low-rank adaptation table. The adjustment accuracy directly affects the balance between the efficiency of model fine-tuning and recommendation performance. The server can perform targeted optimization of the length and width based on the information contained in the first gradient set to achieve precise removal of redundant parameters. In some embodiments, the server can adjust the length of the low-rank adaptation table based on the first gradient set. The length refers to the dynamic rank of the coefficients. For example, before the update, the dimension of each coefficient in the low-rank adaptation table can be d×10 (d is much greater than 10), and the length of the low-rank adaptation table is 10. After the update, the dimension of each coefficient in the low-rank adaptation table can be d×5, and the length of the low-rank adaptation table is 5. In some embodiments, the server can adjust the width of the low-rank adaptation table based on the first gradient set. The width of the low-rank adaptation table refers to the number of coefficients. For example, before the update, the low-rank adaptation table includes 100 coefficients, and its width is 100. After the update, the low-rank adaptation table includes 10 coefficients, and its width is 10. Whether adjusting the length or the width, the adjustment of the low-rank adaptation table is based on the requirement of preserving the necessary change information, so that the adjusted coefficients can be used more accurately to calculate the object embedding features, without including redundant information.

[0040] In box 206, the low-rank adaptation table is updated based on the first gradient set and its adjusted size. After dynamically adjusting the size of the low-rank adaptation table, the updated values ​​of each coefficient in the low-rank adaptation table can be calculated based on the first gradient set. The gradient reflects the direction and magnitude of change of the corresponding coefficient. Updating the first gradient set allows for the calculation of more accurate object embedding features, thereby improving the accuracy of the prediction results.

[0041] According to the method of embodiments of this disclosure, a first gradient set is determined based on an object sample set and a corresponding recommendation resource set. Before updating the coefficients in the low-rank fitness table, the method adjusts the size of the low-rank fitness table based on this first gradient set, for example, by adjusting its length and / or width. The method combines the first gradient set with the adjusted size of the low-rank fitness table to update the low-rank fitness table, thereby achieving model fine-tuning. This method can dynamically adjust the size of the low-rank fitness table based on the object samples used for training, ensuring that the dynamically adjusted low-rank fitness table retains sufficient update information while reducing redundant data.

[0042] Figure 3A schematic diagram of an inference path and a training path according to an embodiment of the present disclosure is shown. In this embodiment, an object sample from a set of object samples is used as an example to illustrate the inference path and the training path. In this embodiment, the model is distributed across multiple nodes (nodes may be servers, for example), and each node stores a subset of the model's parameters. For example, the model has a large total embedding table, and each node stores a subset of coefficients from the embedding table. The sum of the embedding tables of these multiple nodes constitutes the model's total embedding table.

[0043] In the inference path, object sample 302 is input to node 1 of the model. In some embodiments, initial object features are determined based on object sample 302 (i.e., object information). These initial object features may be, for example, features encoded using one-hot encoding. In some embodiments, node 1 may store the initial object features in the inference data cache 304 and simultaneously pass them to the index filter 306 in the direction indicated by arrow 332. The index filter 306 stores the correspondence between object samples and indices. Node 1 passes the indexes to the local EMT cache 308 (i.e., the cache of the model's Embedding Table) in the direction indicated by arrow 334. After determining the index, the index filter 306 may store the index at the corresponding position of the initial object feature in object sample 302.

[0044] If the local EMT cache 308 includes the base coefficient corresponding to index 5, this base coefficient can be obtained to calculate the object embedding feature of the object's initial feature. If the local EMT cache 308 does not include the base coefficient corresponding to index 5, the base coefficient corresponding to index 5 can be obtained from other nodes through remote access. Assuming that node 2's EMT cache includes the base coefficient corresponding to index 5, the base coefficient corresponding to index 5 can be obtained by accessing node 2's EMT cache through remote EMT 322.

[0045] Simultaneously, the node also passes the index to the local low-rank adaptation table 310 in the direction indicated by arrow 336, checking whether the low-rank adaptation table 310 contains a coefficient corresponding to index 5. In some embodiments, if the low-rank adaptation table 310 does not contain a coefficient corresponding to the base coefficient, the object embedding feature is determined based on the object initial feature, the index corresponding to the object initial feature, and the base coefficient corresponding to the index. That is, the object embedding feature is calculated using only the base coefficients obtained from the local EMT cache 308 or via the remote EMT 322. For example, the base coefficients and the object initial feature are passed together to the GPU group for inference on node 1 for subsequent calculations in the direction indicated by arrow 336.

[0046] In some embodiments, object embedding features are determined based on initial object features, an index corresponding to the initial object features, a base coefficient corresponding to the index, and coefficients corresponding to the base coefficients in the low-rank adaptation table. That is, if the low-rank adaptation table 310 has a coefficient corresponding to index 5, the object embedding features are calculated not only using the base coefficients obtained from the local EMT cache 308 or via the remote EMT 322, but also using the coefficients in the low-rank adaptation table 310 (which may be referred to as low-rank adaptation coefficients). For example, the base coefficients, low-rank adaptation coefficients, and initial object features are passed together to the GPU set for inference at node 1 for subsequent calculations, following the direction indicated by arrow 336. In some embodiments, the GPU set for inference determines new recommended objects based on the object embedding features. For an object sample set including multiple object samples, multiple inferences can be performed by repeating the above path multiple times.

[0047] In the training path, node 1 can extract the initial object features and indices of object samples 302 from the inference data cache 304, following the direction indicated by arrow 342, and input them into the low-rank adaptation trainer 312 (or LoRA trainer). In some embodiments, the low-rank adaptation trainer 312 uses the extracted initial object features and indices to re-perform the inference process to obtain predicted resources. In some embodiments, node 1 can store the predicted resources obtained in the aforementioned inference process into the inference data cache 304, and the low-rank adaptation trainer 312 can directly read the predicted resources from the inference data cache 304. Furthermore, the low-rank adaptation trainer 312 can also obtain a recommended resource set as a label for the object sample set. Thus, the low-rank adaptation trainer 312 obtains the predicted resource set corresponding to the object sample set through the inference process, and calculates the loss based on the predicted resource set and the recommended resource set, thereby determining the first gradient set for the first coefficient set in the low-rank adaptation table of the model.

[0048] In some embodiments, the first coefficient set includes multiple coefficients, each coefficient including two sub-coefficients, and the rank adjustment module 314 determines multiple dynamic ranks of the multiple coefficients based on the first gradient set. That is, the rank adjustment module 314 determines a dynamic rank for each coefficient. The dimension of ΔW is described by dimension d1×d2, the dimension of sub-coefficient A is described by dimension d1×r (where d1 is greater than r), and the dimension of sub-coefficient B is described by dimension r×d2 (where d2 is greater than r). The dynamic rank refers to the parameter r, that is, the maximum value of sub-coefficient A and sub-coefficient B. Because any gradient in the first gradient set contains optimization information for a coefficient, each coefficient may have a different dynamic rank due to different gradients.

[0049] As an example, the rank adjustment module 314 can perform principal component analysis on each gradient in the first gradient set to determine multiple eigenvalues ​​for each gradient. For each gradient, a certain number of eigenvalues ​​are selected such that their sum is not less than a specified proportion (e.g., 80%) of the sum of all eigenvalues, and this number is used as the dynamic rank.

[0050] In some embodiments, during the determination of the dynamic rank, taking a gradient from the first gradient set as an example, the rank adjustment module 314 determines multiple orthogonal bases for the gradient based on the covariance matrix for the gradient, wherein the multiple orthogonal bases have multiple eigenvalues. The covariance matrix is ​​used to describe the degree of linear correlation between the dimensions of the gradient. By performing eigenvalue decomposition on the covariance matrix, multiple eigenvalues ​​and corresponding eigenvectors can be obtained. These eigenvectors constitute multiple orthogonal bases, and the magnitude of the eigenvalues ​​reflects the ability of the corresponding orthogonal bases to represent gradient information.

[0051] In some embodiments, the rank adjustment module 314 determines an orthogonal basis subset from multiple orthogonal bases based on multiple eigenvalues ​​and a proportion threshold, wherein the sum of one or more eigenvalues ​​corresponding to the orthogonal basis subset is greater than or equal to the product of the multiple eigenvalues ​​and the proportion threshold. The proportion threshold is a pre-set information retention ratio (e.g., 80%, 90%). Specifically, all eigenvalues ​​are first sorted in descending order, and then the sorted eigenvalues ​​are sequentially accumulated. Accumulation stops when the accumulated sum is greater than or equal to the product of the sum of the multiple eigenvalues ​​and the proportion threshold. At this point, the orthogonal basis corresponding to the accumulated eigenvalues ​​constitutes the orthogonal basis subset. This process can select representative orthogonal bases while preserving core gradient information.

[0052] In some embodiments, the rank adjustment module 314 determines the number of orthogonal bases included in the orthogonal base subset as the first quantity. For example, if the sorted feature values ​​are [15, 10, 5, 3, 2], the proportion threshold is set to 90%, the sum of the feature values ​​is 35, and 90% of that is 31.5. The sum of the first three feature values ​​is 15 + 10 + 5 = 30 (less than 31.5), and the sum of the first four feature values ​​is 15 + 10 + 5 + 3 = 33 (greater than 31.5). Therefore, the orthogonal base subset contains 4 orthogonal bases, and the first quantity is 4. In some embodiments, the rank adjustment module 314 determines the first quantity as the dynamic rank of the coefficients corresponding to the gradient in the first coefficient set. Since the first quantity represents the number of orthogonal bases that can characterize the core information of the gradient, using it as the dynamic rank ensures that the low-rank adaptive coefficients can accurately capture the gradient change pattern at this rank, while avoiding parameter redundancy caused by excessively high rank, thus achieving a match between the dynamic rank and the gradient information representation requirements.

[0053] In some embodiments, the rank adjustment module 314 determines a first dynamic rank based on multiple dynamic ranks. In order to reduce data jitter (i.e., fluctuation differences between different dynamic ranks) and avoid unreasonable rank settings due to individual outliers, the multiple dynamic ranks can be unified into the same value, for example, the median, arithmetic mean or mode of multiple dynamic ranks can be taken as the first dynamic rank, thereby improving the stability of the dynamic rank.

[0054] In some embodiments, the rank adjustment module 314 determines a weighted sum of multiple dynamic ranks based on their respective weights. For example, if the multiple dynamic ranks are r1=8 and r2=12, with corresponding weights w1=0.3 and w2=0.7, the weighted sum is 8×0.3+12×0.7=2.4+8.4=10.8. In some embodiments, the first dynamic rank is determined by rounding the weighted sum. Since the dynamic rank must be a positive integer (the rank represents the number of linearly independent rows or columns of a matrix, and must be an integer), the server determines the first dynamic rank by rounding the weighted sum. The rounding operation can be performed using rounding, rounding up, or rounding down.

[0055] Figure 4A A schematic diagram illustrating the adjustment of the length of a low-rank adaptation table according to an embodiment of the present disclosure is shown. In this embodiment, the low-rank adaptation table 410 includes several coefficients, each coefficient including two sub-coefficients (i.e., sub-coefficient A and sub-coefficient B). For example, before the update, the dynamic rank of each coefficient in the low-rank adaptation table 410 is 10, and the dynamic rank of coefficient 412 is also 10. After the update according to the foregoing embodiment, i.e., the new dynamic rank of coefficient 412 is determined based on the gradient corresponding to coefficient 412, for example, if the new dynamic rank is 6, then coefficient 412 is shortened. Under the limitation of this dynamic rank, the value of coefficient 412 is further updated based on the gradient to obtain coefficient 422. This completes the update of a single coefficient, resulting in the updated low-rank adaptation table 420.

[0056] return Figure 3 In some embodiments, the pruning module 316 performs the pruning operation. In some embodiments, the low-rank fitness table includes multiple coefficients and multiple indexes, and the pruning module 316 can determine a first set of coefficients corresponding to the object sample set from the multiple coefficients of the low-rank fitness table based on the multiple indexes. Since there is a one-to-one correspondence between the object samples and the coefficients in the low-rank fitness table, the first set of coefficients can be found based on the indexes.

[0057] In some embodiments, the pruning module 316 can update the adjustment frequency of the coefficients in the first coefficient set. For example, the pruning module 316 can record the number of times each coefficient is updated, incrementing the count by 1 each time the coefficient is updated. The adjustment frequency of the coefficient is the number of times it is updated divided by the total number of updates. In some embodiments, if the adjustment frequency of a second coefficient in the first coefficient set is less than a frequency threshold, the pruning module 316 can remove the second coefficient from the low-rank fitness table. An adjustment frequency less than the frequency threshold indicates that the coefficient is rarely used, meaning that few objects have information about that object. Therefore, to improve training efficiency, the second coefficient can be removed, thereby reducing the width of the low-rank fitness table 310.

[0058] In some embodiments, the low-rank adaptation table 310 includes a third coefficient, and if the adjustment frequency of the third coefficient is less than a frequency threshold, the third coefficient is removed from the low-rank adaptation table 310. In this embodiment, not only the updated coefficients are checked, but also the unupdated coefficients are checked, and those less active coefficients in the low-rank adaptation table 310 are removed, thereby reducing the width of the low-rank adaptation table 310 and reducing the computational load of the training process.

[0059] Figure 4B A schematic diagram illustrating the adjustment of the width of a low-rank adaptation table according to an embodiment of the present disclosure is shown. In this embodiment, the low-rank adaptation table 430 includes several coefficients, such as coefficient 432. For example, before the update, the low-rank adaptation table 430 includes 5 coefficients. After updating according to the foregoing embodiment, it is determined that the adjustment frequency of coefficient 432 is less than a frequency threshold, therefore coefficient 432 is removed from the low-rank adaptation table 430. This completes the update of a single coefficient, resulting in an updated low-rank adaptation table 440. The updated low-rank adaptation table 440 includes only 4 coefficients.

[0060] Figure 5 A schematic diagram illustrating the adjustment of the size of a low-rank adaptation table according to an embodiment of the present disclosure is shown. In this embodiment, the low-rank adaptation table 510 includes five coefficients: coefficients 512, 514, 516, 518, and 520, each with a dynamic rank of 10. After updating according to the foregoing embodiment, it is determined that the adjustment frequency of coefficients 518 and 520 is less than a frequency threshold, therefore coefficients 518 and 520 are removed from the low-rank adaptation table 510. Furthermore, the first dynamic rank of the five coefficients is determined to be 3, therefore coefficients 512, 514, and 516 are shortened. In some embodiments, the values ​​of coefficients 512, 514, and 516 are updated based on the shortened coefficients 512, 514, and 516 and the corresponding gradients, thereby obtaining an updated low-rank adaptation table 522. As can be seen from the figure, the size of the low-rank adaptation table 522 is much smaller than that of the low-rank adaptation table 510, which ensures accuracy while reducing the computational load of the training process.

[0061] return Figure 3 As indicated by arrow 344, the low-rank adaptation trainer 312 can store the updated content in the low-rank adaptation table 310. Subsequent inference processes of node 1 can use the updated low-rank adaptation table 310, ensuring its freshness (i.e., a sufficiently high update frequency). Furthermore, node 1 can synchronize the low-rank adaptation table 310 to other nodes via the remote LoRA table 318 (i.e., update the LoRA tables of other nodes based on node 1's LoRA table) to unify the latest LoRA tables across all nodes of the model. If the width of the low-rank adaptation table 310 is adjusted, the index content may also be adjusted. As indicated by arrow 346, the indexes of the coefficients retained in the low-rank adaptation table 310 can be synchronized to the index filter 306, allowing these adjusted indexes to access the LoRA table 310 during inference.

[0062] In some embodiments, the server used to run the model employs a distributed processing architecture, including multiple groups of processing units, which achieve parallel and efficient execution of fine-tuning and inference tasks through functional grouping. The first group of processing units, comprising multiple processing units, performs fine-tuning, a computationally intensive task involving gradient calculation and coefficient updates. The second group of processing units, also comprising multiple processing units, performs inference, receiving user requests and outputting recommendation results, requiring high response speed. These processing units can be data processing capabilities such as CPUs, GPUs, Tensor Processing Units (TPUs), or processor cores, and the number of processing units in the two groups can be pre-allocated based on the initial task load (e.g., four GPUs allocated to the first group and two GPUs allocated to the second group).

[0063] Figure 6 A schematic diagram of an adjusted processing unit according to an embodiment of the present disclosure is shown. The server includes multiple core composite chips (CCDs), each chip including a set of processing units. A first set of CCDs 610 can be allocated for the training process, and a second set of CCDs 620 can be allocated for the inference process. The core composite chips may be provided with dedicated memory to improve the data reading efficiency of the processing units. For example, in the CCDs of the first set of CCDs 610, the dedicated memory can store a training cache, which the processing units in the processing unit set can quickly read. Similarly, in the CCDs of the second set of CCDs 620, the dedicated memory can store an inference cache, which the processing units in the processing unit set can quickly read.

[0064] In some embodiments, the server can determine inference latency. Inference latency can be calculated using different metrics. For example, the P99 inference latency of the model can be calculated, which involves statistically analyzing the latency data of all inference requests over a period of time (e.g., 5 minutes), sorting them in ascending order, and taking the latency value corresponding to the 99th percentile. This metric effectively reflects inference performance under extreme conditions and better reflects the user's perception of latency in real-world applications. Furthermore, other quantile latencies such as P50 and P95, or average latency, can be calculated as needed.

[0065] In some embodiments, if the inference latency exceeds a latency threshold, the server can move some processing units from the first group of processing units to the second group of processing units. If the detected inference latency exceeds the latency threshold, it indicates that the computing power of the current second group of processing units cannot meet the inference task requirements, and the server can move some processing units from the first group of processing units to the second group of processing units. For example, one CCD can be moved from the first group of CCDs 610 to the second group of CCDs 620.

[0066] In some embodiments, if the inference latency is less than or equal to a latency threshold and the training speed is less than the speed limit, some processing units in the second group of processing units are moved to the first group of processing units. In some embodiments, while monitoring the inference latency, the server can also statistically analyze the model's training speed in real time. The training speed refers to the number of model fine-tuning iterations completed by the first group of processing units per unit time, which reflects the efficiency of the fine-tuning task. If the detected inference latency is less than or equal to the latency threshold (indicating that the second group of processing units has sufficient computing power and the inference performance meets the standard) and the training speed is less than the speed limit (indicating that the computing power of the first group of processing units still has room for improvement and the fine-tuning efficiency can be optimized), the server can move some processing units in the second group of processing units to the first group of processing units. For example, one CCD can be moved from the second group of CCD 620 to the first group of CCD 610. This ensures that the remaining computing power of the second group of processing units can still maintain the inference latency within the threshold after the move, avoiding a decrease in inference performance due to computing power transfer, and achieving synergistic optimization of fine-tuning efficiency and inference performance.

[0067] Figure 7 A schematic block diagram of an example device 700 according to some embodiments of the present disclosure is shown. Device 700 can be implemented by software, hardware, or a combination of both. Figure 7 As shown, the device 700 includes a determining module 710, an adjusting module 720, and an updating module 730.

[0068] In some embodiments, the determining module 710 may be configured to determine a first gradient set for a first set of coefficients in the low-rank fitness table of the model, based on the object sample set and the corresponding recommendation resource set. The low-rank fitness table indicates fine-tuning of the base coefficients in the model associated with object features, and the recommendation resources in the recommendation resource set are recommended to the object samples in the object sample set. The adjusting module 720 may be configured to adjust the size of the low-rank fitness table based on the first gradient set. The updating module 730 may be configured to update the low-rank fitness table based on the first gradient set and the adjusted size.

[0069] In some embodiments, the first set of coefficients includes a plurality of coefficients, and the adjustment module 720 includes a dynamic rank determination module configured to determine a plurality of dynamic ranks of the plurality of coefficients based on the first gradient set; and a first determination module configured to determine a first dynamic rank based on the plurality of dynamic ranks.

[0070] In some embodiments, the dynamic rank determination module includes a second determination module configured to perform the following operations for each gradient in the first gradient set: determining a plurality of orthogonal bases for the gradient based on the covariance matrix for the gradient, wherein the plurality of orthogonal bases have a plurality of eigenvalues; determining a subset of orthogonal bases from the plurality of orthogonal bases based on the plurality of eigenvalues ​​and a scaling threshold, wherein the sum of one or more eigenvalues ​​corresponding to the subset of orthogonal bases is greater than or equal to the product of the plurality of eigenvalues ​​and the scaling threshold; determining the number of orthogonal bases included in the subset of orthogonal bases as a first number; and determining the first number as the dynamic rank of the coefficients corresponding to the gradient in the first coefficient set.

[0071] In some embodiments, the first determining module includes a weighting module configured to determine a weighted sum of multiple dynamic ranks based on the corresponding weights of the multiple dynamic ranks; and a rounding module configured to determine a first dynamic rank by performing a rounding operation on the weighted sum.

[0072] In some embodiments, the low-rank adaptation table includes multiple coefficients and multiple indexes, and the adjustment module 720 includes a second determination module configured to determine a first set of coefficients corresponding to the object sample set from the multiple coefficients of the low-rank adaptation table based on the multiple indexes; a frequency update module configured to update the adjustment frequency of the coefficients of the first set of coefficients; and a first removal module configured to remove a second coefficient from the low-rank adaptation table in response to the adjustment frequency of a second coefficient in the first set of coefficients being less than a frequency threshold.

[0073] In some embodiments, the plurality of coefficients includes a third coefficient that does not belong to the first set of coefficients, and the apparatus 700 further includes a second removal module configured to remove the third coefficient from the low-rank adaptation table in response to an adjustment frequency of the third coefficient being less than a frequency threshold.

[0074] In some embodiments, a first group of processing units, including multiple processing units, is used to perform fine-tuning, a second group of processing units, including multiple processing units, is used to perform inference, and the apparatus 700 further includes an inference latency determination module configured to determine an inference latency; and a first moving module configured to move a portion of the processing units in the first group of processing units to the second group of processing units in response to an inference latency greater than a latency threshold.

[0075] In some embodiments, the apparatus 700 further includes a second moving module configured to move a portion of the processing units in the second group of processing units to the first group of processing units in response to an inference latency less than or equal to a latency threshold and a training speed less than a speed limit.

[0076] In some embodiments, the apparatus 700 further includes an initial feature determination module configured to determine initial features of an object based on object information; a first embedding feature determination module configured to determine object embedding features based on the initial features of the object, an index corresponding to the initial features of the object, a base coefficient corresponding to the index, and a coefficient corresponding to the base coefficient in an updated low-rank adaptation table; and a resource recommendation module configured to determine new recommended resources based on the object embedding features.

[0077] In some embodiments, the apparatus 700 further includes a second embedding feature determination module configured to determine object embedding features based on object initial features, an index corresponding to the object initial features, and a basic coefficient corresponding to the index in response to the absence of a coefficient corresponding to the basic coefficient in the low-rank adaptation table.

[0078] In some embodiments, the model is distributed across a first node and a second node, a low-rank adaptation table is stored in the first node, and the apparatus 700 further includes a remote update module configured to update the low-rank adaptation table of the second node based on the low-rank adaptation table of the first node.

[0079] The division of modules or units in the embodiments of this disclosure is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional units in the disclosed embodiments may be integrated into one unit, exist as separate physical entities, or two or more units may be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0080] Figure 8 A block diagram of an example device 800 that can be used to implement embodiments of the present disclosure is shown. It should be understood that... Figure 8 The device 800 shown is merely an example and should not be construed as limiting the functionality and scope of the implementation described herein. For example, device 800 may correspond to the implementation described herein. Figure 1The user equipment described above can be used to perform the above-described... Figures 1 to 6 The process. For example, device 800 may correspond to the electronic device of the third aspect of the invention.

[0081] like Figure 8 As shown, device 800 is in the form of a general-purpose computing device. Components of device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of device 800.

[0082] Device 800 typically includes multiple computer storage media. Such media can be any available media accessible to device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof). Storage device 830 can be removable or non-removable media and may include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within device 800.

[0083] Device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 8 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 820 may include computer program product 825 having one or more program modules configured to perform various methods or actions of various implementations of this disclosure.

[0084] The communication unit 840 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of device 800 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, device 800 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0085] Input device 850 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 860 can be one or more output devices, such as a monitor, speaker, printer, etc. Device 800 can also communicate as needed with one or more external devices (not shown) via communication unit 840. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with device 800, or with any device that enables device 800 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0086] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is provided that stores a computer program thereon, which, when executed by a processor, implements the methods described above.

[0087] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0088] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0089] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0091] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for fine-tuning a model, comprising: Based on the object sample set and the corresponding recommendation resource set, a first gradient set is determined for the first coefficient set in the low-rank fitness table of the model, wherein the low-rank fitness table indicates fine-tuning of the basic coefficients in the model that are associated with object features, and the recommendation resources in the recommendation resource set are recommended to the object samples in the object sample set. The size of the low-rank adaptation table is adjusted based on the first gradient set; as well as The low-rank adaptation table is updated based on the first gradient set and the adjusted size.

2. The method of claim 1, wherein the first coefficient set comprises a plurality of coefficients, and adjusting the size of the low-rank adaptation table based on the first gradient set comprises: Multiple dynamic ranks of the multiple coefficients are determined based on the first gradient set; as well as The first dynamic rank is determined based on the plurality of dynamic ranks.

3. The method of claim 2, wherein determining the plurality of dynamic ranks of the plurality of coefficients based on the first gradient set comprises: For each gradient in the first set of gradients, perform the following operation: A plurality of orthogonal bases for the gradient are determined based on the covariance matrix for the gradient, wherein the plurality of orthogonal bases have a plurality of eigenvalues; An orthogonal basis subset is determined from the plurality of orthogonal bases based on the plurality of eigenvalues ​​and a scaling threshold, wherein the sum of one or more eigenvalues ​​corresponding to the orthogonal basis subset is greater than or equal to the product of the plurality of eigenvalues ​​and the scaling threshold; Determine the number of orthogonal bases included in the orthogonal basis subset, and use this as the first number; as well as The first quantity is determined as the dynamic rank of the coefficients in the first set of coefficients corresponding to the gradient.

4. The method of claim 2, wherein determining the first dynamic rank based on the plurality of dynamic ranks comprises: The weighted sum of the multiple dynamic ranks is determined based on the corresponding weights of the multiple dynamic ranks; as well as The first dynamic rank is determined by rounding down the weighted sum.

5. The method of claim 1, wherein the low-rank adaptation table comprises a plurality of coefficients and a plurality of indices, and adjusting the size of the low-rank adaptation table based on the first gradient set comprises: Based on the multiple indexes, the first set of coefficients corresponding to the object sample set is determined from the multiple coefficients of the low-rank adaptation table; Update the adjustment frequency of the coefficients in the first coefficient set; In response to the adjustment frequency of the second coefficient in the first set of coefficients being less than a frequency threshold, the second coefficient is removed from the low-rank adaptation table.

6. The method of claim 5, wherein the plurality of coefficients includes a third coefficient, the third coefficient not belonging to the first set of coefficients, and the method further comprises: In response to the adjustment frequency of the third coefficient being less than the frequency threshold, the third coefficient is removed from the low-rank adaptation table.

7. The method of claim 1, further comprising a first group of processing units comprising a plurality of processing units for performing fine-tuning, a second group of processing units comprising a plurality of processing units for performing inference, and the method further comprising: Determine the reasoning delay; In response to the inference delay being greater than a delay threshold, some processing units in the first group of processing units are moved to the second group of processing units.

8. The method according to claim 7, further comprising: In response to the inference latency being less than or equal to the latency threshold and the training speed being less than the speed limit, some processing units in the second group of processing units are moved to the first group of processing units.

9. The method according to claim 1, further comprising: Determine the initial characteristics of the object based on the object information; The object embedding features are determined based on the initial features of the object, the index corresponding to the initial features of the object, the basic coefficients corresponding to the indexes, and the coefficients corresponding to the basic coefficients in the updated low-rank adaptation table. as well as New recommended resources are determined based on the object embedding features.

10. The method of claim 9, further comprising: In response to the absence of a coefficient corresponding to the basic coefficient in the low-rank adaptation table, the object embedding feature is determined based on the object initial feature, the index corresponding to the object initial feature, and the basic coefficient corresponding to the index.

11. The method of claim 10, wherein the model is distributed across a first node and a second node, the low-rank adaptation table is stored in the first node, and the method further comprises: Update the low-rank adaptation table of the second node according to the low-rank adaptation table of the first node.

12. An apparatus for fine-tuning a model, comprising: The determination module is configured to determine a first gradient set based on a set of object samples and a corresponding set of recommended resources, for a first set of coefficients in a low-rank fitness table of the model, wherein the low-rank fitness table indicates fine-tuning of the basic coefficients in the model that are associated with object features, and the recommended resources in the set of recommended resources are recommended to the object samples in the set of object samples. The adjustment module is configured to adjust the size of the low-rank adaptation table based on the first gradient set; as well as The update module is configured to update the low-rank adaptation table based on the first gradient set and the adjusted size.

13. An electronic device, comprising: At least one processing unit; At least one memory is coupled to the at least one processing unit and stores operations to be performed by the at least one processing unit, the operations causing the electronic device to perform the method according to any one of claims 1 to 11 when performed by the at least one processing unit.

14. A computer program product having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images