Method and system for low-rank adaptive training of interview matrix model based on ai large model

Abstract

The application provides a method and system for low-rank adaptive training of an interview matrix model based on an AI large model, and relates to the technical field of artificial intelligence, which comprises the following steps: obtaining a training data set and an answer label thereof; clustering the information entropy of each interview question sample by using a clustering algorithm to obtain a cluster to which each interview question sample belongs; assigning a corresponding LoRA rank to the interview question samples of different clusters; training the interview matrix model to be trained according to the training data set and the corresponding LoRA rank and answer label to obtain a trained interview matrix model; wherein, during the training process, the cluster-level training loss, the cluster-level gradient norm and the cluster-level verification loss of each cluster are comprehensively considered to dynamically and adaptively adjust the LoRA rank. The application optimizes the allocation of computing resources, enhances the expression of the model and improves the generalization performance of the model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for training an interview matrix model based on a low-rank adaptive training of a large AI model. Background Technology

[0002] Interview matrix models are customized models for recruitment scenarios. To ensure their inference capabilities, they have a huge number of parameters, which brings challenges to fine-tuning and deployment. Low-Rank Adaptation (LoRA) technology emerged to address this challenge. It freezes most of the parameters of the original model layers and introduces and trains a small number of low-rank trainable matrices (increments in the weight matrix). This significantly reduces the number of trainable parameters, lowers computational and storage costs, and achieves performance similar to full-parameter fine-tuning in downstream tasks. The core assumption of LoRA is that the update matrix of the model weights during fine-tuning (…) ΔW It has the low-rank property, meaning it can be expressed as the product of two smaller matrices. ΔW = AB This is approximated by dimensionality reduction and then dimensionality increase, where the original parameters are represented by the weight matrix. Direct updates require training. d × k There are 10 parameters, while the number of parameters in LoRA is determined by decomposing them into 10 parameters. and Only training required r ×( d + k ) parameters, r min( d , k ), where the rank r By controlling the dimensionality of the matrix factorization, the number of parameters is much smaller than that of the original weight matrix W, significantly reducing the number of training parameters. This is achieved by adding the weight matrix to the original weight matrix. This enables efficient fine-tuning (such as...) Figure 1 (as shown), in addition to, ΔW = AB It also sets a scaling factor parameter (such as alpha, i.e.) α ), used to adjust the magnitude of the low-rank matrix update. Among them, h This is the output of the model layer. W 0 represents the original weight matrix. x This is the input vector for this model layer.

[0003] Figure 1 This illustrates the weight update methods for regular fine-tuning and LoRA fine-tuning: Regular fine-tuning ( Figure 1 When (on the left side), directly apply the pre-trained weights. WAdd a complete learnable weight update matrix ΔW LoRA fine-tuning ( Figure 1 When (on the right side), do not update directly. W Instead, two low-rank matrices are used. and To approximate ΔW ,Right now ΔW ≈ AB ,in r It is a low-rank dimension (hyperparameter), which significantly reduces the number of trainable parameters. LoRA decomposes the full matrix update into the product of two low-rank matrices, achieving efficient parameter fine-tuning.

[0004] For ease of understanding, assume that given parameters d =10000, k =5000, such as the dimension of the weight matrix in the attention layer or FFN (Feed-Forward Neural Network) layer of a Transformer. W If the elements are 32-bit floating-point numbers (4 bytes), then the original weight matrix... W The number of parameters is d × k =10000 × 5000 = 50,000,000 (50 million), memory is 50,000,000 × 4 = 200,000,000, bytes is 200MB. If LoRA is used, let the rank... r =4, then the number of parameters is r ×( d + k The result is 4 × (10000 + 5000) = 4 × 15000 = 60,000. The memory usage is only 60,000 × 4 = 240,000 bytes ≈ 0.24MB. Compared to the original weight matrix, the number of parameters is reduced by approximately 99.88%, and the memory usage is reduced from 200MB to 0.24MB.

[0005] Although LoRA is effective, its performance is largely dependent on rank. r The choice and impact of rank. Using a fixed, uniform rank across all layers and modules is often not optimal; different layers or modules may require different ranks. Lower ranks reduce resource consumption but may limit the model's expressiveness and generalization ability, while higher ranks enhance expressiveness and generalization ability but increase computational cost and parameter count. Therefore, this "one-size-fits-all" approach of fixed-rank allocation has certain limitations, and choosing an appropriate rank requires repeated trade-offs among various factors such as performance and efficiency, making the decision difficult. How to solve this problem is the core driving force behind the shift from fixed-rank LoRA to dynamic-rank LoRA. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for training an interview matrix model based on a low-rank adaptive training of a large AI model, so as to optimize the allocation of computing resources, enhance the expressiveness of the model, and improve the generalization performance of the model.

[0007] In a first aspect, the present invention provides a method for training an interview matrix model based on a low-rank adaptive training of a large AI model, comprising: The training dataset and its answer labels are obtained. The interview question samples in the training dataset are divided into two categories: training samples and validation samples. Clustering algorithms are used to cluster the information entropy of each interview question sample to obtain the cluster to which each interview question sample belongs; Assign corresponding LoRA ranks to interview question samples from different clusters; wherein, the LoRA rank of the cluster with higher information entropy is greater than that of the cluster with lower information entropy. Based on the training dataset and the corresponding LoRA rank and answer labels, the interview matrix model to be trained is trained to obtain the trained interview matrix model. During the training process, the cluster-level training loss, cluster-level gradient norm and cluster-level validation loss of each cluster are comprehensively considered, and the LoRA rank is dynamically and adaptively adjusted.

[0008] In an optional implementation, the information entropy of each interview question sample is clustered using a clustering algorithm to obtain the cluster to which each interview question sample belongs, including: The information entropy of each interview question sample is calculated based on the valid tokens in the original vector corresponding to each interview question sample. Clustering is performed on the information entropy of each interview question sample by minimizing the intra-cluster variance, thus obtaining the cluster to which each interview question sample belongs.

[0009] In an optional implementation, the information entropy of each interview question sample is calculated based on the valid tokens in the original vector corresponding to that sample, including: The information entropy of each interview question sample is calculated based on the frequency of occurrence and the number of valid tokens in the original vector corresponding to each interview question sample.

[0010] In an optional implementation, clustering of the information entropy of each interview question sample is performed using a clustering algorithm that minimizes intra-cluster variance, resulting in the cluster to which each interview question sample belongs, including: Randomly select a preset number of initial cluster centers; The information entropy of each interview question sample is assigned to the cluster containing the nearest cluster center; Update the cluster center based on the entropy of each information currently contained in each cluster; The current objective function value is calculated based on the updated cluster centers and the preset objective function based on intra-cluster variance. Determine whether the current objective function value is the same as the previous objective function value at the time of the last update; If not, re-execute the step of assigning the information entropy of each interview question sample to the cluster containing the nearest cluster center; If so, determine the cluster to which each interview question sample is currently assigned as the cluster to which the interview question sample belongs.

[0011] In an optional implementation, the interview matrix model to be trained is trained based on the training dataset and the corresponding LoRA rank and answer labels to obtain the trained interview matrix model, including: Each interview question sample is input into the interview matrix model to obtain the predicted answer of the interview matrix model using the corresponding LoRA rank output; The sample loss of each interview question sample is calculated based on the predicted answer and answer label. Based on the sample loss of each interview question sample, the total training loss, as well as the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss for each cluster are calculated. The LoRA rank of each cluster is dynamically adjusted based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss. Based on the total training loss and the current LoRA rank of each cluster, the LoRA layer parameters of the interview matrix model are updated in reverse using the gradient descent algorithm.

[0012] In an optional implementation, the LoRA rank of each cluster is dynamically adjusted based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss, including: The total cluster score for each cluster is calculated based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss for each cluster. The LoRA rank of each cluster is dynamically adjusted based on the relationship between the total cluster-level score of each cluster and the preset high and low score thresholds.

[0013] Secondly, this invention provides a system for training an interview matrix model based on a low-rank adaptive training of a large AI model, comprising: The data acquisition module is used to acquire the training dataset and its answer labels. The interview question samples in the training dataset are divided into two categories: training samples and validation samples. The sample clustering module is used to cluster the information entropy of each interview question sample using a clustering algorithm to obtain the cluster to which each interview question sample belongs; The rank assignment module is used to assign corresponding LoRA ranks to interview question samples from different clusters; among them, the LoRA rank corresponding to the cluster with higher information entropy is greater than the LoRA rank corresponding to the cluster with lower information entropy. The model training module is used to train the interview matrix model to be trained based on the training dataset, the corresponding LoRA rank, and the answer label, so as to obtain the trained interview matrix model. During the training process, the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster are comprehensively considered, and the LoRA rank of each cluster is dynamically and adaptively adjusted.

[0014] In an optional implementation, the model training module is specifically used for: inputting each interview question sample into the interview matrix model to obtain the predicted answer output by the interview matrix model under the corresponding LoRA rank; calculating the sample loss of the interview question sample based on the predicted answer and answer label of each interview question sample; calculating the total training loss, cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster based on the sample loss of each interview question sample; dynamically adjusting the LoRA rank corresponding to each cluster based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster; and updating the LoRA layer parameters of the interview matrix model in reverse using the gradient descent algorithm based on the total training loss and the current LoRA rank corresponding to each cluster.

[0015] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method of low-rank adaptive training of interview matrix model based on AI large model according to any of the foregoing embodiments.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the method of training an interview matrix model based on a low-rank AI large model according to any of the foregoing embodiments.

[0017] The present invention provides a method and system for adaptively training an interview matrix model based on a large AI model with low rank. The method includes: acquiring a training dataset and its answer labels; dividing the interview question samples in the training dataset into training samples and validation samples; clustering the information entropy of each interview question sample using a clustering algorithm to obtain the cluster to which each interview question sample belongs; assigning corresponding LoRA ranks to the interview question samples in different clusters; wherein the LoRA rank corresponding to the cluster with higher information entropy is greater than the LoRA rank corresponding to the cluster with lower information entropy; training the interview matrix model to be trained based on the training dataset, the corresponding LoRA ranks, and the answer labels to obtain the trained interview matrix model; wherein, during the training process, the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster are comprehensively considered, and its LoRA rank is dynamically and adaptively adjusted. By introducing information entropy as an input complexity metric and performing clustering based on information entropy, different LoRA ranks can be assigned to different clusters to match their computational resource requirements. During the training of the interview matrix model, the LoRA rank of each cluster can be dynamically adjusted based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss. This ensures that the rank adjustment strictly matches the computational requirements, maximizing parameter utilization efficiency and computational efficiency while guaranteeing performance. As a result, computational resource allocation is optimized, model expressiveness is enhanced, and model generalization performance is improved. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A diagram showing the weight updates for regular fine-tuning and LoRA fine-tuning; Figure 2 A flowchart illustrating a method for training an interview matrix model based on a low-rank adaptive training of a large AI model, provided in an embodiment of the present invention; Figure 3 A schematic diagram illustrating the training process of an interview matrix model provided in an embodiment of the present invention; Figure 4 This invention provides an example of sequence cluster partitioning. Figure 5 A schematic diagram illustrating the training process of another interview matrix model provided in an embodiment of the present invention; Figure 6 This invention provides a schematic diagram of the structure of a system for training an interview matrix model based on a low-rank adaptive training of a large AI model, as shown in an embodiment of the invention. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Traditional LoRA methods currently suffer from the following drawbacks: Static fixed rank: The LoRA rank of the entire model is usually a fixed hyperparameter, set before training and remaining unchanged during training. The choice of this rank often relies on experience or is determined through time-consuming hyperparameter search, making it difficult to optimize for specific needs at different training stages for data of varying complexity or models.

[0022] Inflexible resource allocation: Fixed rank means that all data or all adaptation layers are assigned the same number of rank parameters. This may result in too many rank parameters being assigned to simple or well-trained data, leading to overfitting and potential resource waste, while too few rank parameters are assigned to complex or under-trained data, leading to underfitting and insufficient resource allocation.

[0023] Lack of dynamic adaptability: During training, the model's training state and sensitivity to different data features change dynamically, and different inputs differ significantly in semantics and structure. Static rank cannot adapt to this change and cannot adjust resource allocation according to the model's real-time training status and data characteristics.

[0024] Memory and computational efficiency: Fixed-rank LoRA may consume too much memory at high ranks, leading to overcomputation, or consume too little memory at low ranks, resulting in insufficient computation and thus failing to adequately fit complex data, limiting the model's performance.

[0025] Lack of feedback adjustment mechanism: During training, there is a lack of ability to dynamically adjust the LoRA rank based on model feedback (such as loss, gradient, generalization ability).

[0026] Based on this, this invention provides a method and system for adaptively training an interview matrix model using a large AI model with low rank. This method improves the inference ability and training efficiency of the interview matrix model. Specifically, information entropy is introduced as an input complexity metric to measure the uncertainty of token distribution (i.e., the smallest processing unit, such as words, sub-words, characters, or other forms of language fragments). Clustering is then performed based on entropy, automatically dividing the model into three clusters: simple, medium, and complex. Each cluster is assigned a different rank to match its computational resource requirements. Then, based on multiple performance metrics such as cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss, the rank of each cluster is adaptively fine-tuned (increased, decreased, or unchanged). This strategy aims to overcome the limitations of fixed rank, further optimize computational resource allocation, enhance model expressiveness, and improve model generalization performance.

[0027] To facilitate understanding of this embodiment, a method for training an interview matrix model based on a low-rank adaptive training of a large AI model will first be described in detail.

[0028] This invention provides a method for training a low-rank adaptive interview matrix model based on a large AI model. This method can be executed by an electronic device with data processing capabilities and can be applied to talent recommendation systems. The purpose of this invention is to optimize model parameter update efficiency and improve the performance and generalization ability of the interview matrix model through fine-tuning and adaptive resource allocation strategies. See also... Figure 2 The diagram shows a method for training an interview matrix model based on a low-rank adaptive training of a large AI model. This method mainly includes the following steps S210 to S240: Step S210: Obtain the training dataset and its answer labels; the interview question samples in the training dataset are divided into two categories: training samples and validation samples.

[0029] The training dataset contains a large number of interview questions from various industries, positions, and job levels, reflecting real-world interview experiences. It also includes the latest cutting-edge interview questions in the industry, covering multiple question levels such as easy, medium, and difficult. This allows for a more effective assessment of interviewees' actual abilities and helps companies find suitable candidates. The training samples in the training dataset are primarily used for backpropagation to update model parameters and for adjusting the LoRA rank. The validation samples, however, do not participate in backpropagation and are only used for adjusting the LoRA rank and validating the model's training performance.

[0030] Step S220: Cluster the information entropy of each interview question sample using a clustering algorithm to obtain the cluster to which each interview question sample belongs.

[0031] Clustering groups interview question samples with similar complexity levels into a single group, allowing for the assignment of an appropriate LoRA rank to each group. The clustering algorithm divides data points (information entropy) into a predetermined number of clusters, K, for example, K=3, representing simple, medium, and complex categories respectively. It should be noted that this embodiment of the invention does not limit the number of clusters; in other embodiments, it can be divided into fewer or more clusters. For example, the training data can be divided into simple and complex clusters, or into simple, medium, moderately complex, and complex clusters, etc.

[0032] In some possible embodiments, step S220 above may include: calculating the information entropy of the interview question sample based on the valid tokens in the original vector corresponding to each interview question sample; and clustering the information entropy of each interview question sample using a clustering algorithm that minimizes intra-cluster variance to obtain the cluster to which each interview question sample belongs.

[0033] The interview question samples can be converted into vector form to obtain the original vectors (which can also be called sequences) corresponding to the interview question samples. Then, the information entropy of the original vectors corresponding to the interview question samples can be calculated, and clustering can be used to determine the cluster to which each interview question sample belongs.

[0034] In one possible implementation, the information entropy of the interview question sample can be calculated as follows: based on the occurrence count and number of valid tokens in the original vector corresponding to each interview question sample, the information entropy of the interview question sample is calculated.

[0035] In practical implementation, the formula for calculating information entropy can be: ; in, H i Indicates the first i Information entropy of a sample of interview questions T i Indicates the first i The unique set of tokens in the original vector corresponding to each interview question sample. n t Represents token t The number of times it appears, N i Indicates the first i The number of valid tokens in the original vector corresponding to each interview question sample.

[0036] In one possible implementation, the cluster to which the interview question sample belongs can be determined as follows: randomly select a preset number of initial cluster centers; assign the information entropy of each interview question sample to the cluster containing its nearest cluster center; update the cluster center of each cluster based on the information entropy currently contained in each cluster; calculate the current objective function value based on the updated cluster center and a preset objective function based on intra-cluster variance; determine whether the current objective function value is the same as the previous objective function value at the time of the last update; if not, re-execute the step of assigning the information entropy of each interview question sample to the cluster containing its nearest cluster center; if yes, determine the cluster to which each interview question sample is currently assigned as the cluster to which the interview question sample belongs.

[0037] The preset number can be set according to actual needs and is not limited here. When selecting the initial cluster centers, they can be selected based on the theoretical range of information entropy, thereby accelerating the clustering speed.

[0038] Step S230: Assign corresponding LoRA ranks to interview question samples from different clusters; wherein, the LoRA rank of the cluster with higher information entropy is greater than the LoRA rank of the cluster with lower information entropy.

[0039] A pre-defined correspondence between clusters and ranks is established. For example, a simple cluster corresponds to a rank of 4, a medium cluster corresponds to a rank of 16, and a complex cluster corresponds to a rank of 32. Based on this correspondence, a corresponding LoRA rank can be assigned to each interview question sample.

[0040] Step S240: Based on the training dataset and the corresponding LoRA rank and answer label, train the interview matrix model to be trained to obtain the trained interview matrix model; during the training process, the cluster-level training loss, cluster-level gradient norm and cluster-level validation loss of each cluster are comprehensively considered, and the LoRA rank is dynamically and adaptively adjusted.

[0041] In some possible embodiments, step S240 above may include: inputting each interview question sample into the interview matrix model to obtain the predicted answer output by the interview matrix model under the corresponding LoRA rank; calculating the sample loss of the interview question sample based on the predicted answer and answer label of each interview question sample; calculating the total training loss, cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster based on the sample loss of each interview question sample; dynamically adjusting the LoRA rank corresponding to each cluster based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster; and updating the LoRA layer parameters of the interview matrix model in reverse using the gradient descent algorithm based on the total training loss and the current LoRA rank corresponding to each cluster.

[0042] The total training loss mentioned above can be the average of the sample losses of all training samples. In this embodiment, when dynamically adjusting the LoRA rank during training, the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss for each cluster are comprehensively considered. The cluster-level loss (including cluster-level training loss and cluster-level validation loss) is used to evaluate the model's performance on each cluster. The cluster-level training loss can be the average of the sample losses of each training sample in the corresponding cluster, and the cluster-level validation loss can be the average of the sample losses of each validation sample in the corresponding cluster. The cluster-level gradient norm is used to evaluate the sensitivity of model parameters to sequences of different complexities. The cluster-level gradient norm can quantify the magnitude of the gradient of model parameters on a specific cluster, mainly reflecting the magnitude of parameter updates during training. The cluster-level gradient norm can be obtained by calculating the average of the L2 norms of the gradients of all parameters associated with the corresponding cluster.

[0043] In one possible implementation, when dynamically adjusting the LoRA rank, the total cluster-level score of each cluster can be calculated based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster; and the LoRA rank of the cluster can be dynamically adjusted based on the relationship between the total cluster-level score of each cluster and the preset high-score threshold and low-score threshold.

[0044] The cluster-level total score can be obtained by normalizing, inverting, and then weighting the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss. Based on this, the formula for calculating the cluster-level total score is: ; in, Indicates the first k The total cluster-level score for each cluster. Indicates the first k Cluster-level training loss for each cluster, Indicates the first k Cluster-level verification loss for each cluster, Indicates the first k The cluster-level gradient norm of each cluster. This represents the maximum value of the cluster-level training loss. This represents the maximum value of the cluster-level verification loss. This represents the maximum allowable value of the gradient norm. , , These represent the weight factors for training loss, gradient norm, and validation loss, respectively.

[0045] The high and low score thresholds mentioned above can be set according to actual needs and are not limited here. For example, the high and low score thresholds can be 0.9 and 0.6 respectively. k If the total cluster score of a cluster is greater than 0.9, the model for that cluster may be overfitting, and the rank of that cluster can be reduced (e.g., by -2); if the rank of the cluster is greater than 0.9, the model for that cluster may be overfitting, and the rank of that cluster can be reduced (e.g., by -2); kIf the total cluster-level score of a cluster is less than 0.6, the model capacity of that cluster is insufficient, and the rank of that cluster can be increased (e.g., by +2); if the total cluster-level score of the first cluster is less than 0.6, the model capacity of that cluster is insufficient, and the rank of that cluster can be increased (e.g., by +2); k If the total cluster-level score of a cluster is between 0.6 and 0.9, the rank of the cluster can be kept unchanged.

[0046] The method for adaptively training an interview matrix model based on a large AI model with low rank provided in this invention introduces information entropy as an input complexity index, performs clustering based on information entropy, and assigns different LoRA ranks to different clusters to match their computational resource requirements. During the training process of the interview matrix model, the LoRA rank of each cluster is dynamically adjusted based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss, ensuring that the rank increase or decrease is strictly matched with the computational requirements. This maximizes parameter utilization efficiency and computational efficiency while ensuring performance, thereby optimizing the allocation of computational resources, enhancing the model's expressiveness, and improving the model's generalization performance.

[0047] The objectives of this invention include: Optimize resource allocation: By using sequence entropy-based clustering, the input sequence is divided into three categories: simple, medium, and complex, and a different rank is assigned to each cluster. This achieves dynamic quantization matching of "low rank for simple tasks and high rank for complex tasks," thereby optimizing the use of computing and memory resources.

[0048] To achieve adaptive fine-tuning, a dynamic rank adjustment mechanism based on cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss is introduced, enabling the model to adaptively optimize and adjust its rank during training to meet the training requirements of different input sequences.

[0049] Maximizing resource efficiency: By using cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss, we ensure that the rank increase / decrease adjustment is strictly matched with the computational requirements, thereby maximizing parameter utilization efficiency and computational efficiency while ensuring performance.

[0050] Improve model performance and generalization ability: Through more refined resource control, the model's fitting and generalization ability can be better balanced, avoiding overfitting or underfitting on some data, thereby improving the overall performance of the model.

[0051] Reduce the burden of hyperparameter tuning: Reduce the dependence on global fixed rank selection, enabling the model to dynamically and automatically adjust the rank based on training, simplifying the tuning process.

[0052] By comprehensively utilizing data from multiple dimensions, including cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss, the dynamic rank adjustment mechanism of this invention can make more intelligent and adaptive decisions, thereby optimizing the allocation of computing resources and improving model training efficiency and final performance.

[0053] To achieve the above objectives, the overall implementation process of this invention includes: First, information entropy is introduced as a metric for measuring input complexity. By measuring the token distribution, the information content and structural complexity of each input sequence are evaluated. Secondly, based on the above entropy values, clustering is performed to automatically divide the training data into three clusters: simple, medium, and complex, achieving hierarchical division without manual annotation. Then, for different clusters, the corresponding LoRA rank is assigned. For example, simple clusters are assigned a smaller rank (r=4), while complex clusters are assigned a larger rank (r=32) to match their computational resources and meet the requirements of expression generalization. Finally, during training, a weighted scoring function is constructed by comprehensively considering the training loss, gradient norm, and validation loss metrics for each cluster. This function is used to dynamically and adaptively adjust the LoRA rank to achieve more refined parameter tuning driven by feedback.

[0054] like Figure 3 As shown, the training process of the interview matrix model mainly includes: input sequence data (i.e., training dataset composed of interview question samples), sequence entropy calculation, clustering based on entropy values, assigning corresponding ranks to clusters, model forward propagation, sequence loss calculation, total loss calculation, cluster-level loss calculation, total loss backpropagation, cluster-level gradient norm calculation, cluster-level total score, rank adjustment decision, LoRA parameter update, and training on the next batch of data.

[0055] To facilitate understanding and simplify the calculation process, the following uses 10 training sequence data (each with a length of...). l =5, vocabulary size| C |=31, inclusive <pad>Based on tokens, this paper details the main process of dynamic adaptive rank adjustment. The vocabulary size corresponds to the number of answers to each interview question. <pad>A token is a special label commonly used in natural language processing tasks to fill in sequence data.

[0056] The main process includes data preparation, sequence entropy calculation and clustering, calculation of multiple metrics such as training loss, gradient norm, and validation loss, as well as dynamic rank adjustment and parameter updates. The specific implementation steps are as follows: Step 1: Input data.

[0057] Training data: 10 training sequences The vocabulary is defined as follows: .length l =5, vocabulary size| C |=31, for ease of demonstration, let's assume 10 training sequences to simulate a small dataset, with their embedding vectors as follows: ; in, The shape is [10, 5], and each row is a sequence, with each element being a token ID.

[0058] In addition, there are 5 verification sequences. The calculation process is the same as that for the training sequence, so the dataset will not be given here.

[0059] Step 2: Calculate sequence entropy.

[0060] Sequence entropy calculation quantifies the information content or complexity of each input sequence, providing a basis for clustering. Information entropy (sequence entropy) measures the uncertainty and randomness of the distribution of elements in a sequence. The higher the entropy, the more uniform the distribution (the greater the uncertainty), indicating that the sequence is more complex or diverse; when the entropy is as low as 0, the distribution is completely deterministic (all probabilities are concentrated at one point), meaning that the sequence is relatively simple.

[0061] For example, an interview question like, "What is the key difference between sequence A, 'HTTP' and 'HTTPS'?" is very basic, belonging to introductory-level computer science knowledge. The answer is direct and clear; the model only needs to activate simple knowledge associations, without requiring complex reasoning or rich contextual modeling. A smaller rank is sufficient to meet the representation requirements, without wasting more parameters.

[0062] Sequence B, "Compare the differences between SSDs and HDDs in terms of data storage structure, read / write performance, and applicable scenarios, and analyze why more and more enterprises tend to adopt SSDs in high-concurrency server environments?", involves multiple dimensions (structure, performance, scenarios, and development trends), requiring comprehensive analysis and reasoning. The model needs to be expressed at multiple levels, including knowledge retrieval, logical reasoning, and scenario application. Using a smaller rank may easily lead to the loss of complex information across multiple dimensions; using a larger rank ensures that the model can express, analyze, and summarize in detail.

[0063] The formula for calculating information entropy can be: ; The parameters are explained as follows: T i :sequence X i Unique set of tokens (excluding) <pad>).

[0064] n t :token t The number of times it appears.

[0065] N i : Number of valid tokens.

[0066] For each input sequence X i Identify unique tokens other than the filler and their respective occurrence counts. n t ); Calculate the total number of valid tokens ( N i ); Applying the entropy calculation formula, we obtain H i Calculate all sequences to obtain a list of entropy values, where each value corresponds to an input sequence and represents the complexity level of that sequence.

[0067] The calculation is based on the sequence 1, 3, 4, 5, 7 as an example: Sequence 1: X 1 = [1,1,1,0,0]; Valid token: [1,1,1] N 1 = 3, T 1 = {1}, n 1 = 3; .

[0068] Sequence 3: X 3 = [1, 2, 3, 4, 0]; Valid tokens: [1,2,3,4] N 3=4, T 3 = {1, 2, 3, 4} n 1= n 2= n 3= n 4 = 1; .

[0069] Sequence 4: X 4 = [3,3,4,5,6]; Valid tokens: [3,3,4,5,6] N 4=5, T 4 = {3, 4, 5, 6} n 3=2, n 4= n 5= n 6 = 1; .

[0070] Sequence 5: X 5 = [2,3,2,3,2]; Valid token: [2,3,2,3,2] N 5=5, T 5 = {2, 3} n 2=3, n 3 = 2; .

[0071] Sequence 7: X 7 = [4, 5, 6, 7, 8]; Valid tokens: [4,5,6,7,8] N 7 = 5 T 7 = {4, 5, 6, 7, 8} n 4= n 5= n 6= n 7= n 8 = 1; .

[0072] All entropy: H =[0,0,2,1.9220,0.9710,1,2.3219,1,2.3219,1].

[0073] Step 3: Clustering & Rank Mapping.

[0074] Clustering groups sequences with similar levels of complexity together, allowing for the assignment of appropriate LoRA rank to each group. Clustering algorithms divide data points (entropy values) into a predetermined number of K clusters (K=3, representing simple, medium, and complex categories, respectively). Figure 4 As shown). Its goal is to minimize the sum of squared distances between data points within each cluster and their cluster center (i.e., the intra-cluster variance). This means that data points within the same cluster should be as close as possible, while data points in different clusters should have as different entropy values ​​as possible. The "center" of each cluster (… ) is the mean of all data points within the cluster.

[0075] Based on entropy H Ten sequences were divided into three clusters using a clustering algorithm, providing a grouping basis for sequence complexity in the rank adjustment of the dynamic LoRA model. Clustering assigned data points to K clusters by minimizing the within-cluster variance. (The last sentence appears to be incomplete and possibly refers to entropy values.) H i The goal is to minimize the following objective function: ; During the iteration, the average entropy of each cluster is recalculated as the new cluster center. Because the initial centers may be random, updating the centers makes the clusters more accurately represent the entropy distribution of the sequence. The formula for calculating the cluster centers is: ; The parameters are explained as follows: C k :cluster k The sequence set contains all data points belonging to that cluster.

[0076] Cluster center: The center (mean vector) of the k-th cluster, the average value of all points in that cluster.

[0077] K : Number of clusters, from 1 to 3 representing simple, medium and complex categories.

[0078] H i : No. i The entropy eigenvectors of the sequence, i =1,2,…,10.

[0079] J The objective function measures the sum of squared distances between a data point and its cluster center, thereby finding the optimal clustering partition that makes each data point as close as possible to its cluster center.

[0080] : indicates the first k All data points in each cluster i Perform a traversal.

[0081] : Represents data points H i To its respective cluster center The square of the Euclidean distance measures the deviation of the point from the center.

[0082] The process is as follows: Random selection K =3 initial cluster centers ( For example, representative values ​​can be selected based on the range of entropy values; The entropy value of each sequence H i It is assigned to the cluster containing the nearest cluster center; Recalculate the center of each cluster and take the mean of all entropy values ​​in that cluster as the new cluster center; Repeat the allocation and update steps until the cluster allocation results and cluster centers no longer change significantly (i.e., the objective function) J convergence).

[0083] Once stable clusters are formed, each cluster (usually sorted by its average entropy value) is mapped to a predefined LoRA rank. For example, the cluster with the lowest average entropy value (representing the simplest sequence) gets the smallest rank, while the cluster with the highest average entropy value (representing the most complex sequence) gets the largest rank.

[0084] Finally, each sequence is assigned a cluster label (e.g., 0, 1, or 2) and an initial LoRA rank (e.g., 4, 16, or 32) is obtained, which will be used in the next batch of training of the model.

[0085] In the specific implementation, the input entropy value is: H =[0,0,2,1.9220,0.9710,1,2.3219,1,2.3219,1].

[0086] The theoretical range of entropy is [0, log2| C |], where | C |=31, log231≈4.9542. Since not all words are used in the sequence, the actual entropy range is [0, 2.3219], reflecting the complexity difference between a single token (entropy = 0) and multiple tokens (high entropy). Clustering uses entropy differences to group elements, ensuring that the LoRA rank distribution matches the sequence complexity. The objective function is: .

[0087] The allocation rule is to assign the sequence i Assigned to cluster k ,make Minimum: .

[0088] The cluster center is: .

[0089] The calculation process is as follows: 1. Initialize the center.

[0090] Three initial centers were randomly selected (for simplicity, representative values ​​were chosen based on the entropy range): .

[0091] The initial center values ​​cover the entropy range [0, 2.3219] to ensure reasonable initial grouping. In practice, the initial center values ​​may be randomly selected.

[0092] 2. First iteration.

[0093] For each H i Assigned to the nearest cluster center: H 1=0: Closest to μ0=0 (difference 0), assigned to cluster 0; H 2=0: Closest to μ0=0, assigned to cluster 0; H 3=2: Closest to μ2=2 (difference 0), assigned to cluster 2; H 4=1.9220: The distance from μ2=2 (difference 0.078) is closer than that from μ1=1 (difference 0.922), so it is assigned to cluster 2; H 5=0.9710: The distance from μ1=1 (difference 0.029) is closer than that from μ0=0 (difference 0.971), so it is assigned to cluster 1; H 6=1: Closest to μ1=1 (difference 0), assigned to cluster 1; H 7=2.3219: The distance μ2=2 (difference 0.3219) is closer than μ1=1 (difference 1.3219), so it is assigned to cluster 2; H 8=1: Closest to μ1=1, assigned to cluster 1; H 9 = 2.3219: distance μ2 = 2 (difference 0.3219), assigned to cluster 2; H 10 =1: Closest to μ1=1, assigned to cluster 1.

[0094] The cluster allocation results are as follows: Cluster 0: C 0={1,2}, H =[0,0]; Cluster 1: C 1={5,6,8,10}, H =[0.9710,1,1,1]; Cluster 2: C 2={3,4,7,9}, H =[2,1.9220,2.3219,2.3219].

[0095] Update Center: ; ; .

[0096] The objective function is: ; Cluster 0: (0 0) 2 +(0 0) 2 =0; Cluster 1: (0.9710 0.99275) 2 +3 (1 0.99275) 2 =0.000473+3 0.000053 = 0.000632; Cluster 2: (2 2.14145) 2 +(1.9220 2.14145) 2 +2 (2.3219 2.14145) 2 =0.0200+0.0482+2 0.0326 = 0.1334; sum: J =0+0.000632+0.1334=0.134032.

[0097] 3. Second iteration.

[0098] Reallocation: H 1=0: Closest to μ0=0, assigned to cluster 0; H 2=0: Closest to μ0=0, assigned to cluster 0; H 3=2: The distance μ2=2.14145 (difference 0.14145) is closer than μ1=0.99275 (difference 1.00725), so it is assigned to cluster 2; H 4=1.9220: The distance from μ2=2.14145 (difference 0.21945) is closer than that from μ1=0.99275 (difference 0.92925), so it is assigned to cluster 2; H5=0.9710: The distance from μ1=0.99275 (difference 0.02175) is closer than that from μ2=2.14145 (difference 1.17045), so it is assigned to cluster 1; H6=1: Distance μ1=0.99275 (difference 0.00725), assigned to cluster 1; H7=2.3219: distance μ2=2.14145 (difference 0.18045), assigned to cluster 2; H8=1: Distance μ1=0.99275 (difference 0.00725), assigned to cluster 1; H9=2.3219: distance μ2=2.14145 (difference 0.18045), assigned to cluster 2; H10=1: distance μ1=0.99275 (difference 0.00725), assigned to cluster 1.

[0099] Cluster allocation (unchanged): Cluster 0: C 0 = {1, 2}; Cluster 1: C 1 = {5, 6, 8, 10}; Cluster 2: C 2 = {3, 4, 7, 9}.

[0100] Update center (unchanged): μ0=0, μ1=0.99275, μ2=2.14145.

[0101] The allocation result is the same as in iteration 1, the center does not need to be updated, and the algorithm converges.

[0102] Objective function (verification): J = 0.134032 (same as iteration 1), confirming convergence.

[0103] Cluster tags: L =[0,0,2,2,1,1,2,1,2,1,2,1].

[0104] Cluster center: μ =[0,0.99275,2.14145].

[0105] Cluster allocation: Cluster 0 (simple): Sequence 1,2 (entropy [0,0], simplest, single token).

[0106] Cluster 1 (Medium): Sequences 5, 6, 8, 10 (entropy [0.9710, 1, 1, 1], medium complexity).

[0107] Cluster 2 (complex): Sequence 3, 4, 7, 9 (entropy [2, 1.9220, 2.3219, 2.3219], most complex).

[0108] Mapping rules: The cluster centers are sorted (from smallest to largest), and the rules are defined as shown in Table 1 below: Table 1

[0109] The mapping rank results are shown in Table 2 below. Figure 4 As shown: Table 2

[0110] Step 4: Forward propagation.

[0111] Forward propagation will obtain the model's prediction for each input sequence, utilizing the specific LoRA rank dynamically assigned to the cluster to which the sequence belongs.

[0112] The input sequence is fed into the Transformer model for processing. The LoRA mechanism within the model modifies the behavior of specific layers (e.g., the Q (Query) and V (Value) weight matrices in the self-attention mechanism) by freezing the pre-trained weights. W Above 0, add a low-rank update quantity. AB (where the matrix) A and B The dimension of the sequence is determined by the rank assigned to the cluster to which the sequence belongs. The model then outputs a probability distribution over the entire vocabulary for the sequence.

[0113] Based on the previous 10 training sequences (length) l =5, vocabulary size| C |=31), entropy value H =[0,0,2,1.9220,0.9710,1,2.3219,1,2.3219,1], and cluster labels of the clustering results. L =[0,0,2,2,1,1,2,1,2,1], and the rank of the allocation. The Transformer model generates a Softmax prediction probability for each sequence, which will be used for training loss calculation and model optimization.

[0114] Preset: l =5: Sequence length.

[0115] | C |=31: Vocabulary size.

[0116] d model =512: Hidden dimension.

[0117] d k = 64: Attention key / value dimension.

[0118] r k : The rank of the allocation.

[0119] Process: For each training sequence Get its cluster label. L i and the corresponding LoRA rank r k Configure the LoRA layer in the Transformer model to use this rank. r k . will sequence Input the model, perform a forward propagation computation, and obtain the probability distribution of the output predicted over all vocabulary items c. These probabilities will be used to calculate the loss function in subsequent training.

[0120] Update attention (Q, V) weight formula: ; Transformer prediction probability formula: ; The input sequence has the shape [10,5]. M Attention mask, shape [10,1,1,5], to mask <pad>token; L Cluster label determines LoRA parameters; : Predicted probability, with shape [10,5,31].

[0121] Taking the sequence 1, 3, 4, 5, 7 as an example, and assuming that the Transformer outputs the softmax probability... y (To illustrate the calculation process, the following probability data are simplified with assumptions; similarly, other sequences are assumed to be similarly simplified.) Sequence 1: ( X 1 = [1,1,1,0,0], cluster 0, r 0=4): ; .

[0122] Sequence 3: ( X 3 = [3,3,4,5,6], cluster 2. r 2=32): ; .

[0123] Sequence 4: ( X 4 = [2,3,2,3,2], cluster 2, r 2=32): ; y 4 = [0,0,1,…].

[0124] Sequence 5: ( X 5 = [1,1,2,2,0], cluster 1, r 2=16): ; y 5 = [0,0,1,…].

[0125] Sequence 7: ( X 7 = [4,5,6,7,8], cluster 2. r 2=32): ; y 7 = [0, 1, 0, ...].

[0126] Step 5: Sequence loss.

[0127] Sequence loss is used to quantify the difference between the model's prediction of a single sequence and the true distribution. Cross-entropy is used to measure the probability distribution predicted by the model (…). ) and the actual target distribution after one-hot encoding ( y The difference between the two values ​​is as follows: The lower the loss value, the higher the prediction accuracy of the model, and vice versa.

[0128] The calculation process, for each sequence i The predicted probability output by the model is compared with the actual target distribution using cross-entropy loss calculation to obtain a scalar loss value. This value reflects the model's prediction error on that particular sequence.

[0129] The loss is calculated for each training sequence, using the following formula: ; The loss formula defines the cross-entropy loss for each training sequence. This evaluates the difference between the model's predicted probabilities and the true distribution (the difference between the highest probability and the corresponding target). The symbols in this formula have the following meanings: The loss value for the training sequence represents the difference in information entropy between the model's prediction of the sequence and the true label.

[0130] Sum the values ​​for each cluster c (from 1 to 31) in the vocabulary, covering all possible token IDs (excluding...). <pad>(ID=0).

[0131] | C |: Vocabulary size defines the output space, which is the total number of different tokens that the model can predict.

[0132] One-hot encoding of real labels (the first) i (Whether the word class is a real word class) is 1 if correct and 0 otherwise.

[0133] The probability predicted by the model. log It is based on the natural logarithm (2.71828) (subsequent) log (The meanings are all based on this). This is the model prediction of the [number]. i The probability of word class. This probability is obtained from the output of the last layer of the model (e.g., the Softmax layer).

[0134] Only real word classes k of y k =1, the rest y i =0. Therefore, after summing, only terms of the real cluster remain. ,because Adding a negative sign results in a loss. .

[0135] Cross-entropy loss measures the probability distribution predicted by the model. ) and the true label distribution ( The difference lies in the probability distribution between the model's predictions and the actual labels. The loss is zero when the model's predicted probabilities perfectly match the true labels (i.e., a probability of 1 for the correct token cluster and 0 for others). The loss increases if the model's predictions deviate from the true labels (e.g., a lower probability of predicting the correct token cluster). By minimizing this loss, the model, once trained, can accurately predict the next token in the sequence. It's worth noting that calculating the cross-entropy loss on the validation set follows the same process; the validation set loss is primarily used to evaluate the model's generalization ability on unseen data.

[0136] Using sequences 1, 3, 4, 5, and 7 as examples, the calculation process is the same for other sequences, including the validation set sequences. Here, we assume the prediction is correct, and the prediction probability... With real labels If the prediction and the actual value are consistent, the true label with a higher probability is 1, and vice versa, both are 0. However, there is still a loss, which needs to be adjusted through backpropagation. Only if both the prediction and the actual value are 1 will the loss be 0.

[0137] Sequence 1: ; Sequence 3: ; Sequence 4: ; Sequence 5: ; Sequence 7: .

[0138] Step 6: Total Loss.

[0139] Total loss is a comprehensive metric that measures the overall performance of a model across the entire training batch or dataset. This single value is primarily used to calculate gradients and update model parameters via backpropagation.

[0140] The process is as follows: calculate the loss of each sequence. The losses are summed directly and then divided by the total number of sequences in the current training batch. N This yields a single scalar value. This represents the model's average prediction error on the current training data. The calculation formula is as follows: ; Where N is the number of training sequences, which is 10 here; This represents the total training loss.

[0141] In this example, the total loss is:

[0142] .

[0143] Step 7: Cluster-level loss.

[0144] Cluster-level loss is used to evaluate the model's performance on each cluster of sequence data complexity. This is a key input metric for the dynamic rank adjustment mechanism because it reflects how well the currently assigned rank fits each cluster. For each cluster, the cluster-level loss is the average of the loss values ​​of all sequences within it, providing a targeted performance metric and adjustment direction for different subsets of simple, moderate, and complex data.

[0145] The calculation process is as follows: for each cluster k Find all clusters C k sequence i The independent loss values ​​of these sequences L i (From the calculation steps above) Sum the results and then divide by the number of sequences in the cluster. Finally, an average loss value is calculated for each cluster. This average loss reflects the prediction error of the sequence within the cluster; a high loss indicates the need for a higher rank. The calculation formula is: ; in, For clusters k number of sequences; For the first i The loss value for each sequence represents the difference in information entropy between the model's prediction of that sequence and the true label. For clusters k The average loss.

[0146] The specific calculations are as follows: Cluster 0 ( ): ; Cluster 1 ( ): ; Cluster 2 ( ): .

[0147] The final result is: .

[0148] Step 8: Cluster-level gradient norm.

[0149] The cluster-level gradient norm is calculated for each cluster to evaluate the sensitivity of model parameters to sequences of different complexities, providing a basis for dynamic LoRA rank adjustment. The cluster-level gradient norm quantifies the magnitude of the gradient of model parameters on a specific cluster, primarily reflecting the magnitude of parameter updates during training. A larger gradient norm indicates that the model requires a higher rank. The calculation process involves normalizing the gradient obtained from backpropagation to the average value per sequence using the L2 norm. This gradient norm metric quantifies the total loss relative to a specific cluster. k LoRA parameters (i.e., matrix) A k , B k The L2 norm is the magnitude of the gradient. It provides a scalar measure of this gradient magnitude. The average gradient norm is obtained by normalizing by dividing by the number of sequences in the cluster.

[0150] Process: Based on total loss During backpropagation, determine the cluster k Each LoRA parameter is related gradient Calculate all clusters k The L2 norm of the relevant parameter gradients. Divide this norm by the cluster. k number of sequences in The average gradient norm is obtained. G k The calculation formula is: .

[0151] The meanings of each symbol are as follows: :cluster k The corresponding LoRA parameter set (e.g.) A Q , B Q , A V , B V ); Total loss For parameters The partial derivatives (gradient); :cluster k The L2 norm of the parameter gradient is used to measure the magnitude of the gradient; :cluster k The number of sequences in the sequence, used for normalization; G k :cluster k The average gradient norm represents the gradient contribution of each sequence.

[0152] Calculate an average gradient norm for each cluster G k This norm reflects the model's adaptability to the data. Corresponding to the input sequence, simple sequences (low entropy) typically have small gradients, while complex sequences (high entropy) may have large gradients. G k This suggests that the LoRA rank needs to be increased to enhance the model capacity; low rank G k The value may indicate that the parameters of the cluster have stabilized, or that its rank has the potential to decrease.

[0153] Cluster allocation: Cluster 0 (simple): Sequence 1, 2 ( L =[0,0], =2); Cluster 1 (Medium): Sequences 5, 6, 8, 10 L =[1,1,1,1], =4); Cluster 2 (complex): Sequences 3, 4, 7, 9 L =[2,2,2,2], =4); Gradient estimation: Assumption and It is proportional to the gradient contribution of the total loss normalization.

[0154] Cluster-level gradient norm calculation: Cluster 0 (Sequence 1, 2): ; To simplify calculations, we assume the gradient... ,but: ; .

[0155] Cluster 1 (sequences 5, 6, 8, 10): ; Assuming gradient ,but: ; .

[0156] Cluster 2 (sequences 3, 4, 7, 9): ; Assuming gradient ,but: ; .

[0157] The final result is: .

[0158] Cluster 2 height G The number 2 reflects the high complexity and large gradient of its sequence (e.g., 3, 4, 7, 9), suggesting that the rank may need to be increased. The low rank of cluster 0... G A value of 0 indicates that the simple sequence is a good fit and the rank can be reduced.

[0159] Step 9: Cluster-level scoring and rank decision.

[0160] Cluster-level score: A comprehensive score is calculated for each cluster, which incorporates its normalized average training loss. Normalized average gradient norm and the normalized average verification loss Based on the performance of each cluster, the final decision is made to either increase or decrease the rank, or to maintain the existing rank allocation.

[0161] like Figure 5 As shown, after forward propagation of the model, sequence loss is calculated to obtain cluster-level training loss score, cluster-level validation loss score, and cluster-level gradient norm score. Based on the cluster-level training loss score, cluster-level validation loss score, and cluster-level gradient norm score, the cluster-level total score is calculated. Based on the cluster-level total score, a rank adjustment decision is made. The rank adjustment decision includes increasing the rank (+r) when the cluster-level total score is less than the minimum threshold (i.e., the aforementioned low score threshold), decreasing the rank (-r) when the cluster-level total score is greater than the maximum threshold (i.e., the aforementioned high score threshold), and keeping the rank unchanged when the cluster-level total score is within the threshold range (between the minimum threshold and the maximum threshold). Then, backpropagation parameter updates are performed.

[0162] Cluster-level score: for each cluster k Calculate a score It is calculated based on the average training loss, average gradient norm, and average validation loss of this cluster. The formula is as follows: .

[0163] Parameter explanation: : No. k The independent score of each cluster ranges from [0,1], with a higher score indicating better model performance for that cluster.

[0164] and : No. k The average training loss and average validation loss of each cluster are the average of the sequence losses within the cluster, reflecting the local fitting quality of that cluster.

[0165] : No. k The average gradient norm of a cluster is the average L2 norm of the gradient norms of all sequences within the cluster, reflecting the stability of the cluster.

[0166] and The maximum possible value of the loss, used for normalization. For example, for cross-entropy loss, the theoretical maximum value is... Based on the natural logarithm ln (misprediction, is the maximum loss of a single token).

[0167] The maximum allowed value of the gradient norm, used for normalization, is determined based on the initial training gradient peak or empirical value.

[0168] , and Three weighting factors are used to balance the importance of training loss, gradient norm, and validation loss in the overall score.

[0169] and Decision threshold, used to determine whether to adjust the rank (e.g., 0.6 and 0.9), is configurable and can be adjusted according to task requirements.

[0170] and The training loss and validation loss are converted into scores, with higher scores indicating better performance (1 for zero loss and 0 for maximum loss).

[0171] Convert the gradient norm to a score. If the gradient is small relative to a certain maximum value, it may be considered a sign of good stability or sufficient capacity.

[0172] Score for each cluster The evaluation is calculated independently based on its own cluster and does not interact with other clusters. and This is a normalized expression used to measure the quality of the fit; This is a normalized expression used to measure optimization stability. The normalized loss is converted into a score, which ranges between [0,1], facilitating comparison of model performance. Chinese: When (Perfect prediction), score: .when The score is .

[0173] Rank decision process, for each cluster k : Using predefined theoretical maximum values , and For its average training loss and average gradient norm The average validation loss is normalized. Then, the score for the cluster is calculated using the given three weighting factors. .

[0174] if < (e.g., 0.6) If the model capacity of this cluster is insufficient, increase the rank of the cluster, such as +2 (threshold and rank factor are configurable).

[0175] if > (e.g., 0.9) The model for this cluster may be overfitting, so reduce the rank of the cluster, such as -2.

[0176] if ≤ ≤ The rank of the cluster remains unchanged.

[0177] Decision-making logic: If < (e.g., 0.6) indicates poor performance (high loss / large gradient norm may mean insufficient capacity), in which case the rank of the cluster should be increased (e.g., r k ’ = r k +2), the adjustment factor 2 is configurable. If > (e.g., 0.9) indicates very good performance (low loss / small gradient norm). In this case, reducing the rank can be considered to save resources or prevent overfitting (e.g., r k ’ = r k -2). In other cases, the current rank remains unchanged. Then, an updated LoRA rank is generated for each cluster; these new ranks will be used in subsequent training iterations to achieve a dynamic adaptation mechanism.

[0178] The rank adjustment decision rule for each cluster is described above. Low scores (e.g.) =0.5807) indicates that the cluster has a high loss or is unstable in optimization; increasing the rank can improve capacity. High scores (e.g., =0.8011) indicates a good fit; maintaining or reducing the rank avoids overfitting. To reduce frequent adjustments to the rank, batch cumulative sliding window averaging can also be used to smooth the training loss, validation loss, and gradient norm of each cluster, reducing the impact of transient fluctuations.

[0179] Step 10: Reverse update of total loss.

[0180] Adjust the trainable LoRA matrices A and B across all clusters to minimize the model's overall prediction error. Calculate the gradient (partial derivative) of the total backpropagation loss with respect to each trainable parameter (an element in the A and B matrices of the cluster). The parameters are updated in small steps (controlled by the learning rate η) in the opposite direction of their gradients (note that the validation loss does not have its gradient calculated and is not included in the backpropagation parameter updates).

[0181] Calculate total loss The gradient with respect to all elements in the LoRA matrices A and B (using their currently dynamically assigned rank) of all activities in all clusters. The gradient is then transferred from... (The sentence is incomplete and requires further context to translate accurately.) ΔW = A · B Backpropagation is then performed on A and B. The matrices are updated using the calculated gradients and learning rate η, and a similar operation is performed on B. The LoRA parameters A and B matrix parameters are updated and optimized. After this step, the updated AB parameters and the newly adjusted rank are applied to the next training cycle.

[0182] Based on total loss The goal of performing reverse updates is to optimize the trainable parameters. θ (Update LoRA's weight matrices A and B, original weights) W (0 remains unchanged). The formula for the reverse update is based on gradient descent: ; Update A and B respectively: ; .

[0183] The parameters are explained as follows: θ Trainable parameters, such as those for LoRA. , ,in r It is the current rank.

[0184] : No. t The model parameter vector (containing all weights) at the next iteration.

[0185] : The updated model parameter vector.

[0186] Loss function For parameters θ The gradient vector.

[0187] : The total loss scalar of the training dataset, which measures the model's prediction error across all sequences.

[0188] η The learning rate is the step size that controls the parameters during each update (e.g., ...). η =0.001).

[0189] : Dimensionality reduction parameter matrix of the low-rank matrix before update A It is initially a random Gaussian distribution.

[0190] The upgraded parameter matrix of the low-rank matrix before the update B It is initially a zero matrix.

[0191] Loss function pair A The gradient.

[0192] Loss function pair B The gradient.

[0193] This invention proposes a LoRA rank dynamic adaptive adjustment strategy based on data characteristics and the model's real-time learning state. It divides the input data into clusters of different complexity levels by introducing data entropy analysis and clustering algorithms. Subsequently, for each cluster, multi-dimensional performance metrics, including training loss, validation loss, and gradient norm, are calculated. Based on the weighted combined scores of these metrics, appropriate ranks are dynamically assigned to different data clusters. This refined and adaptive resource allocation strategy aims to optimize model parameter efficiency, improve model performance and generalization ability when handling heterogeneous complexity data, and reduce the dependence on and adjustment difficulty of globally fixed rank selection in traditional LoRA methods.

[0194] The key points of the dynamic LoRA rank adjustment scheme provided in this embodiment of the invention include: Entropy-based data complexity analysis and classification: Calculate the entropy of each input sequence to quantify its information content and complexity; use clustering algorithms to divide sequences with different entropy values ​​into different clusters (simple, medium, complex, etc.).

[0195] Cluster-level performance metrics calculation: For each data cluster, key performance metrics are calculated independently, including multi-dimensional performance metrics such as cluster average training loss, gradient norm of LoRA parameters associated with the cluster (reflecting learning difficulty and sensitivity to parameter updates), and cluster average validation loss (used to evaluate generalization ability), to obtain a cluster-level comprehensive total score.

[0196] Dynamic rank adjustment mechanism: The training loss, gradient norm, and validation loss of each cluster are weighted and combined to form a comprehensive evaluation score, thereby dynamically increasing or decreasing the rank of each cluster.

[0197] To avoid misjudgments that may result from a single metric: relying solely on cluster loss may ignore the learning dynamics of the model (such as failing to distinguish whether the model is actively learning or has stagnated under high loss); relying solely on cluster gradient norm cannot directly measure the final fitting effect and may make inappropriate rank reduction decisions in cases of learning saturation or insufficient capacity; combining both can significantly reduce the risk of such misjudgments.

[0198] Integration with standard training processes: This dynamic adjustment mechanism is embedded into the model's standard training loop and executed after each training batch (or several batches). This allows the rank adjustment to respond promptly to the model's learning dynamics, which helps to improve the model's ability to fit complex data while avoiding resource waste and promoting the convergence of parameters to a better and more stable state.

[0199] In summary, the above-mentioned scheme overcomes the fixed-rank limitation of traditional LoRA by combining sequence complexity and multi-dimensional performance indicators, providing a practical and flexible solution for efficiently fine-tuning large language models, and offering important reference value for further exploration of more complex dynamic adaptive adjustment strategies in the future.

[0200] Corresponding to the above-described method for adaptively training an interview matrix model based on a large AI model with low rank, this invention also provides a system for adaptively training an interview matrix model based on a large AI model with low rank. See [link to system]. Figure 6 The diagram shown illustrates the structure of a system for low-rank adaptive training of an interview matrix model based on a large AI model. The system includes: The data acquisition module 601 is used to acquire the training dataset and its answer labels. The interview question samples in the training dataset are divided into two categories: training samples and validation samples. The sample clustering module 602 is used to cluster the information entropy of each interview question sample using a clustering algorithm to obtain the cluster to which each interview question sample belongs. The rank allocation module 603 is used to assign corresponding LoRA ranks to interview question samples from different clusters; wherein, the LoRA rank corresponding to the cluster with higher information entropy is greater than the LoRA rank corresponding to the cluster with lower information entropy. The model training module 604 is used to train the interview matrix model to be trained based on the training dataset, the corresponding LoRA rank and answer labels, and obtain the trained interview matrix model. During the training process, the cluster-level training loss, cluster-level gradient norm and cluster-level validation loss of each cluster are comprehensively considered, and the LoRA rank is dynamically and adaptively adjusted.

[0201] The system for adaptively training an interview matrix model based on a large AI model with low rank provided in this invention introduces information entropy as an input complexity indicator. Clustering is performed based on information entropy, and different LoRA ranks are assigned to different clusters to match their computational resource requirements. During the training process of the interview matrix model, the LoRA rank of each cluster is dynamically adjusted based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss. This ensures that the rank adjustment strictly matches the computational requirements, maximizing parameter utilization efficiency and computational efficiency while guaranteeing performance. As a result, computational resource allocation is optimized, model expressiveness is enhanced, and model generalization performance is improved.

[0202] Furthermore, the sample clustering module 602 is specifically used to: calculate the information entropy of the interview question sample based on the valid tokens in the original vector corresponding to each interview question sample; and cluster the information entropy of each interview question sample using a clustering algorithm that minimizes the intra-cluster variance to obtain the cluster to which each interview question sample belongs.

[0203] Furthermore, the sample clustering module 602 is also used to: calculate the information entropy of the interview question sample based on the occurrence frequency and number of valid tokens in the original vector corresponding to each interview question sample.

[0204] Furthermore, the sample clustering module 602 is also used to: randomly select a preset number of initial cluster centers; assign the information entropy of each interview question sample to the cluster containing the nearest cluster center; update the cluster center of each cluster based on the information entropy currently contained in each cluster; calculate the current objective function value based on the updated cluster center and a preset objective function based on intra-cluster variance; determine whether the current objective function value is the same as the previous objective function value during the last update; if not, re-execute the step of assigning the information entropy of each interview question sample to the cluster containing the nearest cluster center; if yes, determine the cluster currently assigned to each interview question sample as the cluster to which the interview question sample belongs.

[0205] Furthermore, the aforementioned model training module 604 is specifically used for: inputting each interview question sample into the interview matrix model to obtain the predicted answer output by the interview matrix model under the corresponding LoRA rank; calculating the sample loss of the interview question sample based on the predicted answer and answer label of each interview question sample; calculating the total training loss, cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster based on the sample loss of each interview question sample; dynamically adjusting the LoRA rank corresponding to each cluster based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster; and updating the LoRA layer parameters of the interview matrix model in reverse using the gradient descent algorithm based on the total training loss and the current LoRA rank corresponding to each cluster.

[0206] Furthermore, the model training module 604 is also used to: calculate the total cluster score of each cluster based on the cluster-level training loss, cluster-level gradient norm and cluster-level validation loss of each cluster; and dynamically adjust the LoRA rank corresponding to the cluster based on the relationship between the total cluster score of each cluster and the preset high score threshold and low score threshold.

[0207] The system for training an interview matrix model based on a low-rank adaptive training of a large AI model provided in this embodiment has the same implementation principle and technical effects as the aforementioned method embodiment for training an interview matrix model based on a low-rank adaptive training of a large AI model. For the sake of brevity, any parts not mentioned in the system embodiment for training an interview matrix model based on a low-rank adaptive training of a large AI model can be referred to the corresponding content in the aforementioned method embodiment for training an interview matrix model based on a low-rank adaptive training of a large AI model.

[0208] like Figure 7 As shown, an electronic device 700 provided in this embodiment of the invention includes: a processor 701, a memory 702 and a bus. The memory 702 stores a computer program that can run on the processor 701. When the electronic device 700 is running, the processor 701 and the memory 702 communicate through the bus. The processor 701 executes the computer program to implement the above-mentioned method for training an interview matrix model based on a low-rank adaptive training of a large AI model.

[0209] Specifically, the memory 702 and processor 701 mentioned above can be general-purpose memory and processor, without any specific limitations here.

[0210] This invention also provides a computer-readable storage medium storing a computer program. When a processor runs this computer program, it executes the method described in the preceding method embodiments for training an interview matrix model based on a large AI model with low rank. The computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.< / pad> < / pad> < / pad> < / pad> < / pad>

Claims

1. A method for training an interview matrix model based on a low-rank adaptive training of a large AI model, characterized in that, include: Obtain the training dataset and its answer labels. The interview question samples in the training dataset are divided into two categories: training samples and validation samples. The information entropy of each interview question sample is clustered using a clustering algorithm to obtain the cluster to which each interview question sample belongs; Assign corresponding LoRA ranks to the interview question samples of different clusters; wherein, the LoRA rank corresponding to the cluster with higher information entropy is greater than the LoRA rank corresponding to the cluster with lower information entropy; Based on the training dataset, the corresponding LoRA rank, and the answer label, the interview matrix model to be trained is trained to obtain the trained interview matrix model. During the training process, the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster are comprehensively considered, and the LoRA rank is dynamically and adaptively adjusted.

2. The method for training an interview matrix model based on a low-rank adaptive training of a large AI model according to claim 1, characterized in that, The step of clustering the information entropy of each interview question sample using a clustering algorithm to obtain the cluster to which each interview question sample belongs includes: The information entropy of each interview question sample is calculated based on the valid token in the original vector corresponding to each interview question sample. Clustering of the information entropy of each interview question sample is performed using a clustering algorithm that minimizes intra-cluster variance, thereby obtaining the cluster to which each interview question sample belongs.

3. The method for training an interview matrix model based on a low-rank adaptive training of a large AI model according to claim 2, characterized in that, The step of calculating the information entropy of each interview question sample based on the valid tokens in the original vector corresponding to each interview question sample includes: The information entropy of each interview question sample is calculated based on the number of occurrences of each valid token and the number of valid tokens in the original vector corresponding to each interview question sample.

4. The method for training an interview matrix model based on a low-rank adaptive training of a large AI model according to claim 2, characterized in that, The clustering of the information entropy of each interview question sample using a clustering algorithm that minimizes intra-cluster variance, to obtain the cluster to which each interview question sample belongs, includes: Randomly select a preset number of initial cluster centers; The information entropy of each interview question sample is assigned to the cluster containing the nearest cluster center. Update the cluster center of each cluster based on the entropy of each information currently contained in each cluster; The current objective function value is calculated based on the updated cluster centers and the preset objective function based on intra-cluster variance. Determine whether the current objective function value is the same as the previous objective function value at the time of the last update; If not, repeat the step of assigning the information entropy of each interview question sample to the cluster containing the nearest cluster center; If so, the cluster currently assigned to each interview question sample is determined as the cluster to which the interview question sample belongs.

5. The method for training an interview matrix model based on a low-rank adaptive training of a large AI model according to claim 1, characterized in that, The step of training the interview matrix model to be trained based on the training dataset and the corresponding LoRA rank sum and answer labels to obtain the trained interview matrix model includes: Each interview question sample is input into the interview matrix model to obtain the predicted answer output by the interview matrix model using the corresponding LoRA rank. The sample loss of each interview question sample is calculated based on the predicted answer and answer label of each interview question sample. Based on the sample loss of each interview question sample, the total training loss, cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster are calculated. The LoRA rank of each cluster is dynamically adjusted based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss. Based on the total training loss and the current LoRA rank corresponding to each cluster, the LoRA layer parameters of the interview matrix model are updated in reverse using the gradient descent algorithm.

6. The method for training an interview matrix model based on a low-rank adaptive training of a large AI model according to claim 5, characterized in that, The step of dynamically adjusting the LoRA rank of each cluster based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss includes: The total cluster score for each cluster is calculated based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss for each cluster. The LoRA rank of each cluster is dynamically adjusted based on the relationship between the total cluster-level score of each cluster and the preset high score threshold and low score threshold.

7. A system for adaptively training an interview matrix model based on a large AI model with low rank, characterized in that, include: The data acquisition module is used to acquire the training dataset and its answer labels. The interview question samples in the training dataset are divided into two categories: training samples and validation samples. The sample clustering module is used to cluster the information entropy of each interview question sample using a clustering algorithm to obtain the cluster to which each interview question sample belongs; The rank allocation module is used to assign corresponding LoRA ranks to the interview question samples of different clusters; wherein, the LoRA rank corresponding to the cluster with higher information entropy is greater than the LoRA rank corresponding to the cluster with lower information entropy; The model training module is used to train the interview matrix model to be trained based on the training dataset, the corresponding LoRA rank, and the answer label, so as to obtain the trained interview matrix model. During the training process, the LoRA rank of each cluster is dynamically and adaptively adjusted by comprehensively considering the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss.

8. The system for training an interview matrix model based on a low-rank adaptive training of a large AI model according to claim 7, characterized in that, The model training module is specifically used for: inputting each interview question sample into the interview matrix model to obtain the predicted answer output by the interview matrix model under the corresponding LoRA rank; calculating the sample loss of each interview question sample based on the predicted answer and answer label; calculating the total training loss, cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster based on the sample loss of each interview question sample; dynamically adjusting the LoRA rank corresponding to each cluster based on the cluster-level training loss, cluster-level gradient norm, and cluster-level validation loss of each cluster; and updating the LoRA layer parameters of the interview matrix model in reverse using the gradient descent algorithm based on the total training loss and the current LoRA rank corresponding to each cluster.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for training an interview matrix model based on a low-rank adaptive training of an AI large model, as described in any one of claims 1-6.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by the processor, executes the method for training an interview matrix model based on a low-rank adaptive training of a large AI model, as described in any one of claims 1-6.

Images