A Low-Rank Adaptive Fine-Tuning Method Based on Nested Diagonal Blocks and Its Application
By using a nested diagonal block low-rank adaptive method, the non-convex optimization and expressive power bottleneck problems in the fine-tuning of large language models are solved, achieving stable and efficient model adaptation in the field of mental health, and ensuring the integrity of pre-trained knowledge and data quality.
Patent Information
- Application Number
- CN202611321437.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing large language models suffer from non-convex optimization problems, expressive power bottlenecks, and pre-training knowledge corruption issues during fine-tuning in vertical domains, especially in the field of mental health where they are inadequate and data quality is difficult to guarantee.
We employ a nested diagonal block low-rank adaptive method, which initializes and freezes the outer low-rank projection matrix by activation-aware singular value decomposition, replaces the inner structure with a diagonal block matrix, and combines sparse computation optimization to construct a high-quality domain dataset and perform fine-tuning.
It significantly improves the training stability and expressiveness of the model, avoids catastrophic forgetting, maintains the integrity of pre-trained knowledge, and enables efficient fine-tuning on consumer-grade hardware, making it suitable for highly sensitive fields such as mental health question answering.
Smart Images

Figure CN123117263A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing and large language model fine-tuning technology, and particularly to a large model fine-tuning method and its application. Background Technology
[0002] In recent years, large language models have achieved remarkable results in general natural language processing tasks, but adaptation to vertical domains still faces significant challenges. While full fine-tuning can improve domain performance, its computational and storage overhead is extremely large. Parameter-Efficient Fine-Tuning (PEFT) techniques, represented by Low-Rank Adaptation (LoRA), significantly reduce resource consumption by introducing low-rank matrix multiplication to approximate weight updates, and have become the current mainstream paradigm.
[0003] However, existing LoRA and its variants still have three major general core technology limitations:
[0004] 1. Optimizing non-convexity: Traditional LoRA and the inner structure of nested LoRA usually use the product of two low-rank matrices (such as B'A'). This non-convex parameterization method makes the optimization of terrain complex, which is very easy to get trapped in local optima. It also has gradient vanishing and coupling problems, which makes the model unstable in convergence during fine-tuning.
[0005] 2. Expressive power bottleneck problem: Low-rank product structures are limited by the size of the inner rank, making it difficult to fully capture the fine feature interactions and semantic representations in complex tasks, and thus perform poorly in domains that require high-precision semantic understanding.
[0006] 3. Problem of pre-trained knowledge destruction: Existing methods often use random initialization, ignoring the rich semantics already contained in the pre-trained weights. This can easily destroy core knowledge during fine-tuning, leading to catastrophic forgetting.
[0007] To address the aforementioned issues, existing research has explored different directions. Nested Low-Rank Adaptation (NoRA) proposes a nested low-rank structure and uses Activation-aware Singular Value Decomposition (AwSVD) to initialize the outer matrix, significantly improving parameter efficiency and knowledge protection capabilities. However, its inner layer still uses a low-rank product structure, failing to fundamentally solve the non-convex optimization problem. Diagonal Blocks (DiaBlo) has demonstrated the superiority of the diagonal block structure in fine-tuning. By directly updating the diagonal blocks, it avoids the non-convex optimization of matrix multiplication, resulting in more stable training and stronger expressive power. However, this method directly updates the diagonal blocks of the original weights, lacking a protection mechanism for the pre-trained core semantic subspace, which can easily compromise the model's fundamental capabilities. Currently, however, there is no universal fine-tuning framework that organically combines the knowledge protection mechanism of NoRA with the optimization stability advantages of DiaBlo.
[0008] In particular, the aforementioned problems are amplified in highly specialized and sensitive vertical fields such as mental health: on the one hand, the mental health field requires models to identify complex emotional expressions and risk signals; on the other hand, this field has high requirements for the knowledge basis, safety boundaries, and privacy protection of responses, and hallucinations, overdiagnosis, or dangerous guidance may have adverse consequences. Furthermore, existing publicly available data often suffers from problems such as difficulty in tracing the source, a high proportion of single-round question-and-answer sessions, insufficient samples of multi-round support processes and safety preferences, and inadequate source isolation between training and test sets. Simply increasing the sample size cannot guarantee data quality and the verifiability of original contributions.
[0009] Therefore, there is an urgent need for a general fine-tuning framework that can overcome the non-convexity of low-rank optimization, improve expressiveness and training stability, effectively retain the core knowledge of pre-training, and adapt to high-quality domain data, as well as a secure application scheme for this framework in highly sensitive vertical domains. Summary of the Invention
[0010] To address the shortcomings of existing technologies, this invention provides a nested diagonal block low-rank adaptive fine-tuning method and its application, achieving a balance between robustness and efficiency. It employs activation-aware singular value decomposition for initialization and freezing of the outer low-rank projection matrix, accurately preserving the core semantic subspace of the pre-trained model and fundamentally avoiding catastrophic forgetting during fine-tuning. By using a diagonal block matrix instead of the traditional low-rank product as the inner trainable structure, it completely eliminates the inherent non-convex optimization obstacle of low-rank adaptation, ensuring stable convergence during training. Combined with nested architecture design and sparse computation optimization, it significantly improves the model's feature representation ability while maintaining parameter efficiency comparable to standard LoRA, making it widely applicable to multiple vertical fields such as mental health question answering, code generation, and mathematical reasoning. Furthermore, through traceable knowledge units, hierarchical scene expansion, multi-model cross-validation, manual formal review, and source-isolated partitioning, it forms a recordable, reproducible, and group-verifiable data construction process.
[0011] The objective of this invention is achieved as follows: a method for fine-tuning a large language model based on nested diagonal block low-rank adaptation, comprising the following steps:
[0012] Step 1) Construct a high-quality instruction fine-tuning dataset for the target domain through source tracing, hierarchical scenario expansion, multi-model cross-validation, and source isolation partitioning;
[0013] Step 2) Perform activation-perceptual singular value decomposition on the target weight matrix of the large language model, extract the outer low-rank projection matrix and freeze it to preserve the core semantic subspace of the pre-trained model.
[0014] Step 3) Introduce a diagonal block structure between the outer low-rank projection matrices to construct the inner trainable diagonal block adaptation matrix M, and then apply the inner trainable diagonal block adaptation matrix... Perform zero initialization;
[0015] Step 4) Fine-tune the dataset using the high-quality instructions, combining the frozen outer low-rank projection matrix with the inner trainable diagonal block adaptation matrix. Efficiently fine-tuning parameters of large language models;
[0016] Step 5) Apply the trained inner layer trainable diagonal block adaptation matrix The weight increment is obtained by merging the frozen outer low-rank projection matrix and injecting it into the original model to obtain the domain-adapted model.
[0017] Furthermore, step 1) specifically includes:
[0018] 1-1) Select professional literature and industry guidelines in the target field with legal usage rights as data sources, and segment the original text into traceable knowledge units with source document identifiers, chapter identifiers and knowledge fragment identifiers;
[0019] 1-2) Establish a data classification system according to the theme, application scenario, user characteristics, risk level and support strategy of the target domain, and use the first major language model to generate single-turn instruction pairs, multi-turn dialogues and security preference data with traceable sources based on the traceable knowledge units. The security preference data includes inappropriate candidate responses and security rewritten responses generated for the same user's question.
[0020] 1-3) Use at least two second-largest language models, different from the first-largest language model, to cross-validate the consistency of the source, relevance of the question, support, security and completeness of the generated samples. Combine deduplication, privacy information detection, risk rule matching and manual formal review to remove unqualified samples. Divide the training set, validation set and test set according to the source document identifier or knowledge fragment identifier to ensure that the same source fragment is not repeated across sets.
[0021] Furthermore, step 2) specifically includes:
[0022] 2-1) Obtain the input activation matrix of the pre-trained model on a general corpus. , where the target weight matrix Input activation matrix , Indicates the number of samples collected (or batch size). This represents the input feature dimension of the target linear layer; based on the input activation matrix... Calculate the standard deviation diagonal matrix of the input activations. : , among which, diag( This indicates extracting the diagonal elements of a matrix and constructing a diagonal matrix;
[0023] 2-2) Use the input activation standard deviation diagonal matrix obtained in step 2-1 For the target weight matrix Activation weighting is performed to obtain the activation-aware weight matrix. : =WS;
[0024] 2-3) The activation-sensory weight matrix obtained in step 2-2 Perform singular value decomposition ,in, It is a left singular matrix. It is a singular value matrix. A right singular matrix The transpose. Further extraction before... The maximum singular values and their corresponding singular vectors yield the truncated left singular matrix. truncating singular value matrices and truncated right singular matrix ,in, Based on the above truncated components, an outer low-rank projection matrix is constructed. and :
[0025]
[0026]
[0027] in, , , Represents the rank of the outer low-rank projection matrix;
[0028] 2-4) Project the outer low-rank matrix and Freeze it so that it does not participate in gradient updates during subsequent fine-tuning.
[0029] Furthermore, step 3) specifically includes:
[0030] 3-1) Define the inner trainable adaptation matrix Construct it to contain A diagonal array of diagonal pieces Each block ,and ;
[0031] 3-2) Initialize the inner diagonal block matrix M as an all-zero matrix to ensure that the model output in the initial stage of fine-tuning is consistent with the original pre-trained model;
[0032] 3-3) After completing the outer low-rank projection matrix The construction and freezing of the inner diagonal block matrix were completed. After construction, for the input features of the target linear layer Original pre-trained weight matrix The output is In this method, only the frozen outer low-rank projection matrix is used. With trainable inner diagonal block matrix The adaptation increment is generated, and its increment calculation formula is as follows: Therefore, the complete forward propagation formula after introducing the nested diagonal block low-rank adaptor structure is: ;in, This represents the original pre-trained weight matrix. and This represents the outer low-rank projection matrix of the frozen layer. This represents the trainable inner diagonal block adaptation matrix. This represents the input features of the target linear layer.
[0033] Furthermore, step 4) specifically includes:
[0034] 4-1) Set the inner layer trainable diagonal block adaptation matrix block size In contrast to the inner rank of traditional nested low-rank structures The relationship is: This enables the inner layer to train a diagonal block adaptation matrix. The number of parameters is equal to that of the traditional double low-rank product structure;
[0035] 4-2) In the target weight matrix of the pre-trained large language model Introducing the outer low-rank projection matrix and the inner trainable diagonal block adaptation matrix The constructed adaptation structure yields a large language model to be fine-tuned; the target domain dataset is used to efficiently fine-tune the parameters of the large language model to be fine-tuned, and the original target weight matrix is frozen during backpropagation. and the outer low-rank projection matrix Only update the inner trainable diagonal block adaptation matrix. The parameters are set, and the optimization objective is to minimize the cross-entropy loss.
[0036] 4-3) During gradient calculation, the inner trainable diagonal block adaptation matrix is used. The sparsity applies only to each diagonal block. The gradient is calculated and the parameters are updated independently, thereby eliminating the gradient coupling and gradient vanishing problems caused by the traditional low-rank matrix product structure.
[0037] Furthermore, step 5) specifically includes:
[0038] 5-1) After fine-tuning, freeze the outer matrix. With the trained inner matrix Perform explicit merging and calculate the equivalent incremental weights. ;
[0039] 5-2) Add to the original pre-trained weights The above yields a domain-adaptive model with no inference delay.
[0040] 5-3) Input the user's domain query text into the adaptation model to generate a professional response that conforms to the domain specifications.
[0041] An application of a large language model fine-tuning method based on nested diagonal block low-rank adaptation for mental health question answering.
[0042] Furthermore, this includes the following steps:
[0043] Step 1) Construct a mental health domain instruction fine-tuning dataset containing traceable single-turn instructions, multi-turn support dialogues, and safety preference data;
[0044] Step 2) Fine-tune the large model using the aforementioned mental health domain instruction fine-tuning dataset and the aforementioned fine-tuning method to obtain a dedicated mental health question-and-answer model.
[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0046] 1. Based on the convex optimization characteristics of the diagonal block structure, the non-convex optimization obstacle is completely eliminated, significantly improving training stability and expressiveness;
[0047] This invention replaces the inner double low-rank product in the traditional nested low-rank structure with a diagonal block matrix, avoiding parameter coupling and non-convex optimization problems caused by matrix multiplication. The optimization process approximates strict convex optimization, and the gradient can directly reach each independent diagonal block, completely solving the problems of traditional LoRA and its variants being prone to getting trapped in local optima, gradient vanishing, and training instability. Furthermore, the rank of the diagonal block matrix can reach the rank of the outer layer. It broke through the original inner order Overcoming the expressive limitations, it can more precisely characterize complex feature interactions and semantic representations with the same number of parameters.
[0048] 2. Based on an activation-aware subspace locking mechanism, the core knowledge of pre-training is fully preserved, effectively avoiding catastrophic forgetting;
[0049] Unlike traditional low-rank adapters with random initialization, this invention employs activation-perceptual singular value decomposition to extract the most critical semantic subspace from the pre-trained weights and strictly locks it by freezing the outer matrix. Combined with a zero-initialization strategy for the inner matrix, the model is completely equivalent to the original pre-trained model in the initial stage of fine-tuning, and subsequent controlled perturbations are only performed within the stable core subspace. This mechanism fundamentally avoids the loss of general knowledge from pre-training during fine-tuning, significantly reducing the occurrence of hallucinations and common-sense errors, making it particularly suitable for highly sensitive fields such as mental health where the accuracy of responses is critical.
[0050] 3. Hardware-friendly sparse computing architecture, maintaining extremely high parameter efficiency and compatible with large-scale models and datasets;
[0051] The diagonal block structure of this invention inherently possesses sparsity, allowing gradient calculation and parameter updates to be executed independently and in parallel across each diagonal block. Its computational complexity and memory footprint are completely comparable to standard LoRA, enabling efficient fine-tuning of large models with billions of parameters on consumer-grade hardware. Furthermore, this architecture seamlessly integrates with existing acceleration techniques such as mixed-precision training and quantization fine-tuning. Combined with its excellent convergence characteristics, it can significantly shorten the training cycle on large-scale datasets, demonstrating strong engineering practicality and scalability.
[0052] 4. Establish a traceable and scalable data construction process for the target domain;
[0053] This invention enables the recording and reproducibility of the source, purpose, and quality control process of newly added data through source identification, scenario classification, single-round and multi-round data expansion, security preference construction, multi-model cross-validation, and source isolation partitioning. Furthermore, the contribution of each expanded subset can be individually evaluated through D0-D3 group ablation experiments. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0055] Figure 1 This is the overall flowchart of the present invention.
[0056] Figure 2 This is a flowchart illustrating the construction and initialization process of the NDBoRA structure in this invention.
[0057] Figure 3 This is a flowchart of the model training and inference process of this invention. Detailed Implementation
[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0059] Example 1
[0060] A fine-tuning method for large language models based on Nested DiaBlo Low-Rank Adaptation (NDBoRA) is presented. The overall process is shown in Figure 1. The sub-module processes (structure construction and initialization, training and inference) are shown in Figures 2 and 3, respectively. The following is a detailed explanation of each flowchart.
[0061] The specific implementation steps are as follows, and the overall process is shown in Figure 1.
[0062] Step 1) Construction of the target domain traceable instruction fine-tuning dataset
[0063] First, determine the target domain and specific task type that the large language model needs to adapt to, and select professional literature, industry guidelines, technical specifications, question-and-answer corpora or business cases with legal usage rights as data sources; clean, deduplicate, standardize the format, detect privacy information and filter risk rules on the original text, and divide it into traceable knowledge units with source document identifiers, chapter identifiers and knowledge fragment identifiers.
[0064] Then, a data classification system is established according to the theme, application scenario, user characteristics, risk level and support strategy of the target domain. The first major language model is used to generate single-turn instruction pairs, multi-turn dialogues and security preference data with traceable sources based on the traceable knowledge units. The security preference data includes inappropriate candidate responses and security rewritten responses generated for the same user's question.
[0065] Furthermore, at least two second-largest language models, different from the first, are used to cross-validate the consistency of the source, relevance of the question, support, security, and completeness of the generated samples. This is combined with near-duplicate detection, privacy information detection, risk rule matching, and manual review to eliminate unqualified samples. Manual review only checks language completeness, question-answer relevance, source identification, privacy breaches, and obviously dangerous suggestions; the review results are not used as clinical diagnoses or expert conclusions. Training, validation, and test sets are divided according to source document identifiers or knowledge fragment identifiers to ensure that fragments from the same source are not repeated across sets, ultimately forming a high-quality instruction fine-tuning dataset for the target domain.
[0066] Step 2) Loading the pre-trained model, extracting the target weight matrix, and freezing the outer low-rank projection matrix
[0067] To extract the core semantic subspace of the pre-trained model and avoid destroying pre-trained knowledge during fine-tuning, activation-aware singular value decomposition is performed on the target weight matrix of the pre-trained large language model. The specific steps are as follows:
[0068] 2-1) Load the pre-trained large language model and extract the target weight matrix from the model that needs to incorporate the adaptation structure. The target weight matrix This includes the query projection matrix, key projection matrix, value projection matrix, and output projection matrix in the attention layer, as well as the up projection matrix and down projection matrix in the multilayer perceptron layer. Define the target weight matrix. , where m represents the output feature dimension of the target linear layer, and n represents the input feature dimension of the target linear layer.
[0069] 2-2) Collect the input activation matrix of the pre-trained large language model on a small amount of general calibration corpus. ,in, Where b represents the number of samples or batch size, and n represents the input feature dimension of the target linear layer. Based on the input activation matrix... Calculate the input activation standard deviation diagonal matrix : ;
[0070] in, This represents extracting the diagonal elements of a matrix and constructing a diagonal matrix. .when When there are elements with zero or close to zero in the matrix, a small constant can be added. This is to ensure the stability of subsequent inversion operations.
[0071] 2-3) Use the input standard deviation diagonal matrix obtained in step 2-2 to activate the matrix. For the target weight matrix Activation weighting is performed to obtain the activation-aware weight matrix. :
[0072]
[0073] in, This activation weighting operation rescales the target weight matrix according to the importance of the input activations, so that subsequent decomposition processes focus more on the feature directions that have a greater impact on the model output.
[0074] 2-4) The activation-sensory weight matrix obtained in step 2-3 Perform singular value decomposition:
[0075]
[0076] in, Describes a left singular matrix. Represents a singular value matrix. This represents the transpose of the right singular matrix V. Further extraction of the first r largest singular values and their corresponding singular vectors yields the truncated left singular matrix. truncating singular value matrices and truncated right singular matrix ,in, , , , where r represents the rank of the outer low-rank projection matrix.
[0077] 2-5) Construct outer low-rank projection matrices A and B based on the above truncated components:
[0078]
[0079]
[0080] in, The outer low-rank projection matrices A and B constructed in the above manner can preserve the important semantic subspace related to input activation in the pre-trained weights.
[0081] 2-6) Freeze the outer low-rank projection matrices A and B so that they do not participate in gradient calculation and parameter updates during subsequent fine-tuning. This operation locks the relatively critical low-rank semantic subspace in the pre-trained model, which helps to maintain the model's original basic language understanding ability and reduces the risk of catastrophic forgetting during fine-tuning.
[0082] Step 3) Construction and zero initialization of the inner trainable diagonal block adaptation matrix
[0083] A diagonal block structure is introduced between the frozen outer low-rank projection matrices A and B to construct an inner trainable diagonal block adaptation matrix M, which replaces the inner low-rank product structure in traditional nested LoRA. The specific steps are as follows:
[0084] 3-1) Define the inner trainable diagonal block adaptation matrix Where r represents the rank of the outer low-rank projection matrix. M is constructed as a block diagonal matrix containing K diagonal blocks:
[0085]
[0086] Each diagonal block For a trainable square matrix, d represents the block size of a single diagonal block, and satisfies:
[0087]
[0088] 3-2) To ensure that the inner trainable diagonal block adaptation matrix M has the same number of parameters as the inner low-rank product structure in a traditional nested LoRA, the diagonal block size d is set to be the same as the inner rank of the traditional nested LoRA. The following conditions must be met:
[0089]
[0090] Among them, the inner low-rank product structure of the traditional nested LoRA consists of two matrices. and Its composition, its parameter quantity is The number of parameters in the inner trainable diagonal block fitting matrix M is: .because ,therefore .when At that time, the number of parameters in the inner trainable diagonal block adaptation matrix M is: The number of parameters is equal to that of the inner low-rank product structure of traditional nested LoRA.
[0091] For example, when the outer rank r = 64 and the inner rank r' = 16 of the traditional nested LoRA, the number of parameters in the traditional inner low-rank product structure is:
[0092]
[0093] This embodiment takes Then the number of parameters in the inner trainable diagonal block fitting matrix M is:
[0094]
[0095] Both have the same number of parameters.
[0096] 3-3) Initialize the inner trainable diagonal block adaptation matrix M as an all-zero matrix, such that the fine-tuning start time satisfies:
[0097]
[0098] At this point, the model output remains consistent with the original pre-trained model output, thus avoiding the impact of output perturbations introduced by random initialization on model performance.
[0099] Through the above design, this embodiment, based on freezing the outer low-rank projection matrices A and B, trains only the inner trainable diagonal block adaptation matrix M. This structure avoids the parameter coupling and non-convex optimization difficulties caused by traditional inner low-rank product structures. Simultaneously, the effective rank of the diagonal block structure can reach the outer rank r, breaking through the limitations of traditional inner rank structures. The expression limitations, under the same parameter conditions, can provide stronger feature expression capabilities.
[0100] Step 4) Efficient parameter fine-tuning and gradient decoupling update
[0101] Using the high-quality instruction fine-tuning dataset D of the target domain constructed in step 1, the parameters of the pre-trained large language model after incorporating the adaptation structure are efficiently fine-tuned. During training, the original target weight matrix W and the outer low-rank projection matrices A and B are kept frozen, and only the parameters of the inner trainable diagonal block adaptation matrix M are updated. The specific steps are as follows:
[0102] 4-1) Model forward propagation: For input features The output of the original target weight matrix W is The weight increment output generated by the adaptation structure is:
[0103]
[0104] Therefore, the output of the target linear layer after introducing the adaptation structure is:
[0105]
[0106] in, , , In the specific calculation process, first calculate:
[0107]
[0108] in, Since M is a block diagonal matrix, Z is divided into K sub-vectors:
[0109]
[0110] in, Then make each subvector Each with the corresponding diagonal piece Multiplication:
[0111]
[0112] Finally, by concatenating the various sub-outputs, we obtain... The final incremental output is obtained by mapping through the frozen outer low-rank projection matrix B:
[0113]
[0114] 4-2) Loss function calculation: The standard cross-entropy loss function of the autoregressive language model is used as the optimization objective, so that the model learns the professional expression, task response mode and safety response standard of the target domain based on the high-quality instruction fine-tuning dataset D of the target domain.
[0115] 4-3) Backpropagation and Gradient Update: During backpropagation, the gradients of the original target weight matrix W and the outer low-rank projection matrices A and B are truncated. Only the gradient of the inner trainable diagonal block fitting matrix M is calculated and updated. Since M adopts a block diagonal structure, each diagonal block... The gradient depends only on the corresponding local subspace features and their output gradients, remaining decoupled from each other. For a single sample, its gradient can be expressed as:
[0116]
[0117] in, This represents the i-th sub-vector obtained after the input features are projected through the outer low-rank projection matrix A. Indicates the corresponding sub-output The gradient is calculated; during batch training, the gradients across the sample dimensions are summed or averaged.
[0118] This gradient decoupling characteristic reduces the gradient coupling problem caused by parameter interdependence in the traditional low-rank matrix product optimization process, which is beneficial to improving the stability and convergence efficiency of the model fine-tuning process.
[0119] Step 5) Matrix merging and inference-free deployment
[0120] After fine-tuning, the trained inner layer trainable diagonal block adaptation matrix M is explicitly merged with the frozen outer layer low-rank projection matrices A and B to obtain a domain adaptation model without additional inference latency. The specific steps are as follows:
[0121] 5-1) Calculate the equivalent incremental weight matrix:
[0122]
[0123] in, .
[0124] 5-2) Equivalent incremental weight matrix Inject the original target weight matrix W to obtain the domain-adapted target weight matrix:
[0125]
[0126] 5-3) Save the merged complete model weights for subsequent inference deployment.
[0127] Since the merged model structure remains consistent with the original pre-trained model structure, no additional adaptation branches are needed during the inference phase, resulting in no additional computational overhead or inference latency. The original model's inference acceleration framework can be used directly for deployment.
[0128] Example 2
[0129] The application of the NDBoRA fine-tuning method in the field of mental health Q&A follows these steps:
[0130] Step 1) Construction of a high-quality mental health domain instruction fine-tuning dataset
[0131] 1-1) Preparation of Basic Data and Traceable Knowledge Sources: Within the scope permitted by the licensing conditions, publicly available data from the Emotional Large Language Model (EmoLLM) was selected as the basic data pool. After field normalization, semantic deduplication, privacy information detection, and risk rule filtering, approximately 87,000 basic training samples were retained. Simultaneously, six psychology books and guidelines with legal access rights were selected, including *Diagnostic and Statistical Manual of Mental Disorders (DSM-5)*, *Cognitive Behavioral Therapy: Foundations and Applications*, *National Vocational Qualification Training Course for Psychological Counselors*, and *Crisis Intervention Strategies*. Approximately 400,000 words of knowledge text covering five categories—emotional distress, cognitive behavior, developmental psychology, interpersonal relationships, and crisis support—were extracted and formed into traceable knowledge units using a three-level numbering system: "Source Document—Chapter—Knowledge Fragment". This process did not use actual consultation records containing identifiable personal information.
[0132] 1-2) Construction of Original Extended Subset: The Qwen2.5-72B-Instruct model with 72 billion parameters was adopted as the primary generation model. New data was generated based on five categories of tags: topic, scenario, user characteristics, risk level, and support strategy, using traceable knowledge units. In an optimal configuration, approximately 2,000 traceable single-turn instruction pairs, approximately 500 sets of multi-turn support dialogues including stages such as listening, clarification, emotion confirmation, cause exploration, suggestions, and summaries, and approximately 500 sets of safety preference data consisting of "inappropriate candidate responses - safe rewritten responses" were generated. These were merged with approximately 87,000 basic samples to form approximately 90,000 training records.
[0133] 1-3) Cross-validation, manual review, and set partitioning: At least two large language models different from the first generative model are used to check source consistency, question relevance, emotional support, safety guidance, and response completeness. Samples with scores below the threshold or inconsistent validation conclusions are removed, and near-duplicate detection, personal information detection, overdiagnosis, and dangerous suggestion rule matching are performed. Manual review only checks language completeness, question-answer relevance, source identification, privacy breaches, and obviously dangerous suggestions, and the review results are not used as clinical diagnoses or expert conclusions. Training, validation, and test sets are partitioned according to source literature or knowledge fragment identifiers to ensure that the same source fragment is not repeated across sets. Approximately 300 challenge test data entries that are not used in training are constructed separately to cover scenarios such as incomplete information, multiple rounds of risk escalation, self-harm and suicide suggestions, over-reliance, and refusal to answer referrals.
[0134] Step 2) Fine-tuning the dedicated model for mental health Q&A
[0135] The NDBoRA fine-tuning method described in Example 1 was used to fine-tune the Qwen3-8B model using approximately 90,000 mental health training records. The hyperparameters were set as follows: outer rank r=64, number of blocks K=2, block dimension d=32, learning rate 2e-4, batch size 16, maximum sequence length 2048, and training for 3 epochs. An early stopping strategy was employed during training, terminating training prematurely when the validation set loss no longer decreased for two consecutive epochs.
[0136] Step 3) Data contribution assessment and security deployment
[0137] To verify the contributions of the fine-tuned structure and the newly added data, four training configurations were set up: D0 (approximately 87,000 basic samples), D1 (D0 plus traceable single-turn instructions), D2 (D1 plus multi-turn support dialogue), and D3 (D2 plus safety preference data). Standard LoRA and NDBoRA were used for training under the same base model, hyperparameters, and random seeds. Approximately 300 independent challenge test data were used to evaluate source consistency, response relevance, emotional support, and compliance with safety rules. Source consistency was verified based on related knowledge fragments, while the other dimensions were independently scored by at least two evaluation models and manually verified using random sampling. These scores are not interpreted as clinical efficacy. The trained model is deployed only as a tool for mental health information support and emotional support, and is not used for psychological diagnosis or as a substitute for professional treatment. For high-risk inputs, application-layer rules trigger rejection of dangerous guidance and suggestions to contact professionals or emergency assistance channels.
[0138] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.
Claims
1. A method for fine-tuning a large language model based on nested diagonal block low-rank adaptation, characterized in that, Includes the following steps: Step 1) Construct a high-quality instruction fine-tuning dataset for the target domain through source tracing, hierarchical scenario expansion, multi-model cross-validation, and source isolation partitioning; Step 2) Perform activation-perceptual singular value decomposition on the target weight matrix of the large language model, extract the outer low-rank projection matrix and freeze it to preserve the core semantic subspace of the pre-trained model. Step 3) Introduce a diagonal block structure between the outer low-rank projection matrices to construct the inner trainable diagonal block adaptation matrix M, and perform zero initialization on the inner trainable diagonal block adaptation matrix M; Step 4) Fine-tune the dataset using the high-quality instructions, combining the frozen outer low-rank projection matrix with the inner trainable diagonal block adaptation matrix. Efficiently fine-tuning parameters of large language models; Step 5) Merge the trained inner layer trainable diagonal block adaptation matrix M with the frozen outer layer low-rank projection matrix to obtain the weight increment and inject it into the original model to obtain the domain adaptation model.
2. The method for fine-tuning a large language model based on nested diagonal block low-rank adaptation according to claim 1, characterized in that, Step 1) specifically includes: 1-1) Select professional literature and industry guidelines in the target field with legal usage rights as data sources, and segment the original text into traceable knowledge units with source document identifiers, chapter identifiers and knowledge fragment identifiers; 1-2) Establish a data classification system according to the theme, application scenario, user characteristics, risk level and support strategy of the target domain, and use the first major language model to generate single-turn instruction pairs, multi-turn dialogues and security preference data with traceable sources based on the traceable knowledge units. The security preference data includes inappropriate candidate responses and security rewritten responses generated for the same user's question. 1-3) Use at least two second-largest language models, different from the first-largest language model, to cross-validate the consistency of the source, relevance of the question, support, security and completeness of the generated samples. Combine deduplication, privacy information detection, risk rule matching and manual formal review to remove unqualified samples. Divide the training set, validation set and test set according to the source document identifier or knowledge fragment identifier to ensure that the same source fragment is not repeated across sets.
3. The method for fine-tuning a large language model based on nested diagonal block low-rank adaptation according to claim 2, characterized in that, Step 2) specifically includes: 2-1) Obtain the input activation matrix of the pre-trained model on a general corpus. , where the target weight matrix Input activation matrix , Indicates the number of samples collected (or batch size). This represents the input feature dimension of the target linear layer; based on the input activation matrix... Calculate the standard deviation diagonal matrix of the input activations. : , among which, diag( This indicates extracting the diagonal elements of a matrix and constructing a diagonal matrix; 2-2) Use the input activation standard deviation diagonal matrix obtained in step 2-1 For the target weight matrix Activation weighting is performed to obtain the activation-aware weight matrix. ; 2-3) The activation-sensory weight matrix obtained in step 2-2 Perform singular value decomposition ,in, It is a left singular matrix. It is a singular value matrix. A right singular matrix Transpose; before extraction The maximum singular values and their corresponding singular vectors yield the truncated left singular matrix. truncating singular value matrices and truncated right singular matrix ,in, Construct the outer low-rank projection matrix based on the above truncated components. and : ; ; in, , , Represents the rank of the outer low-rank projection matrix; 2-4) Project the outer low-rank matrix and Freeze it so that it does not participate in gradient updates during subsequent fine-tuning.
4. The method for fine-tuning a large language model based on nested diagonal block low-rank adaptation according to claim 3, characterized in that, Step 3) specifically includes: 3-1) Define the inner trainable adaptation matrix Construct it to contain A diagonal array of diagonal pieces Each block ,and ; 3-2) Initialize the inner diagonal block matrix M as an all-zero matrix to ensure that the model output in the initial stage of fine-tuning is consistent with the original pre-trained model; 3-3) Based on the frozen outer low-rank projection matrix and And the inner trainable diagonal block adaptation matrix Construct a nested diagonal block low-rank adaptation matrix; wherein, the equivalent weight increment of the nested diagonal block low-rank adaptation matrix structure is .
5. The method for fine-tuning a large language model based on nested diagonal block low-rank adaptation according to claim 4, characterized in that, Step 4) specifically includes: 4-1) Set the inner layer trainable diagonal block adaptation matrix block size In contrast to the inner rank of traditional nested low-rank structures The relationship is: This enables the inner layer to train a diagonal block adaptation matrix. The number of parameters is equal to that of the traditional double low-rank product structure; 4-2) In the target weight matrix of the pre-trained large language model Introducing the outer low-rank projection matrix and the inner trainable diagonal block adaptation matrix The constructed adaptation structure yields a large language model to be fine-tuned; the target domain dataset is used to efficiently fine-tune the parameters of the large language model to be fine-tuned, and the original target weight matrix is frozen during backpropagation. and the outer low-rank projection matrix Only update the inner trainable diagonal block adaptation matrix. The parameters are set, and the optimization objective is to minimize the cross-entropy loss. 4-3) During gradient calculation, the inner trainable diagonal block adaptation matrix is used. The sparsity applies only to each diagonal block. The gradient is calculated and the parameters are updated independently, thereby eliminating the gradient coupling and gradient vanishing problems caused by the traditional low-rank matrix product structure.
6. The method for fine-tuning a large language model based on nested diagonal block low-rank adaptation according to claim 5, characterized in that, Step 5) specifically includes: 5-1) After fine-tuning, freeze the outer matrix. With the trained inner matrix Perform explicit merging and calculate the equivalent incremental weights. ; 5-2) Add to the original pre-trained weights The above yields a domain-adaptive model with no inference delay. 5-3) Input the user's domain query text into the adaptation model to generate a professional response that conforms to the domain specifications.
7. An application of the fine-tuning method as described in any one of claims 1-6, characterized in that, Used for mental health Q&A.
8. The application according to claim 7, characterized in that, Includes the following steps: Step 1) Construct a mental health domain instruction fine-tuning dataset containing traceable single-turn instructions, multi-turn support dialogues, and safety preference data; Step 2) Fine-tune the large model using the aforementioned mental health domain instruction fine-tuning dataset and the aforementioned fine-tuning method to obtain a dedicated mental health question-and-answer model.
