Incremental tuning and knowledge injection method for few samples / rare diseases

Abstract

The invention discloses an incremental tuning and knowledge injection method and system for few samples / rare diseases, and relates to the field of continuous learning of a large language model (LLM) and medical artificial intelligence. In order to solve the technical problem that catastrophic forgetting is easily generated when an existing model is subjected to incremental tuning in small-sample and high-value fields (such as rare diseases), the invention provides an optimization strategy based on PEFT (Parameter Efficient Fine Transfer) and knowledge integration loss. The method comprises the following steps: initializing a PEFT module and adding the PEFT module to a pre-training LLM; taking few-sample data labeled by experts as new task data; constructing a dual loss function containing new task loss and knowledge distillation loss (measuring the difference between the output of the new model and the output of the basic model); only by optimizing the parameters of the PEFT module, the basic universality is protected from being damaged to the maximum extent while it is guaranteed that the model quickly masters rare disease knowledge. According to the invention, the efficiency and the stability of the medical LLM in a continuous updating scene are remarkably improved.

Description

Technical fields:

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to an incremental optimization and knowledge injection method for few-sample / rare diseases. Background technology:

[0002] In recent years, large language models based on the Transformer architecture have demonstrated powerful capabilities in general knowledge question answering, text generation, and other fields. However, in specific vertical domains, especially in the healthcare field, LLMs face serious challenges in knowledge updating and adaptability.

[0003] First, medical knowledge demands high levels of specialization, rapid iteration, and accuracy. Many rare diseases or emerging treatments exist only in a limited amount of expert literature or experience, making it difficult to cover them with large-scale pre-trained data; therefore, incremental tuning is necessary.

[0004] Secondly, existing incremental tuning methods, such as traditional full fine-tuning, suffer from a severe catastrophic forgetting (CF) problem. When training the model with a small amount of domain-specific rare disease data, the new gradient signals can overwrite and destroy the key parameters learned by the model on large-scale general data. This causes the model to lose most of its original general capabilities (such as common sense question answering and general language understanding) while completing the new task (answering rare disease questions). This is unacceptable in high-security medical scenarios.

[0005] To address the inefficiency of full-parameter fine-tuning, Parametric Efficient Fine-Tuning (PEFT) techniques, such as LoRA or Adapter, have been proposed. These methods alleviate catastrophic forgetting to some extent by introducing a small number of trainable parameters and freezing most of the original parameters. However, simply freezing the parameters does not completely solve the problem. The optimization direction of the new PEFT parameters may still conflict with the function of the original parameters, especially when the amount of data is extremely small. In such cases, the model is prone to overfitting to the new task in a very short time and exhibiting "local catastrophic forgetting"—that is, forgetting general knowledge related to the new knowledge.

[0006] Therefore, there is an urgent need for a method that can quickly and efficiently inject a small amount of expert-annotated knowledge into a large model, while continuously protecting and consolidating the model's basic general capabilities during training through refined loss function design, thereby ensuring the stability and reliability of the model in medical applications. Summary of the Invention:

[0007] The main objective of this invention is to provide a method, system, and medium for incremental optimization and knowledge injection for few samples / rare diseases, aiming to solve the core technical problems faced by existing large language models when incrementally updating extremely sparse, high-value professional data, such as catastrophic forgetting, high risk of overfitting, large consumption of training resources, and degradation of general capabilities.

[0008] (I) Technical Issues

[0009] After in-depth research into the continuous learning mechanism of large language models, we found that existing technical solutions face the following insurmountable technical bottlenecks when dealing with the task of "injecting high-value knowledge from a small number of samples":

[0010] First, the underlying mechanisms of catastrophic forgetting have not been effectively suppressed. In the weight space of neural networks, general knowledge is typically distributed across a broad parametric manifold. When using traditional fine-tuning methods for specific vertical domains (such as rare disease data),… Task During training, due to D Task Data distribution P Task (χ) and the distribution of pre-training data P Pretrain (χ²) exhibits a significant covariate shift, causing a sharp conflict between the gradient direction of the loss function and the gradient direction required to preserve existing knowledge. To minimize the loss on new tasks, the optimizer may forcefully update the weights, leading to a catastrophic drop in the model's performance on its original general tasks. This kind of forgetting is fatal in the medical field, as it can cause a model to learn to diagnose a rare disease while losing the ability to diagnose common colds or understand basic medical terminology.

[0011] Second, the risk of overfitting under small sample conditions is extremely high. Expert data on rare diseases or specific fields are often extremely scarce (e.g., only a few dozen to a few hundred data points). Fine-tuning a model with hundreds of billions of parameters on such a small dataset is tantamount to using a sledgehammer to crack a nut. The model is prone to remembering the noise of the samples rather than the features, resulting in extremely poor generalization ability on new tasks, exhibiting a "rote memorization" phenomenon, and being unable to handle similar cases it has never seen before.

[0012] Third, the unsustainability of computing resources and storage costs. With the exponential growth of the number of LLM parameters (from 7B to 70B or even higher), the video memory (VRAM) and computing power (FLOPs) required for full parameter fine-tuning increase linearly. Each knowledge update requires saving a complete copy of the model (hundreds of gigabytes), which is unacceptable in terms of storage costs and version management complexity for real-world applications that require frequent knowledge base updates (such as weekly updates to medical guidelines).

[0013] Fourth, existing PEFT methods lack explicit constraints at the knowledge level. While parameter-efficient fine-tuning methods such as LoRA and Adapter reduce computation by freezing base parameters, they only achieve "parameter-level isolation." Research shows that even small PEFT parameter updates, without explicit regularization of the output semantic space, can still erode the capabilities of the base model by altering the behavior of the feature extractor. Existing PEFT methods primarily focus on "how to learn efficiently" while neglecting "how to learn safely."

[0014] (II) Technical Solution

[0015] To address the aforementioned issues, this invention proposes an incremental tuning architecture characterized by "dual constraints and structural isolation." This approach combines efficient parameter fine-tuning techniques with output space regularization based on knowledge distillation.

[0016] Specifically, this invention provides an incremental optimization and knowledge injection method for few-sample / rare diseases, comprising the following detailed steps:

[0017] Step S1: Construct a structurally isolated model training environment.

[0018] A large language model pre-trained on a large-scale general-purpose dataset is used as the base model (TeacherModel). To preserve general knowledge, all weight parameters of the base model (including the attention layer, feedforward neural network layer, and normalization layer) are completely frozen. Based on this, at key positions in the Transformer layer of the base model (preferably the query projection layer W with a self-attention mechanism)... Q Sum projection layer W V The system employs a parallel injection of a trainable parameter-efficient fine-tuning module (PEFTModule, or Student Adapter). This PEFT module consists of low-rank matrices A and B, with a parameter count far smaller than that of the base model (typically less than 0.1%), achieving physical isolation of incremental knowledge.

[0019] Step S2: Construct an adversarial knowledge anchor dataset.

[0020] To prevent forgetting, not only is new task data needed, but also a knowledge anchor dataset constructed by sampling from a general data distribution. This invention employs a dynamic adversarial sampling strategy: during training, the model's worst-performing general knowledge category on the validation set is monitored in real time, and the sampling weight of that category in the anchor dataset is increased accordingly. This ensures that the anchor data always covers the knowledge blind spots most easily forgotten by the model, rather than being a simple random sampling.

[0021] Step S3: Construct a dual loss function based on knowledge distillation to optimize the objective.

[0022] This invention abandons the traditional single cross-entropy loss and innovatively designs a total loss function $L_{Total}$ that includes two components: "acquiring new knowledge" and "consolidating old knowledge".

[0023] Component 1: New Task Adaptation Loss (L) Task For rare disease expert-annotated data, the cross-entropy loss between the model's predicted output and the true labels is calculated. This component drives the parameters of the PEFT module to move towards the optimal solution for the new knowledge.

[0024] Component Two: Knowledge Integration Consistency Loss (L) Consolidation This is a regularization term based on Kullback-Leibler Divergence. It calculates the difference in output probability distributions (Logits) between the "base model (frozen)" and the "currently trained model (base + PEFT)" on general anchor data. This loss enforces that even with the addition of the PEFT module, the model's general domain thinking patterns (probability distributions) must remain highly consistent with the original base model. This is equivalent to imposing a strong "soft constraint" on the semantic output space.

[0025] Step S4: Perform gradient-isolation-based iterative optimization.

[0026] The data batches, which combine new task data and anchor data, are input into the model. During backpropagation, the total loss function L is calculated. Total The key is that the system automatically masks all gradient updates pointing to the base model parameters, applying only the gradients to matrices A and B of the PEFT module. In this way, new knowledge is precisely "imprinted" into the PEFT module, while the generality of the base model remains completely intact.

[0027] Step S5: Dynamic weight adjustment and convergence control.

[0028] Introducing an adaptive weighting factor γ to balance L Task and L Consolidation In the early stages of training, γ is appropriately reduced to allow the model to quickly explore new knowledge domains; in the later stages of training, γ is gradually increased to strengthen the protection of general capabilities and prevent overfitting.

[0029] 3. Beneficial effects of the present invention

[0030] Through the aforementioned unique technical solution, this invention brings the following significant beneficial effects:

[0031] It fundamentally solves the problem of catastrophic forgetting: by introducing LConsolidation This loss term, based on knowledge distillation, mathematically guarantees that the new model's behavioral distribution in general domains closely approximates the original model. Experiments show that after injecting rare disease knowledge, the model's accuracy on general commonsense question-and-answer sets (such as MMLU) decreases by less than 0.5%, achieving truly "lossless" incremental learning.

[0032] Significantly improved accuracy for few-sample learning: By leveraging the low-rank property of PEFT, the number of trainable parameters was greatly reduced, thereby lowering model complexity. With only a small number of samples (e.g., 50 cases), overfitting was effectively avoided, resulting in a rare disease diagnosis accuracy improvement of over 15% compared to full-parameter fine-tuning.

[0033] This significantly reduces training and deployment costs: requiring only about 0.05% parameter updates, it reduces GPU memory usage for training by more than 60% and increases training speed by 4 times. Furthermore, incremental knowledge is represented as a few megabytes (MB) of LoRA weight files, making them extremely easy to store, distribute, and deploy on edge devices.

[0034] It achieves modular and pluggable management of knowledge: different professional knowledge (such as "cardiology module" and "pediatrics module") can be trained as independent PEFT weights. In practical applications, specific knowledge modules can be dynamically loaded or unloaded according to user needs, or multiple modules can be weighted and merged, giving LLM extremely high flexibility and scalability.

[0035] Enhanced safety in medical scenarios: By strictly preserving the general reasoning and safety alignment capabilities of the base model, the risk of harmful content or logical errors arising from fine-tuning is prevented, meeting the stringent safety requirements of medical AI. Attached image description:

[0036] Figure 1 This is a flowchart of a method for anti-hallucination training and punishment mechanism for medical texts provided in an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram illustrating the principle of loss function calculation based on positive and negative sample comparison provided in an embodiment of the present invention.

[0038] Figure 3 This is a structural block diagram of an anti-hallucination training and punishment system for medical texts provided in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific hardware environment, data processing flow, and algorithm steps. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0040] Part 1: System Architecture and Hardware Environment Configuration

[0041] This embodiment is built on a high-performance distributed deep learning training cluster.

[0042] 1. Hardware Environment Configuration

[0043] Compute node: A high-performance server equipped with eight NVIDIA A100 Tensor Core GPUs (80GB VRAM per card). The GPUs are interconnected via NVLink and NVSwitch to achieve a bidirectional high bandwidth of 600GB / s, supporting fast communication of large model parameters.

[0044] Storage system: Configured with 4TB NVMe SSD high-speed cache disks to store pre-trained large model weights and datasets to eliminate I / O bottlenecks.

[0045] Memory configuration: 2TB DDR4 ECC memory, ensuring the ability to load massive knowledge graphs and index data.

[0046] 2. Software Environment Configuration

[0047] Operating system: Ubuntu 22.04LTS.

[0048] Deep learning framework: PyTorch 2.1.0, with CUDA 12.1 and cuDNN 8.9 acceleration library.

[0049] Core dependency library: transformers (v4.36.0): used to load mainstream base models such as Llama 2 and Baichuan.

[0050] peft (v0.7.1): Specifically designed for implementing efficient parameter fine-tuning algorithms such as LoRA and P-Tuning.

[0051] accelerate(v0.25.0): Used to implement distributed training and mixed precision training (FP16 / BF16) across multiple GPUs.

[0052] deepspeed (v0.12.4): Used to further optimize video memory usage and enable ZeRO-Stage 2 optimization.

[0053] Part Two: Data Preparation and Enhanced Preprocessing

[0054] Data quality is crucial for successful few-shot learning. This embodiment designs a sophisticated data processing pipeline for the "rare disease diagnosis and treatment" scenario.

[0055] 1. Task Dataset (D) Task Construction and enhancement of )

[0056] Original data source: 500 Q&A pairs on the diagnosis and treatment of Gaucher disease, reviewed by experts from top-tier hospitals. Each data point includes "patient's chief complaint / examination results" as input and "diagnosis and treatment recommendations" as output.

[0057] Data Augmentation: To overcome overfitting caused by scarce samples, this embodiment adopts the following augmentation strategy to expand the data to 2000 entries: Synonym replacement: Using a medical thesaurus (such as UMLS), "hepatosplenomegaly" is replaced with "enlarged liver and spleen volume", and "thrombocytopenia" is replaced with "decreased platelet count".

[0058] Back Translation: Translate Chinese medical records into English and then back into Chinese, enhancing the robustness of the model by introducing the diversity of linguistic expressions.

[0059] Prompt engineering enhancement: Build multiple different Prompt templates for each sample (e.g., "Please diagnose the following case...", "Based on the description, the patient may have...", "Doctor, please see this patient..."), allowing the model to learn to adapt to different questioning styles.

[0060] 2. Knowledge Anchor Dataset (D) Anchor Construction of )

[0061] Data source: 50,000 high-quality samples were selected from large-scale general medical datasets (such as MedQA and CMB) and general common sense datasets (such as C-Eval).

[0062] Selection strategy: Instead of random sampling, a mutually exclusive sampling method based on semantic similarity is adopted. The semantic distance between general data and rare disease data is calculated using an embedding model, and samples that are semantically far from rare disease data (representing general ability) and those that are close to rare disease data but easily confused (representing differential diagnostic ability) are preferentially retained.

[0063] Dynamic updates: During training, the proportion of each subclass in the anchor dataset is dynamically adjusted based on the model's performance on the validation set.

[0064] Part Three: Model Architecture and PEFT Module Initialization

[0065] 1. Base Model Selection

[0066] This embodiment uses Llama 2-70B-Chat as the base model (Teacher Model). This model has powerful general reasoning capabilities, but lacks specific knowledge about rare diseases.

[0067] 2. Fine-grained configuration of the LoRA module

[0068] This invention employs LoRA (Low-Rank Adaptation) as the PEFT technique. The core idea of ​​LoRA is to decompose the weight update matrix ΔW into the product of two low-rank matrices: ΔW = B × A. Target Modules: To maximize the knowledge injection effect, we attach LoRA modules to all linear layers of the Transformer, including: the self-attention layers q_proj (query projection), k_proj (key projection), v_proj (value projection), and o_proj (output projection).

[0069] The gate_proj, up_proj, and down_proj of the feedforward neural network (MLP) layer.

[0070] Note: Compared to injecting only Q / V layers, injecting fully linear layers can improve the model's ability to fit complex medical logic.

[0071] Rank (r): Set r = 16. This is an empirical value that ensures sufficient parameter space to store new knowledge while limiting the number of parameters to prevent overfitting.

[0072] Scaling factor (Alpha, α): Set α = 32. The LoRA output will be multiplied by... A higher α value helps the model quickly capture the features of a new task in the early stages of training.

[0073] Dropout rate: Set lora_dropout = 0.05 as an additional regularization measure.

[0074] Parameter freezing: The code explicitly executes `model.requires_grad_(False)`, then only executes `requires_grad_(True)` on LoRA parameters. The final number of trainable parameters is approximately 160 million, representing only 0.23% of the total parameters.

[0075] Part Four: Training Process of the Dual Loss Function

[0076] This section contains the core algorithm logic of the present invention. The training process is divided into several Epochs (5 in this embodiment).

[0077] Step 1: Batch Construction

[0078] In each iteration (Step), the DataLoader simultaneously retrieves data from D... Task and D Anchor Data extracted from: B Task It contains data on 4 rare diseases.

[0079] B Anchor It contains 4 general anchor point data.

[0080] The total batch size is 8.

[0081] Step 2: Forward Pass

[0082] Teacher's reasoning: B Anchor Input into the frozen base model. Calculate the output Logits: z Teacher =Model Base (B Anchor ).

[0083] This process does not require gradient calculation; the torch.no_grad() context manager is used to save GPU memory.

[0084] Student reasoning: B Task and B Anchor The concatenated data is then input into the currently trained model (base + LoRA). The output Logits are calculated: Z. Student =Model LoRA (B Task ∪B Anchor ).

[0085] Step 3: Loss Calculation

[0086] Calculate the loss L of the new task Task :

[0087] Extract Z Student The middle corresponds to B Task The part that is related to the real label Y Task Calculate the standard cross-entropy loss.

[0088]

[0089] This loss guides the LoRA parameters to learn rare disease knowledge. The knowledge integration loss L is calculated. Consolidation :

[0090] Extract Z Student The middle corresponds to B Anchor Part With Teacher's output Z Teacher Calculate the KL divergence loss.

[0091] To make the probability distribution smoother, a temperature coefficient T = 2.0 is introduced.

[0092]

[0093] P Teacher =Softmax(Z) Teacher / T)

[0094] L Consolidation =T 2 ×KL D ivergence(P Teacher ||P Student )

[0095] This loss forces the Student model to approximate the Teacher model's probability distribution on general data, thus preserving its generality. Calculate the total loss L. Total :

[0096] L Total =L Task +γ×L Consolidation

[0097] In this embodiment, the initial weight γ is set to 1.0.

[0098] Step 4: Backpropagation and Parameter Update

[0099] Execute `loss.backward()` to calculate the gradient. Due to the previous freeze operation, the gradient will only be propagated back to the LoRA matrices A and B.

[0100] Execute `optimizer.step()` to update the LoRA parameters. The optimizer chosen is AdamW, and the learning rate is set to 2×10⁻⁶. -4 This is combined with a Cosine Annealing learning rate decay strategy.

[0101] Part 5: Dynamic Weight Adjustment Mechanism

[0102] To further optimize the training effect, this embodiment introduces a dynamic adjustment mechanism for γ.

[0103] At the end of each epoch, evaluate the model's performance on the validation set: if the model performs well on D... Task If the accuracy improvement is slow (<2%), reduce γ (e.g., γ = γ × 0.8) to make the model more focused on new tasks.

[0104] If the model is in D Anchor If the perplexity level (PPL) increases significantly (>1%), then increase γ (e.g., γ = γ × 1.2) to strengthen the constraint on old knowledge.

[0105] This dynamic adjustment mechanism ensures that the model always maintains the best balance between "learning new knowledge" and "not forgetting old knowledge".

[0106] Part 6: Model Deployment and Inference Acceleration

[0107] After training is complete, you will get a fine-tuned LoRA weight file (e.g., adapter_model.bin, approximately 300MB in size).

[0108] 1. Runtime dynamic loading (Adapter Pattern)

[0109] In the inference service, the original Llama2-70B pedestal model is loaded. Based on the type of user request (e.g., "rare disease consultation"), the corresponding LoRA weights are dynamically loaded into GPU memory. Because the LoRA weights are extremely small, the switching time is only in the millisecond range.

[0110] 2. Weight Merging

[0111] To achieve the ultimate inference speed, LoRA weights can be merged back into the base model during the offline phase:

[0112]

[0113] The merged model W Final Structurally identical to the original model, with no additional computational overhead, it can be directly deployed using high-performance inference engines such as vLLM and TensorRT-LLM.

[0114] 3. Quantization

[0115] To enable deployment on edge devices or in environments with low video memory, this embodiment also includes 4-bit GPTQ quantization of the merged model. Experiments show that the model trained and quantized using the method of this invention can run smoothly on a single consumer-grade graphics card with 24GB of video memory (such as an RTX 4090), and the accuracy loss in rare disease diagnosis tasks is negligible.

[0116] Part 7: Alternative Examples

[0117] In addition to the LoRA-based implementation described above, the present invention can also take the following alternative forms:

[0118] P-Tuning v2 / Prefix Tuning: Trainable parameters are not only injected as weight matrices, but also prepended as virtual tokens to the key / value pairs of the input layer and each Transformer layer. The dual loss function of this invention is also applicable to optimizing these prefix parameters.

[0119] Adapter layer: An Adapter layer consisting of downsampled projection and upsampled projection is inserted sequentially after the Transformer's feedforward network (FFN).

[0120] MSE loss substitutes for KL divergence: in calculating L Consolidation When Z can be calculated directly Student and Z Teacher The mean squared error between the two values. Although the effect is slightly inferior to KL divergence, it has the advantage of faster computation speed.

[0121] Through the detailed description of the above embodiments, those skilled in the art can clearly understand how the present invention achieves efficient and robust rare disease knowledge injection through hardware and software collaboration, sophisticated data processing, and innovative algorithm design.

Claims

1. A method for incremental optimization and knowledge injection for few-sample / rare diseases, characterized in that, Includes the following steps: Step S1: Obtain a pre-trained Large Language Model (LLM) and a small dataset of rare disease knowledge annotated by experts; S2: In the Transformer layer of the LLM, initialize a trainable parameter efficient fine-tuning (PEFT) module and freeze the original parameters of the LLM; S3: Define a dual loss function L Total The L Total Including new task loss L Task And knowledge integration loss L Consolidation Step S4: Input the rare disease knowledge dataset into the LLM (with attached PEFT module), and use the L... Total Calculate the gradient and update only the parameters of the PEFT module; Step S5: Repeat step S4 until the parameters of the PEFT module converge to obtain a new model that retains the original general capabilities.

2. The method according to claim 1, characterized in that, The knowledge integration loss L Consolidation This is used to measure the difference in output between the LLM with the added PEFT module and the pre-trained LLM on a general anchor dataset to prevent catastrophic forgetting.

3. The method according to claim 2, characterized in that, The knowledge integration loss L Consolidation The Kullback-Leibler (KL) divergence between the Logits of the LLM output after the addition of the PEFT module and the Logits of the pre-trained LLM output.

4. The method according to claim 1, characterized in that, The dual loss function L Total Expressed as: L Total =L Tadk +γ·L Consolidation Among them, L Task γ is the cross-entropy loss on the rare disease knowledge dataset; γ is the weight hyperparameter of the knowledge integration loss.

5. The method according to claim 1, characterized in that, The PEFT module includes a LoRA (Low-Rank Adaptation) matrix or an Adapter module.

6. An incremental optimization and knowledge injection system for few-sample / rare diseases, characterized in that, include: The base model module is used to store the pre-trained large language model and freeze its original parameters; The PEFT injection module is used to initialize and manage the PEFT module, and to receive gradients and update its parameters during the training phase. The data management module provides the rare disease knowledge dataset and the general anchor dataset; the dual loss calculation module calculates the new task loss L. Task and the knowledge integration loss L Consolidation .

7. The system according to claim 6, characterized in that, The dual-loss calculation module is used during the inference phase to evaluate the general knowledge retention of the new model and provide feedback signals.

Images