Parameter efficient adapter for artificial intelligence systems
By employing low-rank adaptation and low-dimensional nonlinear adaptation techniques, the problems of high computational resources and insufficient adaptability of large language models in domain-specific tasks are solved, achieving efficient and low-storage fine-tuning of models and improving performance in specific tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MITSUBISHI ELECTRIC CORP
- Filing Date
- 2024-11-06
- Publication Date
- 2026-06-02
AI Technical Summary
Existing large language models (LLMs) have high computational and storage requirements during training and inference, and perform suboptimally in domain-specific tasks. Traditional parameter efficient fine-tuning techniques (PEFT) have limited task-specific adaptability.
We employ low-rank adaptation (LoRA) and low-dimensional nonlinear adaptation (LoDA) techniques, constrain weight updates through low-rank decomposition matrices, and combine nonlinear modifiers and AI trainers to fine-tune pre-trained models to adapt to specific tasks or domains.
It achieves efficient improvement of model performance in a specific domain, reduces storage and power consumption, and maintains high-quality inference performance under limited computing resources and data conditions.
Smart Images
Figure CN122139191A_ABST