OPTIMIZATION OF LARGE LANGUAGE MODELS WITH DOMAIN-ORIENTED MODEL COMPRESSION

Optimizing LLMs with domain-oriented model compression enhances their performance on specific tasks by balancing general and domain knowledge, reducing parameters, and improving efficiency for resource-constrained devices.

DE112024003352T5Pending Publication Date: 2026-06-03NEC LABORATORIES AMERICA INC
View PDF 6 Cites 0 Cited by

Patent Information

Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
NEC LABORATORIES AMERICA INC
Filing Date
2024-08-16
Publication Date
2026-06-03

AI Technical Summary

Technical Problem

Large Language Models (LLMs) face challenges in performing specific language tasks due to their size and complexity, making them impractical for devices with limited resources and limiting their ability to balance general and domain-specific knowledge.

Method used

A method for optimizing LLMs through domain-oriented model compression by determining meaning weights for general and domain knowledge, iteratively fine-tuning, and pruning unnecessary parameters using gradient descent and regularization, resulting in a domain-compressed LLM that retains general knowledge while enhancing domain-specific performance.

Benefits of technology

The optimized LLMs perform domain-specific tasks more efficiently and effectively, reducing parameter count by half or more, making them suitable for resource-constrained environments and enabling versatile deployment across various domains.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Systems and methods for optimizing large language models (LLMs) with domain-oriented model compression. Meaning weights for general knowledge in a trained LLM pre-trained with deep learning can be determined (110) by calculating the error of removing a weight from the trained LLM.The trained LLM can be iteratively optimized (120) to obtain a domain-compressed LLM with domain knowledge while preserving general knowledge by: iteratively fine-tuning (130) the trained LLM with domain knowledge using the general knowledge meaning weights to obtain a fine-tuned LLM; determining (140) meaning weights for domain knowledge in the LLM with a regularization term by using gradient descent to optimize parameters when training the fine-tuned LLM with domain knowledge; and pruning (150) learned knowledge based on meaning weights for domain knowledge. A corrective action can be performed on a supervised entity using the domain-compressed LLM (160).
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • 18/805,978

  • US18805978B1

  • US63539681B1

  • US63532924B1

  • 63/539,681