Machine-Trained Model Data Structure for Local Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge is to efficiently implement a large language model on a local execution platform, which often lacks sufficient storage and memory capacity, and also faces latency and privacy concerns when interacting with network-accessible resources.
Innovation Solution
A data structure is used to store a machine-trained model, with paths from a root node to leaf nodes representing sequences of processing blocks. Each node on the main root-to-leaf path is associated with base model weights, while non-main paths include model-variance information, allowing for efficient storage and execution of the model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a complete large language model is stored locally, then model execution capability is improved, but storage and memory requirements become prohibitive
Solution Approach 1:
The patent segments the language model into a data structure with multiple root-to-leaf paths, where only the main path is stored locally in full detail and alternative paths are stored as variants. This segmentation allows the system to execute model operations locally with reduced storage requirements while maintaining the ability to handle different query types.
Solution Approach 2:
The patent extracts frequently used model operations along the main root-to-leaf path and stores them locally, while alternative operations are stored as separate variant data. This extraction enables the local system to execute common operations without storing the complete model, reducing storage requirements while maintaining execution capability for essential functions.
2Measurement precision
If the complete model weights are transferred from online source, then model accuracy is improved, but transfer time becomes significant
Solution Approach 1:
The patent segments the model weights into main path weights and variant weights, transferring only the essential main path weights to the local system. The variant weights are kept online and transferred on-demand when alternative execution paths are needed, significantly reducing initial transfer time while preserving model accuracy through selective weight application.
Solution Approach 2:
The patent implements partial action by transferring and storing only the critical portion of model weights (main path) locally, sufficient for basic model accuracy. Additional variant weights are transferred partially and on-demand based on query requirements, reducing overall transfer time while maintaining necessary model performance.
3Object-affected harmful factors
If interaction with network-accessible resources is limited, then privacy protection is improved, but latency costs increase
Solution Approach 1:
The patent segments model operations into locally-executable main path operations and network-accessible variant operations. By keeping the main execution path locally, the system minimizes network interactions and reduces latency for common operations while maintaining privacy protection. Network access is only invoked when variant operations are specifically needed.
Solution Approach 2:
The patent enables the local system to self-serve by executing model operations using locally stored main path weights without requiring continuous network access. The system independently handles the majority of inference tasks locally, reducing latency and network dependency while maintaining privacy, and only accesses network resources when variant operations are required.
Data Source
AI summary
A data structure describes a machine-trained model using a data structure that includes a plurality paths between a root node and respective leaf nodes. One such path is a main root-to-leaf (RTL) path, while other paths are referred to as non-main-RTL paths. Each node along the RTL path is associated with a portion of base model weights. At least one node along a non-main-RTL path is associated with a portion of model-variance information. A training system trains the portions of model-variance information as variations of corresponding portions of base model weights, while keeping the portion of base model weights fixed. In some cases, a local system obtains portions of model weights described by the data structure from a source system on an as needed-basis. The above characteristics contribute to the efficient storage, transfer, and execution of the machine-trained model.