Model Training with Type-Specific Parameter Information
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine-learned models face performance issues when trained on limited or irrelevant data, leading to substandard performance in target environments, especially when there is a lack of robust training sets specific to those environments.
Innovation Solution
The technology combines data items with type-specific parameter information to create supplemented data items, allowing for iterative model weight adjustments and adaptation to new subject matter domains, enabling the model to transfer knowledge across different types of data even without specific training data, and uses trainable parameter information to refine and fine-tune the model for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If a model is trained on limited or irrelevant data, then training resource consumption is reduced, but model performance in the target environment deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-training a model on general domain data before fine-tuning it on specific target domain data. This preliminary training phase establishes a foundational understanding that can be later adapted, allowing the system to achieve good performance even when target domain data is limited or unavailable, thus resolving the contradiction between reduced training resources and maintained model performance
Solution Approach 2:
The patent employs parameter changes by adjusting model weights during fine-tuning on target domain data. The system modifies specific parameters of the pre-trained model to adapt to the target environment, enabling the model to maintain high performance with minimal target domain training data, thereby resolving the contradiction between training resource consumption and model performance
2Reliability
If a model is trained on data from the target environment, then model performance is improved, but the ability to adapt to new domains deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the training process into distinct phases: pre-training on general domain data and fine-tuning on target domain data. This segmentation allows the model to first learn universal patterns that enhance adaptability, then specialize on target domain specifics, thereby maintaining both high performance and domain adaptation capability
Solution Approach 2:
The patent implements universality by designing a model architecture and training approach that enables the same model to perform across multiple domains. The pre-training phase on diverse general domain data gives the model universal knowledge, while the fine-tuning phase adapts it to specific domains, allowing the model to maintain both specialized performance and broad adaptability
3Reliability
If type-specific parameter information is used during training, then model performance is improved, but device complexity increases
Solution Approach 1:
The patent applies local quality by using type-specific parameter information tailored to different data types during training. Instead of applying a uniform training approach, the system adjusts parameters specifically for each data type (e.g., text, code, multilingual data), which improves model performance while keeping the complexity increase localized and manageable through systematic parameter management
Data Source
AI summary
A training process produces a machine-learned model that, once trained, can be applied to process different types of data items. The training process accomplishes this result by combining data items in a training set with type-specific parameter information, to produce supplemented data items. The training process then trains a model based on the supplemented data items. Training involves adjusting model weights together with the type-specific parameter information. In an inference stage of processing, the technology combines a new data item with an appropriate type of trained parameter information, and then maps the resultant supplemented data item to an output data item. The technology is particularly effective in adapting an initial model to a new subject matter domain in those situations in which a robust set of data items that pertain to the subject matter domain and which have a desired type is lacking.