Quantitative Language Model Training With Target-Specific Detailed Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques fail to account for the differences in distribution of numerical values for different targets and attributes, leading to inaccurate model training and estimation.
Innovation Solution
A training device and method that normalizes numerical values and units, specifies targets and attributes, and converts units into detailed units corresponding to the target-attribute combinations, using a quantitative-representation language model for accurate estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional techniques are used for training, then the training process is simple, but the model accuracy deteriorates due to inability to account for distribution differences
Solution Approach 1:
The patent segments the training process into distinct stages: pre-training with general data, then attribute-specific fine-tuning. It also segments the unit conversion into target-attribute-specific detailed units, allowing the model to learn distribution characteristics for each combination separately. This segmentation enables the model to achieve high accuracy by focusing on specific distributions rather than treating all data uniformly.
Solution Approach 2:
The patent applies local quality by creating target-attribute-specific detailed units that capture the unique distribution characteristics of each combination. Instead of using a single universal unit representation, the system creates localized unit representations tailored to each target-attribute pair, allowing the model to account for distribution differences specific to each local case while maintaining overall system coherence.
2Measurement precision
If numerical values are normalized without considering target-attribute combinations, then the normalization process is simple, but the estimation accuracy deteriorates
Solution Approach 1:
The patent segments unit conversion into target-attribute-specific detailed units, where each combination of target and attribute receives its own specialized unit representation. This segmentation allows the system to normalize and convert units according to the specific distribution characteristics of each target-attribute pair, significantly improving estimation accuracy while managing complexity through structured organization.
Solution Approach 2:
The patent changes the parameter of unit representation from generic to specific, creating detailed units that are parameters of the target-attribute combination. This parameter change enables the system to adapt unit representations to the specific requirements of each combination, allowing for accurate normalization and conversion that accounts for distribution differences without overwhelming complexity.
Data Source
AI summary
A training device includes: a quantitative-expression specifying unit that specifies a quantitative expression that expresses a quantity using a numerical value and a unit from the training source data; a numerical-value normalizing unit that normalizes the numerical value; a unit normalizing unit that normalizes the unit; a unit detailing unit that specifies a target, which is a physical entity, and an attribute, which is the property of the target and indicated by a numerical value and a unit, in the training source data, and converts a normalized unit into a detailed unit uniquely corresponding to a combination of the specified target, the specified attribute, and the normalized unit; and a quantitative-expression training unit that trains a quantitative-representation language model for estimating normalized numerical values by using data including detailed units and normalized numerical values.