Text Attribute Normalization for ML Model Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models face challenges in determining the predictive utility of text attributes, which are crucial for accurate predictions, due to their large token counts and varying token frequencies, leading to increased training time and resource consumption.
Innovation Solution
A normalization-based technique is employed to convert text attributes into categorical attributes, allowing for the computation of correlation metrics and identification of predictive utility, thereby reducing the complexity of text attributes and enhancing model efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If text attributes with large token counts are used directly in machine learning models, then the model can capture more information from text data, but the training time and computational resources increase significantly
Solution Approach 1:
The patent extracts only the most relevant and frequent tokens from text attributes, removing redundant and less informative tokens. This selective extraction maintains the essential information content while significantly reducing the total number of tokens that need to be processed during model training, thus resolving the contradiction between information capture and training efficiency
Solution Approach 2:
The patent transforms text attributes into categorical representations that are computationally cheaper to handle. By converting verbose text data into compact categorical forms with fewer distinct values, the model can process the information more efficiently during training, reducing computational resources and time while preserving predictive utility
2Adaptability or versatility
If text attributes with varying token frequencies are used directly, then the model can capture diverse text patterns, but the complexity of determining predictive utility increases
Solution Approach 1:
The patent changes the parameter representation of text attributes from raw token counts to normalized categorical representations. By applying frequency thresholds and converting to categorical forms, the patent standardizes the representation of text attributes with varying token frequencies, making predictive utility determination more straightforward and less complex while still capturing diverse text patterns
3Loss of information
If more text attributes are included in the data set, then the model can leverage more information for predictions, but the memory consumption and training resources increase
Solution Approach 1:
The patent extracts and retains only the most predictive and frequent tokens from text attributes, discarding redundant tokens. This selective extraction reduces the quantity of data that needs to be stored in memory while preserving the essential predictive information, thus resolving the contradiction between information retention and memory consumption
Solution Approach 2:
Instead of storing and processing all original text tokens, the patent inverts the approach by storing only the essential categorical representations of the most important tokens. This inversion reduces memory requirements while maintaining the predictive power of the text attributes
Data Source
AI summary
Respective correlation metrics between token groups of a particular text attribute of a data set and a prediction target attribute are computed. Based on the correlation metrics, a predictive token group list is created. For various observation records of the data set, values of a derived categorical attribute corresponding to the particular text attribute are determined based on matches between the particular text attribute value and the predictive token group list. A measure of the predictive utility of the particular text attribute is obtained using correlations between the categorical attribute and the prediction target attribute.