Automated Feature Engineering via Variable Property Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated Feature Engineering (AFE) faces challenges in efficiently extracting properties from mathematical transformations in documents, leading to excessive computational efforts and resource wastage, as existing methods require manual intervention and lack targeted feature generation.
Innovation Solution
A computer-implemented method that parses documents to extract mathematical formulas, identifies variables, and determines constraints using ontologies and industry standards, allowing for automatic generation of candidate features and operations like regression, clustering, or classification, without requiring semantic matches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual intervention is used to extract properties from mathematical transformations, then extraction accuracy is improved, but productivity deteriorates due to excessive computational efforts and resource wastage
Solution Approach 1:
The system automatically extracts properties of variables from mathematical transformations without requiring manual intervention. The property extraction module autonomously identifies variables, determines their properties (such as domain, range, data type), and generates candidate features, enabling the system to serve itself and eliminate human involvement in the extraction process.
Solution Approach 2:
The patent replaces manual mechanical extraction processes with automated computational methods. Instead of human experts manually analyzing mathematical transformations to extract variable properties, the system uses automated property extraction algorithms that parse mathematical expressions, identify variables, and determine their properties programmatically, thereby improving productivity while maintaining accuracy.
2Reliability
If all input data columns are processed for feature engineering, then completeness of feature generation is improved, but loss of energy increases due to processing irrelevant data
Solution Approach 1:
The system extracts and processes only the relevant subset of input data columns that conform to the determined variable properties. By filtering out columns that do not match the expected data types, domains, or ranges of the extracted variables, the system avoids processing irrelevant data while maintaining completeness of feature generation for applicable columns.
Solution Approach 2:
The patent applies different processing quality to different data columns based on their relevance. Columns that match the extracted variable properties receive full processing attention for candidate feature generation, while non-matching columns are excluded from processing. This localized quality approach ensures completeness where needed while conserving computational resources where not needed.
3Adaptability or versatility
If automated feature engineering processes all mathematical transformations without constraints, then versatility is improved, but device complexity increases due to lack of targeted processing
Solution Approach 1:
The system performs preliminary actions by extracting variable properties and determining constraints from mathematical transformations before processing the input data. This preliminary property extraction and constraint determination step organizes the processing requirements in advance, enabling the subsequent feature engineering to be targeted and efficient rather than brute-force processing of all possible transformations.
Data Source
AI summary
A computer-implemented method of extracting properties of one or more variables of a mathematical transformation in a document identified by automatic feature engineering (AFE) is provided. The document is parsed to extract at least one of a mathematical formula including identifying data, a textual description of the one or more variables of the mathematical transformation, or a category identifier to which the document belongs. Constraints are determined to apply to the one or more variables extracted from the mathematical transformations in the document. At least one candidate feature is automatically generated from a portion of an input data that conforms to the determined constraints.