Date Proximity Feature Engineering With Epoch-Date Lookup Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for calculating date differences in feature engineering require extensive computing resources and storage due to the need for large matrices and repeated calculations, leading to inefficiencies in processing time and resource allocation.
Innovation Solution
The use of data structures with epoch dates to index and store integer differences, allowing for efficient determination of days since a previous target date and until a next target date without recalculating, utilizing simple one-dimensional arrays instead of matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional methods are used to calculate date differences by creating a D×H matrix of integer values, then the completeness of date difference calculations is improved, but the storage requirements and processing time increase significantly
Solution Approach 1:
The patent extracts only the necessary date difference information from the complete D×H matrix by using a hash map to store only the unique date differences needed for the analysis. This selective extraction maintains measurement precision while dramatically reducing storage requirements compared to storing the entire matrix.
Solution Approach 2:
The patent creates a simplified representation of the date difference data using a hash map that copies only the essential information (date differences) rather than copying the entire D×H matrix. This copying approach preserves the necessary measurement precision while using minimal storage space.
2Measurement precision
If conventional methods are used to calculate date differences by looping through the matrix row by row, then the accuracy of finding minimum and maximum values is improved, but the processing time increases significantly
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing date differences in a hash map before the main processing loop. This allows the subsequent processing to simply retrieve pre-computed values rather than recalculating them, maintaining accuracy while significantly reducing processing time.
Solution Approach 2:
The patent replaces the mechanical system of nested loops for calculating and searching date differences with a more efficient hash map-based lookup system. This substitution maintains the accuracy of finding minimum and maximum values while dramatically reducing the processing time required.
3Adaptability or versatility
If conventional methods are used to calculate all date differences, then the comprehensiveness of the analysis is improved, but the computational resources required increase significantly
Solution Approach 1:
The patent applies local quality by calculating and storing only the specific date difference values that are actually needed for the analysis, rather than computing all possible date differences. The hash map stores only the unique date difference values that appear in the data, maintaining comprehensiveness while reducing computational resource usage.
Solution Approach 2:
The patent changes the parameter of how date differences are stored and processed by using a hash map data structure instead of a complete matrix. This parameter change allows the system to maintain comprehensive analysis capabilities while significantly reducing the computational resources required for processing.
Data Source
AI summary
Methods and systems that improve upon the speed of engineering features related to date differences. Such methods and systems comprise one or more data structures that can use epoch dates to establish an index in the data structure, and integer differences as values in the data structure. The data structures can then be used as a lookup for a date of interest, to efficiently determine the number days since a previous target date and the number of days until a next target date.