Parameter Condition Injection for Data Set Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face inefficiencies when handling large data sets for complex analytic processes, such as molecular docking, which require significant time and resources due to the need to evaluate extensive data sets.
Innovation Solution
The method involves configuring computer processors to receive a data set, identify parameter conditions with predefined relationships, and evaluate each data record to generate a reduced data set, thereby streamlining the processing by performing analytic routines only on relevant data, using a data abstraction model to decouple the user from the underlying database and allowing for abstract queries independent of the database schema.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large data set is processed by complex analytic routines, then comprehensive analysis coverage is achieved, but processing time and resource consumption increase significantly
Solution Approach 1:
The system performs preliminary filtering of the data set before executing complex analytic routines. By evaluating data records against predefined conditions and parameters in advance, the system identifies and removes irrelevant records, creating a reduced data set that maintains analysis coverage while significantly decreasing processing time and resource requirements
Solution Approach 2:
The system extracts only the relevant subset of data records that meet specific conditions and parameters. This extraction process separates useful data from unnecessary data, allowing complex analytic routines to operate efficiently on a smaller, more targeted data set while preserving the integrity and comprehensiveness of the analysis
2Productivity
If parameter conditions are injected to reduce data set size, then processing efficiency improves, but system complexity increases
Solution Approach 1:
The system introduces a condition injection mechanism that acts as an intermediary between the data set and complex analytic routines. This intermediary layer evaluates data records against predefined conditions and parameters, filtering the data set before it reaches the analytic routines. The mechanism manages the complexity of data filtering while preserving processing efficiency, as it systematically applies conditions without requiring complex modifications to the underlying analytic routines
3Measurement precision
If comprehensive data sets are used for analytic processes, then result accuracy is maintained, but resource consumption increases
Solution Approach 1:
The system applies different quality criteria to different portions of the data set by evaluating each data record against specific conditions and parameters. Relevant records that meet the criteria are retained for analysis, while irrelevant records are filtered out. This local quality approach ensures that the reduced data set maintains the necessary accuracy for meaningful analysis while significantly reducing resource consumption compared to processing the entire comprehensive data set
Data Source
AI summary
Systems, methods and articles of manufacture are disclosed for injecting parameter conditions to reduce a size of a data set to be processed. A data set may be received to be processed by an analytic routine. The data set may include results from executing a first query. Further, input may be received from a user, the input specifying a parameter specific to the analytic routine to be performed on the data set. At least one condition associated with the specified parameter may be identified. A second query may be generated from the first query and the identified condition. The second query may be executed to generate a reduced data set. The analytic routine may be performed on the reduced data set to generate an output to the user.