Reliance Level Filtering in Data Stream Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data stream management systems face challenges in handling real-time, continuous data streams with varying reliability, as traditional database management systems are not equipped to differentiate between reliable and unreliable data sources, leading to potential inaccuracies in query results.
Innovation Solution
The introduction of a reliance level attribute in the Continuous Query Language (CQL) allows users to specify a minimum reliability level for data streams, enabling the data stream management system to filter and execute queries only on trusted sources, ensuring accurate and reliable data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional database management systems are used to manage data streams, then the system can handle continuous real-time data, but the system cannot differentiate between reliable and unreliable data sources leading to inaccurate query results
Solution Approach 1:
The patent introduces a reliance level parameter associated with each data streaming source, transforming the traditional binary trust model into a multi-level parameterized approach. This parameter allows the system to differentiate between reliable and unreliable sources quantitatively, enabling accurate query execution while maintaining manageable system complexity through standardized parameter assignment and filtering mechanisms
2Measurement precision
If the system filters data streams based on reliance levels, then query accuracy improves, but processing time increases due to additional filtering operations
Solution Approach 1:
The system performs preliminary filtering of data streams based on reliance levels before query execution. By pre-evaluating and filtering out unreliable data sources in advance, the system ensures high query accuracy while minimizing processing time during actual query execution, as the filtering framework is already established and ready to exclude low-reliance streams immediately upon query registration
3Productivity
If the system processes all data streams without filtering, then processing speed is maintained, but query results include data from unreliable sources reducing accuracy
Solution Approach 1:
The system extracts and separates reliable data streams from unreliable ones based on reliance level thresholds. By taking out only the necessary high-reliance data streams for processing and excluding low-reliance streams, the system maintains high processing speed for relevant data while ensuring query result reliability through selective data inclusion rather than processing all streams
Data Source
AI summary
A computer-implemented method comprises receiving a plurality of data streams into a data stream management system, wherein each data stream is obtained from a data streaming source and a reliance level is associated with each of the data streaming sources. When a continuous query is registered that specifies a reliance level attribute, the query is executed over the plurality of data streams using only those data streams obtained from a data streaming source associated with a suitable reliance level. For example, the reliance level attribute may establish a minimum reliance level, such that data stream having a lesser reliance level are not used. CQL may be enhanced with such an extra attribute which allows end users to specify the desired reliance level when submitting a query.