Dual-Window Data Querying for Real-Time and Complete Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of querying big data is the high query difficulty due to its large volume, diverse formats, and scattered sources, making comprehensive and accurate query processing difficult.
Innovation Solution
A data query method involving a first database for long-term data storage ensuring comprehensiveness and consistency, and a second database for short-term data storage with high update frequency to ensure real-time performance, complementing each other to enhance query efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a single database stores all big data for comprehensive querying, then data comprehensiveness is improved, but query difficulty and time increase due to large data volume
Solution Approach 1:
The patent divides the database into multiple partitions or segments based on data characteristics (e.g., hot data, warm data, cold data). Each partition stores specific types of data, allowing the query system to selectively access only relevant partitions rather than scanning the entire database. This segmentation reduces query time while maintaining comprehensive data coverage across all partitions.
Solution Approach 2:
The patent introduces a temporal dimension by implementing multi-version data storage, where data is stored with version timestamps. The query system can access different data versions based on time requirements - latest versions for real-time queries or historical versions for trend analysis. This adds a time dimension to data organization, enabling efficient querying across different time periods without scanning all data.
2Loss of information
If a single database stores all big data for accurate querying, then data completeness is improved, but query accuracy decreases due to data diversity and scattering
Solution Approach 1:
The patent applies local quality by creating specialized sub-databases or partitions optimized for specific data types or query patterns. Each partition has tailored data structures, indexing strategies, and access methods suited to its specific data characteristics. This allows high query accuracy for each data type while maintaining complete data coverage across all specialized partitions.
Solution Approach 2:
The patent introduces a data integration layer or intermediary that aggregates data from multiple scattered sources and presents a unified view to users. This intermediary layer performs data cleaning, standardization, and correlation analysis, transforming scattered data into structured, queryable formats. It maintains data completeness from all sources while improving query accuracy through centralized data management and correlation capabilities.
3Quantity of substance
If data is stored in distributed architecture for big data management, then data storage capacity is improved, but query difficulty increases due to data scattering
Solution Approach 1:
The patent merges multiple distributed data nodes into a logical unified database system with centralized coordination. While data is physically distributed across multiple nodes for scalable storage, the system provides unified query interfaces and centralized metadata management that abstract away the distribution complexity. Users query the unified system without needing to understand or manage the underlying distributed architecture, reducing query complexity while maintaining large storage capacity.
Data Source
AI summary
The present disclosure provides a data query method. In the method, a first query result of a first database in response to a first query request is obtained. The first database stores data within a first time window. A second query result of a second database is obtained when the first query result does not meet a preset check condition. The second database stores data within a second time window. A start time of the second time window being after a start time of the first time window. A target query result based on at least one of the first query request and the second query result is generated by processing circuity. An update frequency of the second database is higher than an update frequency of the first database.