Unified Data File Querying for Online-Offline Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data inconsistency occurs between online and offline query modes due to different data write timings, leading to inaccurate query results in scenarios involving large data sets.
Innovation Solution
Utilize a unified data file system for both online and offline queries, employing index information for online queries and address information based on keywords for offline queries to ensure consistency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate databases are used for online and offline queries, then query performance is optimized for each mode, but data inconsistency occurs between the two databases
Solution Approach 1:
The patent merges the storage systems for online and offline queries into a single data file system. Both query modes access the same underlying data storage, eliminating data inconsistency while maintaining separate query processing paths for optimized performance.
Solution Approach 2:
The patent segments the query processing into separate online and offline paths, but both paths access the unified storage system through different mechanisms (index-based for online, keyword-based for offline). This segmentation maintains performance optimization while ensuring data consistency.
2Productivity
If different storage systems are used for online and offline queries, then query efficiency is improved, but data query accuracy decreases due to deviations
Solution Approach 1:
By merging the storage systems, the patent ensures that both online and offline queries access the same data source, eliminating deviations and ensuring query accuracy while maintaining efficiency through separate query processing paths.
Solution Approach 2:
The patent introduces index information and keyword-based address information as intermediaries that map query requests to the unified data file system. These intermediaries enable efficient query processing while ensuring accurate data retrieval from the consistent storage source.
3Reliability
If a unified data file system is used for both query modes, then data consistency is ensured, but query performance may be impacted
Solution Approach 1:
The patent segments the query processing into separate online and offline paths with different optimization strategies. Online queries use index-based access for fast retrieval, while offline queries use keyword-based address information for comprehensive search, both accessing the same unified storage system.
Solution Approach 2:
The patent applies different access methods (index-based vs. keyword-based) tailored to the specific needs of online and offline queries. This local quality approach optimizes performance for each query type while maintaining data consistency through the unified storage system.
Data Source
AI summary
The present disclosure provides a data processing method and apparatus, a computer device and a storage medium. The method comprises: receiving a query instruction for data to be queried stored in a data file system; in response to the query instruction instructing to perform online query on the data, determining index information corresponding to the data, and querying the data in the data file system on the basis of the index information; and in response to the query instruction instructing to perform offline query on the data, determining address information of the data in the data file system on the basis of a keyword of the data, and querying the data in the data file system on the basis of the address information.