OLAP String Field to Numeric Mapping via Row-Based Association
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Online Analytical Processing (OLAP) systems, the processing speed for string type fields is slower than numerical type fields due to storage form issues, leading to performance bottlenecks during complex queries and data updates.
Innovation Solution
A data processing method and apparatus that utilize a row-based table to store associated data pairs, allowing string type fields to be converted into numerical type fields, and enabling efficient point queries and updates by decoupling storage and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If string type fields are stored directly in the database, then the data can be stored with high fidelity, but the querying speed and processing efficiency deteriorate
Solution Approach 1:
The patent transforms string type field values into numerical type field values through encoding. Specifically, it creates an associated data pair set where string fields are mapped to numerical identifiers, and stores these numerical values in the database instead of the original string values. This parameter transformation from string to numerical type resolves the contradiction by enabling faster querying and processing while maintaining data integrity through the association mapping.
Solution Approach 2:
The patent introduces an associated data pair set as an intermediary structure that contains the mapping relationship between string fields and their corresponding numerical values. This intermediary layer allows the system to store compact numerical representations in the database while preserving the ability to retrieve and interpret the original string data through the association mapping, thus resolving the speed-fidelity contradiction.
2Loss of information
If join operations are performed to associate string fields with numerical fields, then data completeness is improved, but operation cost and complexity increase
Solution Approach 1:
The patent performs the association mapping setup in advance by pre-establishing the associated data pair set that maps string fields to numerical fields. This preliminary action eliminates the need for complex join operations during query execution, as the mapping relationship is already prepared and stored. The system can directly use the pre-computed numerical identifiers without performing costly join operations at query time, thus reducing operation complexity while maintaining data completeness.
3Adaptability or versatility
If string type fields are used in OLAP operations, then data expressiveness is improved, but processing speed deteriorates
Solution Approach 1:
The patent applies parameter transformation by converting string type field values into numerical type field values through encoding schemes. The system creates an associated data pair set that preserves the semantic meaning of string fields while representing them as compact numerical identifiers suitable for efficient OLAP processing. This transformation enables fast numerical comparisons, aggregations, and sorting operations while the association mapping maintains the original data expressiveness when needed.
Data Source
AI summary
A data processing method based on online analytical processing, an electronic device and a storage medium are provided. The method includes: receiving a data operation request, wherein the data operation request includes a first field; determining a target associated data pair from a set of associated data pairs stored in a row-based manner based on the data operation request, and determining a second field associated with the first field according to the determined target associated data pair; taking a field value corresponding to the second field in a fact table of a first database as a target field value corresponding to the first field, and executing an operation indicated by the data operation request for the target field value; wherein fields belonging to the second data type among the associated data pairs are stored in the fact table.