Inverted Index System for Multimodal Data Crossover Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data processing, multi-dimensional crossover queries in data warehouses often result in complex SQL logic and long query times due to the need to query multiple data tables, leading to low data index efficiency.
Innovation Solution
A data processing method and apparatus that constructs a metadata management table from target data tables with multiple index keywords, performs feature mapping, and merges data feature tables to create an index table, enabling efficient crossover queries through an inverted index system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multi-dimensional crossover queries are performed on data tables using traditional SQL methods, then query coverage across multiple data types is achieved, but query time increases and data index efficiency decreases
Solution Approach 1:
The patent pre-generates inverted indexes for multiple data types (text, numerical, LBS, classification) before queries are executed. This preliminary action stores data in an optimized format with pre-computed indexes, so that when crossover queries are performed, the system can directly utilize these pre-prepared structures rather than processing raw data tables in real-time, thereby reducing query time while maintaining comprehensive query coverage
Solution Approach 2:
The patent introduces an inverted index as an intermediary data structure between the original data tables and the query interface. This intermediary layer translates complex multi-dimensional crossover queries into efficient index-based lookups, mediating between the user's query requirements and the underlying data storage, thus improving query efficiency without sacrificing adaptability
2Adaptability or versatility
If complex SQL logic expressions are used to handle multi-dimensional crossover queries, then query flexibility is maintained, but system complexity increases
Solution Approach 1:
The patent changes the parameter representation of data by converting original data into inverted index format with specific parameters for each data type (text indexes, numerical ranges, LBS grids, classification codes). This parameter transformation allows the system to handle diverse query types through standardized index operations rather than complex SQL logic, maintaining query flexibility while reducing system complexity
Solution Approach 2:
The patent segments the data storage system into multiple specialized inverted index structures, each optimized for a specific data type (text indexes for string data, numerical indexes for numbers, LBS indexes for location data, classification indexes for categorical data). This segmentation allows each index type to handle its specific queries efficiently without requiring complex cross-type SQL logic, thereby reducing system complexity while maintaining query flexibility
Data Source
AI summary
A data processing method and apparatus, a device, and a non-transitory computer-readable storage medium are provided. A target data table is obtained that includes at least two index keywords corresponding to a target object. Metadata information corresponding to the target data table is merged to obtain a metadata management table. Feature mapping is performed on data in the target data table according to the metadata management table in order to obtain data features corresponding to the at least two index keywords in the target data table and in order to generate data feature tables. The data feature tables are merged to obtain an index table. A metadata management table is constructed through the metadata information of the target data table. An index table corresponding to the target data table is generated according to the metadata management table to implement the inverted index of the data through the index table.