Database Management System Asynchronous Data Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge is to reduce power consumption in data processing systems while maintaining high-frequency data import processes, especially with increasing data amounts, as existing methods like data conversion between row and column store formats are time-consuming and disrupt import processes, leading to outdated search responses.
Innovation Solution
A database management system performs asynchronous data conversion processes, selecting non-converted chunks based on priority points such as data retention and access frequency, allowing for concurrent import and search operations without stopping the import process, and uses a mixed store format approach to reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If data conversion process is performed to reduce power consumption, then power consumption is reduced, but import process must be stopped causing loss of time
Solution Approach 1:
The table data is divided into multiple chunks, and the data conversion process is performed on a per-chunk basis. This segmentation allows the import process to continue writing new chunks while the data conversion process converts existing chunks in the background, eliminating the need to stop the entire import process.
Solution Approach 2:
The data conversion process is performed in advance on chunks that have already been imported, before they are needed for search operations. This preliminary conversion allows the system to prepare converted data ahead of time while the import process continues independently.
2Use of energy by moving object
If data conversion process is performed to reduce power consumption, then power consumption is reduced, but search response timeliness deteriorates
Solution Approach 1:
The system dynamically manages multiple versions of table chunks (original and converted) and adapts search operations to use the most appropriate version. Converted chunks are used when available for power-efficient searches, while original chunks serve as fallback, creating a flexible system that balances power consumption and response speed.
Solution Approach 2:
The system introduces an intermediary mechanism that manages both original and converted table chunks, allowing search operations to selectively access converted data when available. This intermediary layer ensures that search response timeliness is maintained by providing fallback options while still enabling power-efficient operations.
3Use of energy by moving object
If all tables are stored in column store format from the start, then power consumption is reduced, but data conversion process may still be necessary causing loss of time
Solution Approach 1:
Instead of uniformly converting all tables to column store format, the system applies data conversion selectively to specific chunks or portions of tables based on their access patterns and power consumption characteristics. This local approach converts only the necessary data while leaving other data in the original format, reducing overall conversion time.
Solution Approach 2:
The system performs partial data conversion on selected chunks rather than complete conversion of all tables. This partial action is sufficient to achieve power consumption reduction for frequently accessed data while avoiding the time cost of converting all data, representing a balanced approach between power efficiency and conversion overhead.
Data Source
AI summary
A database management system (DBMS) performs, in response to a conversion request to convert a store format of a table in a database, a process (base type change process) for changing a store format indicated by information (base type) indicating the store format of the table to a store format according to the conversion request, and performs a data conversion process asynchronously with the aforementioned process. In an import process after the base type change process, the DBMS imports, into the table, a chunk in the store format indicated by the base type after change. In the data conversion process, the DBMS converts a chunk in a store format different from the store format indicated by the base type into a chunk in the store format indicated by the base type.