Offline Tree Data Preparation for Bulk Insert Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Creating tree data structures from bulk data is time-consuming and processing-intensive, particularly when inserting data items step-by-step, which blocks read operations and increases network traffic.
Innovation Solution
A method for generating a tree data structure from bulk data by resorting sorted data items according to levels and node ordering, using a data table to represent the tree structure, allowing efficient bulk insertion and reducing network utilization by preparing the data table offline before sending it to the server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If step-by-step insertion of data items is used to create a tree data structure, then the tree structure is built correctly, but processing time and processing power consumption increase significantly
Solution Approach 1:
The patent applies preliminary action by sorting the bulk data items before insertion and pre-determining their positions in the tree structure. The server system sorts the received data items and calculates insert positions in advance, rather than determining positions during the insertion process itself. This preliminary preparation significantly reduces the processing time required during actual insertion operations.
2Manufacturing precision
If step-by-step insertion of data items is used, then the tree structure is built correctly, but processing power consumption increases significantly
Solution Approach 1:
The patent reduces processing power consumption by performing data sorting and position calculation in advance before the insertion operations begin. The server system prepares the sorted data and determines all insert positions beforehand, which minimizes the computational resources required during the actual bulk insertion process.
3Productivity
If step-by-step insertion is performed, then data is inserted into the tree structure, but read operations are blocked during the long insertion process
Solution Approach 1:
The patent applies segmentation by dividing the bulk data into smaller batches or groups that can be inserted in sequential segments. Rather than performing a single long-running insertion operation that blocks read access, the data is processed in multiple smaller insertion operations, allowing read operations to potentially occur between batches and reducing the overall blocking impact.
4Productivity
If bulk data is sent to the server for tree creation, then the tree structure is built, but network traffic increases
Solution Approach 1:
The patent merges multiple small data insertion operations into a single bulk operation. By combining all the data items into one bulk insert request rather than sending multiple individual insertion requests, the patent reduces network overhead and utilization. The sorting and position determination are performed on the server side after receiving the bulk data, eliminating the need for multiple round-trip communications.
Data Source
Figure 1~2
Figure 3
Figure 4~6
AI summary
The present disclosure relates to a method for providing a tree data structure from bulk data comprising data items. The method comprises: providing an algorithm adapted for resorting sorted data items in accordance with levels and the ordering of the nodes within each level of a tree data structure, each data item corresponding to a node of the tree data structure; receiving at a client computer the data items to be stored on the tree data structure; sorting the received data items in accordance with a predefined sorting order; processing at the client computer the algorithm for resorting the sorted received data items; generating a data table for representing the tree data structure in a tabular form having columns and rows, wherein in accordance with the resorting order each row of the data table represents a respective resorted data item, wherein values in the columns are determined in accordance with the order of the rows; storing the data table as the tree data structure.