Hierarchical Statistics Collection via Lockless Subtree Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Collecting statistical information for hierarchically-organized data collections is resource-intensive and time-consuming, especially in large datasets, often requiring system downtime and leading to incomplete or redundant data gathering due to changes in the data during scheduled collection periods.
Innovation Solution
The method leverages container-level modification timestamps and a lockless data structure to selectively gather statistical information only from resources that have changed since the last gathering, using subtree-level change information to optimize data collection and reduce system resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical information is collected from every resource in a hierarchical collection, then complete and accurate statistics are obtained, but system resources and time are consumed excessively, interfering with other processing
Solution Approach 1:
The patent segments the hierarchical collection into subtrees rooted at container resources, and further segments the statistics collection process by identifying only those subtrees that contain changed resources. This segmentation allows the system to collect statistics from a subset of resources (only changed ones) rather than the entire collection, reducing resource consumption while maintaining statistical accuracy for the changes that occurred.
Solution Approach 2:
Instead of collecting statistics from all resources in the hierarchical collection, the patent applies partial action by collecting statistics only from resources that have changed since the last collection. This partial collection approach is sufficient for updating statistics while avoiding the excessive action of processing unchanged resources, thereby improving system productivity.
2Object-affected harmful factors
If statistics are collected during scheduled system downtime, then interference with client processing is minimized, but the collection may be incomplete or require extension beyond allocated time
Solution Approach 1:
The patent performs preliminary action by tracking changes to container resources and their subtrees before the statistics collection process. By maintaining information about which subtrees contain changed resources, the system can prepare a targeted collection plan that fits within the scheduled downtime window, ensuring both completeness and timing constraints are met.
Solution Approach 2:
The patent introduces dynamics by using modification timestamps and change tracking mechanisms that adapt the statistics collection process to actual data changes. This dynamic approach allows the system to collect statistics only when and where changes occur, rather than following a rigid scheduled process, thereby ensuring completeness within the allocated time while minimizing interference with client operations.
3Measurement precision
If the collection of statistics continues beyond scheduled downtime, then complete statistics are gathered, but time-critical processing is interfered with
Solution Approach 1:
The patent segments the statistics collection task into identifying changed resources versus unchanged resources. By segmenting this way, the system can complete the essential statistics collection for changed resources within the scheduled downtime, while knowing that unchanged resources do not require re-collection, thus avoiding time-critical processing interference.
Solution Approach 2:
The patent applies preliminary action through change tracking and modification timestamp mechanisms that identify which resources need statistics collection before the actual collection process begins. This preliminary identification ensures that the collection can be completed within the allocated downtime by focusing only on necessary resources, preventing the need to extend into time-critical periods.
4Measurement precision
If statistics are collected from all resources every time, then current statistical information is maintained, but redundant data gathering occurs due to data changes during collection
Solution Approach 1:
The patent performs preliminary action by tracking changes to container resources and their subtrees before initiating statistics collection. By maintaining a record of which resources have changed since the last collection, the system can prepare a targeted list of resources to collect statistics from, ensuring currency of statistical information while avoiding redundant collection of unchanged resources and the associated resource consumption.
Solution Approach 2:
The patent applies partial action by collecting statistics only from resources that have changed, rather than all resources. This partial collection maintains statistical currency for the changes that occurred while avoiding the excessive resource consumption that would result from collecting statistics from all resources, especially those that remain unchanged.
Data Source
AI summary
Techniques are provided for efficiently collecting statistics for hierarchically-organized collections of data. A database system leverages container-level modification time stamps and stored subtree-level change information to gather statistical information from only those resources in a hierarchical collection for which the statistics have changed since the last time that statistics were gathered for the collection. A lockless data structure is also described for storing the subtree-level change information in which an identifier corresponding to each subtree in a collection containing a changed resource may be stored. This data structure may be a table that is distinct from one or more tables representing the collection. In one embodiment of the invention, the immediate parent resource of a particular modified resource may be omitted from the subtree table by leveraging modification time stamps while gathering statistics based on tracked subtree-level information.