Data Formatting System Using Segmented Objects and Notifications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Formatting data by multiple data formatters in series is time-consuming, while parallel access to the data set often leads to inefficient use of memory, storage, and processing resources, potentially exceeding resource limitations.
Innovation Solution
A system comprising a data populator, multiple data formatters, and a notification manager that creates data objects in memory, relates them in a hierarchical structure, and provides notifications for formatting, allowing efficient access and formatting of data by multiple formatters without overwhelming resource constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data formatters access the data set in series, then resource usage is reduced, but formatting time increases
Solution Approach 1:
The data set is divided into multiple data objects that can be independently accessed and formatted. Each data formatter works on different data objects simultaneously, enabling parallel processing while maintaining manageable resource consumption through segmented data handling.
Solution Approach 2:
The system dynamically manages data formatter access to data objects based on resource availability and formatting requirements. The notification manager coordinates access patterns to optimize both parallel processing efficiency and resource utilization, adjusting the degree of parallelism dynamically.
2Productivity
If data formatters access the data set in parallel, then formatting speed increases, but memory and processing resources are used inefficiently
Solution Approach 1:
The data set is segmented into discrete data objects that can be independently formatted. This segmentation allows multiple data formatters to work in parallel on different objects without requiring excessive memory resources, as each formatter only needs to load and process its assigned data objects.
Solution Approach 2:
Each data formatter is configured with specific formatting capabilities and accesses only the data objects it is designed to format. This localized approach ensures that processing resources are used efficiently, as each formatter operates on a subset of data rather than all data simultaneously.
3Productivity
If the data set is replicated for parallel access, then formatting speed increases, but storage resources increase
Solution Approach 1:
Instead of replicating entire data sets for parallel processing, the system creates lightweight references or pointers to data objects that can be shared among multiple formatters. This allows parallel access without duplicating the actual data, significantly reducing storage resource requirements while maintaining formatting speed.
Data Source
AI summary
In one embodiment, apparatus includes a data populator that is provided to 1) receive a plurality of events, 2) create a plurality of data objects in memory, with at least some of the data objects corresponding to logical groupings of data implied by ones of the events, 3) relate data corresponding to ones of the events to ones of the data objects, and 4) store the data in the memory. A number of data formatters 1) accesses ones of the data objects, and 2) retrieves and format the data that is related to the data objects, the data being retrieved in response to notifications received by the data formatter(s). A notification manager receives indications of events from the data populator, and provides notifications of ones of the events to one or more of the data formatters. Other embodiments are also disclosed.