Distributed Visualization Processing with Mixer and Shard Servers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The sheer scale of television reporting data poses challenges in visualization processing, requiring extensive data transfer and processing that is time-consuming and bandwidth-intensive, often exceeding the capabilities of standard machines, leading to inefficiencies and limitations in generating visual representations of consumer behavior.
Innovation Solution
A distributed processing system utilizing a mixer server and shard servers, where requests for visualizations are translated into sharded requests, processed in parallel by shard servers, and aggregated by the mixer server, reducing the need for transferring large datasets and allowing for fault-tolerant and scalable visualization generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single processing device processes all television reporting data to generate visualizations, then the visualization can be generated, but the time to generate the visualization becomes unacceptably long
Solution Approach 1:
The patent divides the television reporting data into multiple shards and distributes them across multiple processing devices (shard servers). Each shard server processes a specific subset of data independently, and the mixer server aggregates the results. This segmentation enables parallel processing, significantly reducing the time required to generate visualizations compared to single-device processing of the entire dataset.
2Productivity
If large amounts of data are transferred to a single machine for processing, then the visualization can be generated, but substantial bandwidth is required
Solution Approach 1:
The patent segments the data into smaller shards that are distributed across multiple shard servers. Instead of transferring the entire large dataset to a single machine, only the necessary shard data is transferred to each processing device. This reduces the bandwidth requirement for data transfer while maintaining the capability to generate comprehensive visualizations through aggregation at the mixer server.
3Productivity
If a single machine processes vast amounts of data, then the visualization can be generated, but the machine requires substantial memory capacity
Solution Approach 1:
The patent distributes the data across multiple shard servers, each handling a specific subset of data. This segmentation allows each individual machine to process only a portion of the total data, significantly reducing the memory capacity requirement per machine. The system achieves high overall processing capability through parallel processing across multiple machines rather than requiring a single machine with enormous memory capacity.
4Ease of manufacture
If the data is held in a data store and transferred to another machine for processing, then the data can be processed, but the transfer time increases
Solution Approach 1:
The patent segments the data into smaller shards that can be transferred in parallel to multiple shard servers simultaneously. This reduces the total transfer time compared to transferring the entire dataset to a single machine. The mixer server coordinates the transfer and aggregation process, enabling efficient data movement and processing.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for analyzing data. In one aspect, a mixer server receives a request for a visualization of television reporting data, translates the request for the visualization of the television reporting data into sharded requests, and provides each sharded request to a respective shard server. Each shard server processes a respective proper subset of the television reporting data to generate data representing the visualization, and provides the data representing the visualization to the mixer server. The mixer server aggregates the data representing the visualizations received from each of the shard servers, and provides the aggregated data representing the visualization as a response to the request for the visualization of the television reporting data.