Tensor Dataset Visualization via Dynamic Batch Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large and complex training datasets pose challenges in communication, storage, processing, and visualization, lacking effective tools for engineers and data scientists to understand and visualize them.
Innovation Solution
A method and system for visualizing tensor datasets by identifying display regions, selecting subsets of tensors based on zoom levels, and dynamically fetching and caching data to reduce storage and network resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the entire large training dataset is loaded and displayed, then complete data visualization is achieved, but storage requirements and network bandwidth consumption increase significantly
Solution Approach 1:
The patent segments the large training dataset into multiple batches, where each batch contains a subset of samples. Instead of loading the entire dataset at once, the system loads and displays one batch at a time, allowing complete data visualization over time while minimizing instantaneous storage requirements and network bandwidth consumption.
Solution Approach 2:
The patent implements dynamic batch loading where the system automatically loads new batches of data as users navigate through the dataset. This dynamic approach allows the visualization to cover the entire dataset while maintaining low storage requirements by only having active batches in memory at any given time.
2Loss of information
If all tensor types for each sample are displayed, then comprehensive data analysis is enabled, but processing complexity and display time increase
Solution Approach 1:
The patent segments the tensor data into different tensor types (e.g., input tensors, output tensors, intermediate tensors). The system allows users to select and display specific tensor types based on their analysis needs, rather than displaying all tensor types simultaneously. This segmentation enables comprehensive data analysis while reducing processing complexity by focusing on relevant tensor subsets.
Solution Approach 2:
The patent applies local quality by allowing different display configurations for different regions of the interface. Users can specify which tensor types to display in different areas or at different times, optimizing the display for specific analysis tasks rather than attempting to display all information uniformly.
3Measurement precision
If high-resolution display of all samples is provided, then visualization quality is improved, but network bandwidth consumption and processing time increase
Solution Approach 1:
The patent implements partial action by loading and displaying only the necessary portion of the dataset at any given time (one batch at a time). This approach provides sufficient visualization quality for analysis purposes while avoiding the excessive processing time and network bandwidth consumption that would result from loading and rendering all samples at high resolution simultaneously.
Data Source
AI summary
A method can include identifying, by one or more processors, filtering criteria for displaying a multi-dimensional sample dataset within a display region. The method can include selecting, by the one or more processors, based on the filtering criteria, a subset of samples from the multi-dimensional sample dataset, each sample of the subset of samples associated with a respective set of tensors that are to be displayed within the display region. The method can include mapping, by the one or more processors, display data associated with the respective set of tensors of each sample of the subset of samples to respective display locations for display in the display region. The method can include presenting, by the one or more processors, the display data associated with the respective set of tensors in the display region according to the mapping.