Meta-File System for Big Data Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques fail to efficiently manage and analyze Big Data due to its sheer size, volume, and complex structure, leading to difficulties in data storage, retrieval, and analysis, particularly with unstructured data, and lack the ability to capture contextual and semantic information across multiple file systems.
Innovation Solution
A computer-implemented method and system that models Big Data using meta-information and marker representations, enabling efficient data management and analysis by creating non-specific representations of Big Data datasets, allowing for quick access and analysis without the need to transfer large amounts of data, leveraging meta-data maps and Flash storage for instant access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional techniques are used to manage and analyze Big Data, then data storage and retrieval can be performed using standard methods, but the sheer size, volume, and complex structure of Big Data lead to inefficiency and inability to capture contextual and semantic information
Solution Approach 1:
The patent introduces meta-information and marker representations as intermediary layers between users and the actual Big Data. These intermediaries capture contextual and semantic information, enabling efficient data management and analysis without directly handling the entire volume of raw data. The meta-information acts as a mediator that simplifies access to complex datasets.
Solution Approach 2:
The patent creates non-specific representations (copies) of Big Data through meta-information and marker representations. These copies capture essential contextual and semantic properties of the original data without requiring the full data volume to be transferred or processed, thereby improving efficiency while managing complexity.
2Loss of information
If large amounts of data are transferred for analysis, then comprehensive data can be accessed, but data transfer time and resource consumption increase significantly
Solution Approach 1:
The patent extracts only the essential contextual and semantic information from the large datasets and stores it as meta-information and marker representations. This extraction allows users to access critical information without transferring the entire dataset, significantly reducing data transfer time while preserving important information.
Solution Approach 2:
The patent creates simplified copies (meta-information and markers) that represent the essential characteristics of the original data. These copies can be transferred and processed quickly, providing access to contextual and semantic information without the time penalty of moving the complete large dataset.
3Measurement precision
If detailed representations of Big Data are created, then accurate data modeling is achieved, but system complexity and processing overhead increase
Solution Approach 1:
The patent applies local quality by creating detailed representations only where necessary - specifically in the meta-information layer that captures contextual and semantic properties. The actual Big Data remains in its original form without requiring detailed processing everywhere, thus maintaining accuracy where needed while limiting complexity to specific areas.
Solution Approach 2:
The patent adds a new dimension of representation by introducing meta-information and marker representations that exist alongside the original data. This additional dimension captures contextual and semantic information without requiring the original data structure to be fundamentally altered, maintaining accuracy while managing complexity through dimensional separation.
Data Source
AI summary
A computer implemented method, system, and apparatus for modeling a Big Data dataset, the method comprising creating non-specific representations of the Big Data dataset by representing, as objects in a computer model, non-specific representations including metaInformation, DataSet, BigData and Properties representations, and creating non-specific representations of mapping of the Big Data by representing, as objects in a computer model, non-specific representations including User representations and marker representations, where the user representations are mapped to one or more marker representations and the marker representations are matched to one or more elements of the Big Data dataset.