Encoded Dataset Mapping for Decode-Free ML Data Manipulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data encoding methods require decoding and subsequent reencoding, which is computationally expensive and inefficient.
Innovation Solution
A system and method that processes encoded data without decoding, using a dictionary file and mappings to manipulate encoded data files, allowing operations like compression, encryption, and computations directly on the encoded data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is encoded and then decoded for manipulation, then the data can be accessed and manipulated, but the computational cost and time increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-processing the data file into an encoded format with segment mappings before any manipulation operations are needed. The data is divided into segments and mapped to encoded representations in advance, so that when manipulation is required, the system can work directly with the pre-encoded segments without needing to decode first. This eliminates the time-consuming decode-manipulate-encode cycle while maintaining full manipulation capability.
2Ease of operation
If data is decoded for manipulation, then the data can be accessed and modified, but the computational overhead increases
Solution Approach 1:
The system performs preliminary encoding and segment mapping before any data access or manipulation operations. By pre-processing the data into encoded segments with stored mappings, the system enables direct manipulation of encoded data without the energy-intensive decode-manipulate-encode cycle, significantly reducing computational energy consumption while maintaining full data accessibility.
Solution Approach 2:
The patent extracts only the necessary information needed for manipulation by creating segment mappings that reference encoded data portions. Instead of decoding the entire data file for access, the system extracts and stores mapping information that allows selective access to specific encoded segments, reducing the computational energy required for data operations.
3Quantity of substance
If traditional encoding methods are used, then data can be compressed or encrypted, but subsequent manipulation requires full decoding and reencoding
Solution Approach 1:
The patent divides the data file into multiple segments and creates separate encoded representations for each segment, storing mapping information that links segments to their encoded forms. This segmentation allows the system to manipulate individual encoded segments independently without decoding the entire data file, reducing processing complexity while maintaining compression or encryption benefits.
Solution Approach 2:
The system performs preliminary segmentation and encoding of data into distinct segments with stored mappings before any manipulation operations. This pre-processing creates a structure where encoded segments can be manipulated directly, eliminating the need for full decode-reencode cycles and reducing processing complexity while preserving data size reduction from compression or encryption.
Data Source
AI summary
A computer-implemented method, computer program product and computing system for: processing an unencoded data file to identify a plurality of file segments, wherein the unencoded data file is a dataset for use with an ML process; mapping each of the plurality of file segments to a portion of a dictionary file to generate a plurality of mappings that each include a starting location and a length, thus generating a related encoded data file based, at least in part, upon the plurality of mappings; receiving a request to manipulate the unencoded data file from the ML process; and processing the related encoded data file based, at least in part, upon the plurality of mappings and the dictionary file to generate a modified encoded data file that represents the requested manipulations of the unencoded data file.