Data Protection System for Structured and Unstructured Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data protection methods, such as file transformation applications and APIs, are inadequate for encrypting and tokenizing sensitive data stored in files or streamed as structured and unstructured data, particularly failing to support multiple field processing and disparate file formats, and often require complex batch processing.
Innovation Solution
A system comprising a configuration engine, cryptography engine, and generation engine that processes structured and unstructured data within a common framework, using extensible APIs for particular data type encryption and tokenization, supporting batch and multi-threaded processing of streamed data, including format-preserving encryption and tokenization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing file transformation applications and APIs are used for data protection, then data encryption and tokenization can be performed within existing applications, but they cannot process multiple field processing and multiple disparate file formats, and require complex batch processing
Solution Approach 1:
The patent implements a universal data protection system that can handle multiple file formats (structured and unstructured data) and multiple fields within a single framework. The system uses format-preserving encryption that adapts to different data types including CSV, JSON, XML, and other formats without requiring separate processing mechanisms for each format type.
Solution Approach 2:
The patent segments the data processing into record-level operations that can be applied independently to each record in a stream or file. This allows multiple field processing where different encryption rules can be applied to different fields within the same record, and enables processing without complex batch operations by handling data in manageable record-sized units.
2Productivity
If batch processing is used to achieve high performance in data protection, then processing speed improves, but the complexity of the system increases
Solution Approach 1:
The patent enables continuous processing of data streams and files through record-level operations that can be performed sequentially without requiring complex batch processing coordination. The system maintains high productivity by processing records continuously as they are read, applying encryption transformations immediately, and writing encrypted records without interruption or complex batching overhead.
3Reliability
If data is protected using traditional encryption methods, then sensitive data is secured, but the data cannot be processed efficiently in stream format or multiple file formats
Solution Approach 1:
The patent employs format-preserving encryption that changes the cryptographic parameters while maintaining the original data format characteristics. This allows encrypted data to remain in the same format as the plaintext (preserving field structures, data types, and lengths), enabling the encrypted data to continue being processed efficiently in stream format and supporting multiple file formats without requiring format conversion or specialized processing pipelines.
Data Source
AI summary
Example implementations relate to protecting data of a particular data type. For example, a system for protecting data of a particular type may include a configuration engine to receive a stream of data, where a portion of the stream of data includes data of a particular data type. The configuration engine may determine the particular data type to be protected based, at least in part, on a format associated with the stream of data. Further, the system for protecting data of a particular type may include a cryptography engine to protect the data of the particular data type, and a generation engine to output the stream of data such that at least the data of the particular data type is protected.