Adaptive Serialized Data Slicing for Low-Bandwidth Cloud Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing platforms face inefficiencies in data transmission due to large data volumes, leading to high electricity consumption and vulnerability to cyber-attacks, with existing solutions requiring costly and risky reengineering.
Innovation Solution
A method and system for optimizing data transmission using dynamic and adaptive slicing and reduction of serialized data through a mutable and polymorphic data reduction matrix, which reduces data size without altering existing systems, using bit arrays and additional compression techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If data is transmitted in traditional serialized formats (JSON, XML, HTML), then data is human understandable and self-descriptive, but data volume is large leading to high bandwidth consumption and electricity usage
Solution Approach 1:
The patent segments serialized data into fixed-size blocks (e.g., 64KB) and processes each block independently through compression algorithms. This segmentation allows efficient parallel processing and reduces the memory footprint required for handling large datasets, thereby lowering energy consumption while maintaining data integrity through structured block management
Solution Approach 2:
The patent transforms data from human-readable formats (JSON, XML, HTML) into compact binary representations by changing the encoding parameters. This format transformation reduces data volume by 5-10 times while preserving all information through reversible encoding schemes, directly addressing the contradiction between readability and energy efficiency
2Ease of operation
If data is transmitted in traditional serialized formats, then data structures are self-descriptive and human understandable, but data volume increases leading to higher electricity consumption in data centers
Solution Approach 1:
The patent applies compression algorithms to data blocks before transmission or storage operations. By pre-compressing data into compact binary formats, the system reduces the amount of data that needs to be transmitted over networks and stored in data centers, directly lowering electricity consumption for data transmission and storage operations while maintaining the ability to decompress and interpret data when needed
3Productivity
If cloud platforms process large volumes of data, then service capabilities are maintained, but systems become vulnerable to cyber-attacks and security issues arise
Solution Approach 1:
The patent changes the data format parameter from text-based to binary representation, which reduces data volume by 5-10 times. This parameter change allows the system to process the same amount of information with fewer computational resources, reducing the attack surface and vulnerability to cyber-attacks while maintaining full data processing capacity through efficient binary data handling
4Loss of energy
If complete reengineering of cloud platforms is performed to improve efficiency, then data transmission optimization may be achieved, but implementation costs increase and risks of errors and defects arise
Solution Approach 1:
The patent introduces a modular compression layer that segments data processing into independent blocks with standardized interfaces. This segmentation allows the compression functionality to be added as a separate module without reengineering existing cloud platform components, reducing implementation complexity and risk while achieving data transmission optimization through systematic block-level compression
Data Source
AI summary
There is provided a system and a method for dynamically and adaptively slicing, reducing and bit packing a streamed serialized data sequence. A streamed serialized data sequence is received, and a reduction matrix is iteratively applied on the streamed serialized data sequence to obtain a plurality of sliced values. For a given iteration, it is determined if a corresponding given sliced value of the plurality of sliced values is present in an index. If the given sliced value is not present, the given sliced value is added by bit packing an unused id associated with the given sliced value using a first number of bits based on a current index length and using a second number of bits to encode the sliced data value to thereby obtain a given bit packed sliced value. The given bit packed sliced value is aggregated to obtain a bit packed data stream and transmitted.