Semantic Data Compression for Enterprise Network Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression and transmission methods fail to efficiently manage data transfer across disparate enterprise systems, particularly in scenarios with limited or unreliable network bandwidth, as they do not consider the content or corpus of the data, leading to increased bandwidth usage and processing workload.
Innovation Solution
The implementation of network-aware semantic data compression and transmission systems using advanced Hadoop-based Natural Language Processing (NLP) algorithms and analytics to prioritize and compress data based on relationships between textual elements and artifacts, reducing the size of artifacts during transfer by replacing unnecessary content with identifiers and optimizing batch sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data compression methods are used, then data transmission is performed, but bandwidth usage is excessive and processing workload increases
Solution Approach 1:
The patent extracts and removes redundant information from data artifacts before transmission by performing semantic analysis to identify and eliminate unnecessary content, keeping only essential information that maintains data meaning while reducing volume
Solution Approach 2:
The system changes the parameter of data representation from raw binary format to semantically compressed format using identifiers and references, transforming how data is encoded to achieve higher compression ratios without losing information
2Quantity of substance
If source deduplication is implemented, then bandwidth and storage are reduced, but server workload increases
Solution Approach 1:
The patent performs preliminary semantic analysis and compression of data at the source before transmission, preparing optimized data packages in advance that reduce the need for complex real-time processing during data transfer operations
3Volume of stationary object
If target deduplication is used, then storage space is reduced, but data transmission volume remains high
Solution Approach 1:
The system performs preliminary compression and optimization of data at the source before transmission, so that data arrives at the target in a already-compressed state, eliminating the need to transmit full-volume data for subsequent deduplication processing
4Quantity of substance
If semantic analysis is performed on all data, then compression efficiency improves, but processing time increases
Solution Approach 1:
The patent applies semantic analysis selectively to portions of data that will benefit most from compression, using heuristics to identify high-value compression targets rather than uniformly processing all data, achieving good compression ratios with reduced overall processing time
Data Source
AI summary
Computer implemented methods and systems directed to a technological improvement in electronic data compression and transmission between two computer systems using semantic analysis are disclosed. The method includes the step of compressing, at a first computer, a plurality of queued artifacts based on one or more network decision variables. The compression includes prioritizing the queued artifacts. The compression further includes determining a first set of artifacts in a set of queued artifacts to transmit and a second set of artifacts in a set of queued artifacts to only send links. The compression further includes replacing unnecessary content in the set of queued artifacts with one or more identifiers. The method further includes the step of transmitting, from the first computer, one or more batches of the compressed data over a network to a second computer.