Compression Dictionary Updates for Inter-Document Web Redundancy

The patent segments the compression dictionary into two distinct parts: a primary dictionary containing recent data from the sliding window, and a secondary dictionary containing historical data from previously compressed documents. This segmentation allows the system to efficiently utilize inter-document redundancies without compromising the speed of accessing recent data, thereby improving compression ratios while maintaining manageable processing complexity.

The patent performs preliminary action by pre-processing and storing historical document data in the secondary dictionary before compression occurs. This pre-computed secondary dictionary is then integrated with the primary dictionary to enhance compression performance. By preparing the secondary dictionary in advance, the system avoids the complexity of real-time inter-document analysis during compression, thus improving compression ratios without proportionally increasing processing complexity.

The patent divides the compression dictionary into a primary dictionary (for recent data) and a secondary dictionary (for historical data). This segmentation allows the system to maintain a manageable primary dictionary in memory while storing the larger secondary dictionary in less frequently accessed memory or on disk, thus improving compression performance without proportionally increasing active memory requirements.

The patent applies partial action by selectively incorporating only the most relevant portions of historical data into the secondary dictionary, rather than storing entire previous documents. This selective approach captures sufficient inter-document redundancies to improve compression performance while keeping the dictionary size and memory requirements at manageable levels.