Compression Encoder Buffering Short Mismatch Strings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods, such as dictionary type compression, are inefficient for short data sequences, leading to increased data volume and storage costs due to the need to output mismatched characters as they are, which can exceed the efficiency of longer matches.
Innovation Solution
A method that buffers mismatch character strings of a certain length and outputs their index if they match existing strings, improving compression efficiency by encoding mismatch information and index, rather than the characters themselves, when the mismatch is shorter than the minimum match length.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dictionary type compression is used with minimum match length of 3-4 characters, then compression efficiency is improved for long matches, but short mismatch character strings (2 characters or less) cannot be compressed and must be output as they are
Solution Approach 1:
The patent segments the compression process into two distinct paths: one for matches of minimum length or more (using dictionary compression) and another for mismatches shorter than minimum length (using a separate buffer and index-based encoding). This segmentation allows each path to be optimized independently, resolving the contradiction by enabling compression of short strings without affecting the efficiency of long match compression.
Solution Approach 2:
The patent introduces an intermediary buffer that stores mismatch character strings shorter than the minimum match length. This buffer acts as a mediator between the dictionary compression process and the final output, allowing short mismatch strings to be encoded efficiently via index references rather than being output in their original form. The buffer size is carefully controlled to balance compression efficiency with memory usage.
2Ease of manufacture
If mismatch character strings are output as they are, then implementation simplicity is maintained, but data volume increases and storage capacity is reduced
Solution Approach 1:
The patent extracts the handling of short mismatch character strings from the main dictionary compression flow and processes them through a separate buffer mechanism. By taking out this specific case, the patent can apply a specialized compression technique (index-based encoding) without complicating the overall dictionary compression implementation. The extraction allows for targeted optimization of short string handling while preserving the simplicity of the main compression algorithm.
3Productivity
If buffer size for mismatch strings is increased, then compression efficiency for short strings is improved, but memory usage increases and storage capacity is reduced
Solution Approach 1:
The patent dynamically adjusts the buffer size parameter based on the specific compression scenario and data characteristics. Rather than using a fixed large buffer that would consume excessive memory, the buffer size is optimized to hold only the necessary mismatch strings for effective compression. This parameter adjustment resolves the contradiction by achieving sufficient compression efficiency with minimal memory footprint.
Data Source
AI summary
According to one embodiment, a method includes identifying an input string of less than the predetermined length as a mismatch character string, determining whether the mismatch character string matches the character strings in a buffer, storing, as to the matched mismatch character string, a first data set including flag information indicating a match and index information indicating the entry of the buffer in which the mismatch character string determined to be a match is stored, in a nonvolatile memory, and storing the unmatched mismatch character string in the buffer, and storing, as to the unmatched mismatch character string, a second data set including flag information indicating the mismatch and the mismatch character strings, in the nonvolatile memory.