Text Encoding With Unified Dictionaries and Huffman Code Lengths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional encoding techniques fail to perform entropy encoding based on the occurrence frequency of character strings in text data, as they use static and dynamic codes with predetermined lengths and require an identification bit to differentiate between dictionaries, disrupting the frequency-based encoding.
Innovation Solution
The encoding program generates static-encoded data using a static dictionary and dynamic-encoded data using a dynamic dictionary, then creates a Huffman tree based on the occurrence frequencies of both types of codes, eliminating the need for identification bits by assigning code lengths according to frequency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an identification bit is added to static codes and dynamic codes to differentiate between dictionaries, then the decoding process can correctly identify which dictionary to use, but the code length deviates from the optimal entropy encoding length based on occurrence frequency
Solution Approach 1:
The patent merges the static dictionary and dynamic dictionary into a single unified dictionary structure. Both static codes (from frequent words) and dynamic codes (from repeated character strings) are stored together with their respective occurrence frequencies, eliminating the need for identification bits and enabling unified entropy encoding based on actual frequencies.
Solution Approach 2:
The patent changes the parameter of code representation by storing not just the code itself but also its occurrence frequency in the unified dictionary. This allows the encoding system to use the frequency parameter directly for entropy encoding, optimizing code length according to actual usage rather than using fixed or predetermined lengths.
2Productivity
If a static dictionary with predetermined code lengths is used for encoding, then the encoding process is simple and fast, but entropy encoding according to occurrence frequency cannot be performed
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing the occurrence frequencies of all codes (both static and dynamic) in the unified dictionary before the actual encoding process. This preparation allows the encoding step to directly use these pre-computed frequencies for entropy encoding, maintaining speed while achieving optimal compression.
Solution Approach 2:
The unified dictionary serves as an intermediary structure that bridges the simplicity of static encoding and the efficiency of entropy encoding. It stores both the code mappings and their frequencies, allowing the system to use a simple lookup process while simultaneously enabling frequency-based optimization through the stored frequency data.
3Adaptability or versatility
If a dynamic dictionary with predetermined code lengths is used, then repeated character strings can be encoded, but the code length does not reflect the occurrence frequency of the character strings
Solution Approach 1:
The patent combines the dynamic dictionary functionality with frequency tracking into the unified dictionary structure. When character strings are dynamically encoded, their occurrence frequencies are simultaneously recorded and stored alongside the code mappings, enabling both dynamic adaptation and frequency-based optimization in the same structure.
Data Source
AI summary
An encoding device generates static-encoded data from input text data, utilizing a static dictionary that associates a character strings and static codes, respectively the static-encoded data including a plurality of static codes corresponding to a plurality of character strings that are registered in the static dictionary, generates dynamic-encoded data from the static-encoded data by encoding a character string or the static code that occurs more than once in the static-encodes data into a dynamic code, creates a dynamic dictionary associating character strings or static codes with corresponding dynamic codes, respectively and creates a Huffman tree and data of a leaf in the Huffman tree based on an occurrence frequency of the dynamic codes and the static codes in the input text data.