Attribute-Based Dynamic Dictionaries for Faster Text Encoding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional encoding techniques face inefficiencies when using dynamic dictionaries, particularly when encoding XML data or similar large and varied datasets, as the dynamic code allocation based on word order leads to long processing times and inefficient dictionary usage due to the inclusion of low-frequency and unknown words.

Innovation Solution

An encoding apparatus that determines the dynamic dictionary based on attribute information of words, allocating dynamic codes efficiently by registering words in corresponding dynamic dictionaries, thereby optimizing dictionary usage and reducing processing time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If dynamic code allocation is based on word order of appearance, then all words including low-frequency and unknown words are registered in the dynamic dictionary, but this leads to long processing times and inefficient dictionary usage

Engineering Contradiction:

Improvedictionary coverageVSAvoidencoding speed

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The dynamic dictionary is divided into multiple sub-dictionaries based on word attributes (e.g., frequency ranges, word types). Instead of registering all words in a single dynamic dictionary, the system segments the dictionary space and assigns words to appropriate sub-dictionaries based on their attributes, reducing the search space and improving encoding efficiency while maintaining comprehensive coverage

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different sub-dictionaries are optimized for different word characteristics. High-frequency words are placed in one sub-dictionary with optimized structures, while low-frequency or unknown words are placed in another. This local optimization allows each sub-dictionary to be tailored for its specific word type, improving overall processing speed without sacrificing versatility

Inventive Principle:

Principle #3Local quality

2Adaptability or versatility

If a single dynamic dictionary is used for all word types, then the dictionary can handle any word, but the processing time increases due to the large size and variety of words

Engineering Contradiction:

Improveword type coverageVSAvoidprocessing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The single large dynamic dictionary is segmented into multiple smaller sub-dictionaries organized by word attributes such as frequency ranges or lexical categories. This segmentation reduces the time required to search and register words by limiting the search space to relevant sub-dictionaries only

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Words are pre-categorized by attribute before being registered in the dynamic dictionary. This preliminary classification allows the encoding system to directly access the appropriate sub-dictionary without performing exhaustive searches across all words, significantly reducing processing time while maintaining comprehensive word type coverage

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11323132B2Encoding method and encoding apparatus

Publication Date: 2022.05.03 FUJITSU LTD

AI summary

An encoding apparatus includes a memory and a processor configured to acquire text data, specify a first dynamic dictionary among a plurality of dynamic dictionaries based on attribute information of a first word included in the text data, register the first word in association with a first dynamic code in the first dynamic dictionary, and encode the first word into the first dynamic code.