Character Code Search Index Using Split Bigram Bit Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional character component tables face challenges in managing large file sizes due to numerous character codes, leading to resource constraints and increased search noise when reduced, and prolonged processing times for generating these tables.
Innovation Solution
A method involving the generation of combined identification information for consecutive character codes, using a 2n-branching Huffman tree to compress and expand character codes efficiently, reducing file size while maintaining search speed by dividing character codes into upper and lower bits and allocating bit strings accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the character component table is increased by adding tables for two sequential characters, three sequential characters, four sequential characters to reduce search noise, then search precision is improved, but file size drastically increases causing resource constraints
Solution Approach 1:
The patent segments the character code representation by dividing 16-bit character codes into two 8-bit halves (upper and lower bits). Instead of creating separate character component tables for two-character, three-character, and four-character sequences which would cause file size explosion, the invention segments each character code into manageable 8-bit portions and uses bit string operations to represent character sequences compactly. This segmentation allows the system to maintain search precision for multi-character sequences while keeping the file size manageable through efficient bit-level representation.
2Quantity of substance
If the file size is reduced using a hash function to manage resource constraints, then file size is reduced, but search noise increases and search speed is slowed
Solution Approach 1:
The patent replaces the conventional hash function approach with a bit string manipulation system based on Huffman coding. Instead of using hash functions that scatter character code information and require additional processing to resolve collisions, the invention uses direct bit string operations on segmented character codes. This substitution eliminates hash function overhead, reduces search noise by maintaining exact character code relationships, and improves search speed through efficient bit-level parallel operations while keeping file size compact.
3Measurement precision
If character component tables for two sequential characters, three sequential characters, four sequential characters are generated to reduce search noise, then search precision is improved, but processing time to create these tables increases
Solution Approach 1:
The patent performs preliminary segmentation of character codes into upper and lower 8-bit portions during the table generation phase. By pre-dividing all character codes in the character component table into these segmented forms, the system eliminates the need to generate separate tables for two-character, three-character, and four-character sequences. The segmented representation allows all multi-character sequence searches to be performed using the same base table structure, significantly reducing the processing time required for table creation while maintaining the ability to perform precise multi-character searches through bit string operations.
Data Source
AI summary
A recording medium stores an information processing program that causes a computer having a storage unit storing therein a file group in which character code strings are described, to execute generating combined identification information by dividing into two portions, at least one among identification information for a preceding character code and identification information for a succeeding character code and respectively combining the portions with the identification information that is not divided, the preceding and succeeding character codes constituting a character code string for two-consecutive grams in a file among the file group; storing to the storage unit, various consecutive-gram divided maps obtained by allocating to each type of combined identification information generated, a string of bits corresponding to the quantity of files in the file group; and updating in the consecutive-gram divided maps, a bit indicating whether the character code string for the two-consecutive grams is present in the file.