Word Segmentation Using K-Means Clustering on Spacing Profiles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing word segmentation methods are not fast and accurate, particularly in handling short lines and single word lines, which often result in poor segmentation due to unclear bimodal spacing length distributions and the need for heuristic thresholds.
Innovation Solution
The method employs a k-means algorithm with k-means++ initialization to classify spacing segments into character and word spacings, using a compactness measure to prune results and exclude single word lines, thereby dynamically distinguishing between word and character spacings without requiring heuristic thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hierarchical clustering method is applied to spacing segments, then word segmentation can be performed, but the method is slow and requires heuristic thresholds
Solution Approach 1:
The patent changes the clustering parameter from hierarchical clustering to k-means clustering with k=2, which significantly improves computational speed while maintaining segmentation accuracy. The k-means algorithm with k-means++ initialization provides faster convergence and avoids the need for complex hierarchical clustering steps.
Solution Approach 2:
The patent replaces the mechanical hierarchical clustering process with a more efficient k-means clustering mechanism. This substitution eliminates the need for iterative hierarchical merging and splitting operations, achieving both speed improvement and reduced dependency on heuristic thresholds.
2Productivity
If heuristic thresholds are used for spacing classification, then word segmentation can be achieved, but accuracy deteriorates on short lines and single word lines
Solution Approach 1:
The patent enables the system to automatically determine spacing categories through k-means clustering without requiring external heuristic thresholds. The algorithm self-adjusts to the specific characteristics of each text line, including short lines and single word lines, by analyzing the actual spacing distribution in each line.
Solution Approach 2:
The patent introduces dynamic adaptation where the clustering results are line-specific rather than using fixed global thresholds. Each text line is processed independently with k-means clustering that adapts to the line's specific spacing characteristics, improving accuracy for edge cases like short lines and single word lines.
3Productivity
If k-means algorithm with k-means++ initialization is applied, then segmentation speed improves, but complexity of algorithm increases
Solution Approach 1:
The patent optimizes the k-means algorithm by setting k=2 specifically for distinguishing word and character spacing, which simplifies the clustering process compared to hierarchical clustering. The k-means++ initialization parameter is used to achieve fast convergence without requiring complex iterative processes.
Data Source
AI summary
A word segmentation method for processing a document image applies clustering analysis to the spacing segments of a line. The spacing segments are generated by thresholding a one-dimensional vertical projection profile of the line. Taking advantage of the bimodal distribution of spacing length distribution of text lines, a k-means clustering algorithm is used, with the number of clusters pre-set to two, to classify the spacing segments as either character spacing or word spacing. Moreover, k-means++ initialization is used to enhance performance of cluster analysis. The clustering result such as cluster centers and compactness is used to prune single-word text line, single table item, etc. The locations of the word spacing segments are then used to segment the line of text into words.