Document Similarity Index Using Segment Concentration Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document similarity determination systems cannot evaluate whether parts with high similarity are concentrated in a certain paragraph or dispersed throughout the entire document.
Innovation Solution
A document similarity evaluation system that includes a segment search unit to find common segments and identify their appearance range, and a similarity index calculation unit using the equation similarity index=F(NTC)/G(NCC)×NS, where NTC and NCC are sums of characters in the appearance range and common segments, and F and G are monotonically increasing functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of common segments is used as the similarity metric, then the evaluation of overall document similarity is achieved, but the concentration or dispersion of similar parts cannot be determined
Solution Approach 1:
The patent transitions from one-dimensional segment matching to two-dimensional evaluation by introducing the appearance range (spatial position) as an additional dimension. The similarity index incorporates both the number of common segments and their spatial distribution characteristics, enabling assessment of whether similar content is concentrated or dispersed throughout the document.
Solution Approach 2:
The patent applies local quality analysis by examining the appearance range of common segments within the document structure. Instead of treating all segments uniformly, it evaluates the local concentration or dispersion of similar content at different positions, providing nuanced insight into the spatial distribution of similarity.
2Productivity
If only the number of common segments is counted, then the calculation is simple, but the nuanced evaluation of similarity concentration is lost
Solution Approach 1:
The patent segments the document into identifiable units (segments) and tracks their appearance ranges. By dividing the document structure into manageable segments and analyzing their spatial distribution, the system achieves both computational efficiency and precise measurement of similarity concentration patterns.
Solution Approach 2:
The patent introduces a composite similarity index that changes parameters beyond simple segment counting. The index incorporates the appearance range and spatial distribution characteristics of common segments, transforming the evaluation from a single-parameter count to a multi-parameter assessment that captures concentration nuances.
Data Source
AI summary
Disclosed is a document similarity evaluation system or the like which can evaluate a degree of concentration and dispersion of parts with high similarity in at least two kinds of documents. The system includes a segment search unit which finds common segments (CS) in first and second segment strings, counts the number of CS, and identifies an appearance range (AR) within CS; and a similarity index (SI) calculation unit which calculates a first sum that is a sum of the numbers of characters of each segment (NCS) in AR and a second sum that is a sum of NCS of CS and calculates SI between the first and second segment strings by the following equation, SI=F(NTC)/G(NCC)×NS (where, NTC is the first sum, NCC is the second sum, NS is the number of the CS, functions F and G monotonically increase at larger than 0).