Multi-Dimensional Text Similarity Recognition Beyond Full-Text Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text similarity recognition methods based on vector representation only consider full-text semantics, leading to lower accuracy in determining text similarity.
Innovation Solution
A method for text similarity recognition that involves obtaining multi-dimensional similarity features from words, sentences, and full-text dimensions, including features such as word and sentence counts, common words, and character string distributions, to determine the similarity between texts without relying on vector representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If vector representation method is used for text similarity recognition, then the process is simplified, but the accuracy decreases because only full-text semantics are considered
Solution Approach 1:
The patent segments text analysis into three distinct dimensional features: word-level features (word count, common words), sentence-level features (sentence count, common sentences), and full-text features (character string distribution, TF-IDF). This segmentation allows comprehensive analysis at multiple granularities rather than treating text as a single vector, thereby improving accuracy while maintaining systematic processing.
Solution Approach 2:
The patent transitions from single-dimensional vector representation to multi-dimensional feature space by extracting features across word, sentence, and full-text dimensions. Each dimension provides independent similarity metrics that are aggregated to produce the final similarity score, effectively adding dimensional depth to the analysis and capturing nuances that single-vector methods miss.
2Measurement precision
If multi-dimensional similarity features are extracted from words, sentences, and full-text, then the accuracy of text similarity recognition is improved, but the computational complexity increases
Solution Approach 1:
The complexity is managed by segmenting the feature extraction process into three independent modules: word-level feature extraction (counting words, identifying common words), sentence-level feature extraction (counting sentences, identifying common sentences), and full-text feature extraction (character string distribution, TF-IDF calculation). Each module operates independently and can be implemented using standard text processing techniques, making the overall complex system manageable and modular.
3Loss of information
If multiple dimensional features are analyzed, then detailed insights into text similarities are provided, but the processing time increases
Solution Approach 1:
The patent performs preliminary processing by first extracting word-level features (which are computationally cheapest), then sentence-level features, and finally full-text features. This hierarchical preliminary action allows the system to quickly eliminate obviously dissimilar texts at the word level before investing more computational resources in detailed analysis, thereby reducing average processing time while maintaining complete information analysis for texts that require it.
Data Source
AI summary
A method for text similarity recognition method includes: obtaining a first text and a second text; determining a multi-dimensional similarity feature for the first text and the second text, where the multi-dimensional similarity feature includes at least one of a word dimensional similarity feature, a sentence dimensional similarity feature, or a full-text dimensional similarity feature; and determining a recognition result indicating similarity between the first text and the second text based on the multi-dimensional similarity feature.