Semantic Topic Identification via Vector Clustering and Frequency Thresholding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies lack efficient methods for automatically determining the semantic topic of unstructured textual content without requiring manual user input or the use of resource-intensive neural networks.
Innovation Solution
A computer system preprocesses textual content by tokenizing, lemmatizing, and filtering it, then generates data vectors that are clustered based on similarities, with the system identifying a semantic topic by selecting the most frequent word within each cluster, adjusting based on a threshold value in a training data set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual user input or review is used to determine semantic topics, then accuracy is improved, but productivity deteriorates
Solution Approach 1:
The system performs automatic text preprocessing, clustering, and semantic topic identification without requiring manual user input or review. The computer system serves itself by autonomously processing unstructured textual content through algorithmic operations including tokenization, lemmatization, filtering, and frequency-based topic selection, thereby achieving both high productivity and acceptable accuracy without human intervention
Solution Approach 2:
The patent replaces the mechanical process of manual text review with an automated computational system. Instead of human users manually analyzing and categorizing text, the system uses algorithmic operations including data vector generation, clustering algorithms, and frequency analysis to automatically determine semantic topics, substituting human cognitive work with automated information processing
2Measurement precision
If computerized neural networks are used to interpret text, then accuracy is improved, but use of energy deteriorates
Solution Approach 1:
The system uses lightweight, computationally inexpensive processing operations instead of resource-intensive neural networks. The approach employs simple algorithmic operations including tokenization, lemmatization, filtering, data vector generation, and frequency analysis that consume minimal computational resources and energy, providing an efficient alternative to heavy machine learning models
Solution Approach 2:
The patent extracts and uses only the essential computational operations needed for semantic topic identification, removing unnecessary complexity. By focusing on core operations like frequency analysis and clustering rather than deploying full neural network architectures, the system achieves text interpretation with significantly reduced computational resource requirements and energy consumption
Data Source
AI summary
In an example method, a system accesses a plurality of data vectors representing a plurality of text segments; clusters the plurality of data vectors into one or more clusters; determines semantic topic of each of the one or more clusters; generates a data structure representing the semantic topic of each of the one or more clusters; and stores the data structure. Determining the semantic topic of a cluster includes: parsing fields of the data vectors of the cluster, determining a first word representing the cluster, determining a first value representing a frequency of the first word in a training data set, comparing the first value to a threshold value. Responsive to determining that the first value is less than the threshold value, the first word is identified as a semantic topic of the cluster.


