Text Vector Clustering for Faster Database Registration and Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data registration processes for multidimensional vector analysis in artificial intelligence are burdensome due to the large load and time required to assign and accumulate vectors for words, sentences, and meaningful units, especially when dealing with a vast number of text types, making it difficult to efficiently register and search within databases.
Innovation Solution
A method involving the calculation of sentence vectors, clustering of sentences into groups based on these vectors, and associating representative vectors with cluster IDs, allowing for efficient registration and search processes by using a sentence transposition index that reduces the load on data registration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If vectors are assigned and accumulated for each word, sentence, and meaningful unit in existing data registration processes, then the database can store multidimensional vector data for AI analysis, but the registration load and time required become excessively large
Solution Approach 1:
The patent segments the data registration process by dividing text data into meaningful units (words, sentences, etc.), calculating vectors for each unit separately, and organizing them into a structured database format. This segmentation allows for systematic processing and retrieval while reducing the overall registration burden through modular organization.
Solution Approach 2:
The patent performs preliminary vector calculation and text analysis during the data registration phase, pre-processing the text data to extract meaningful units and compute their vectors before actual storage. This preliminary action reduces the computational load during query operations and improves overall system efficiency.
2Adaptability or versatility
If all text data is processed and stored in detail, then comprehensive search capability is achieved, but the time and computational resources required for registration and searching increase significantly
Solution Approach 1:
The patent extracts only the essential meaningful units (words, sentences) from the text data and calculates vectors specifically for these extracted elements. This selective extraction approach maintains comprehensive search capability by preserving key semantic information while reducing the overall data volume that requires processing and storage.
Solution Approach 2:
The patent applies different processing qualities to different parts of the text data by calculating vectors with varying levels of detail for different meaningful units. Words receive vector processing appropriate to their semantic importance, and sentences are processed based on their contextual significance, optimizing the balance between search capability and processing efficiency.
Data Source
AI summary
A processing method includes: calculating, when receiving a plurality of characters or a plurality of character strings to be registered in a database, a plurality of vectors corresponding to the plurality of characters or the plurality of character strings; classifying the plurality of vectors into a plurality of clusters according to inter-vector distances; acquiring, for each of the plurality of clusters, a representative vector which represents one or more vectors included in each cluster and a distance corresponding to a magnitude of the one or more vectors included in each cluster from the representative vector of each cluster; and storing, in a memory, dictionary information in which the representative vector and the distance are associated with cluster identification information which identifies each of the plurality of clusters.