Similarity Calculation Using Reduced-Dimensional Vector Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing similarity calculation systems face inefficiencies when dealing with a large number of dimensions in target vectors, leading to prolonged clustering times and increased computational burden.
Innovation Solution
The system generates second target vectors with fewer dimensions through processing, clusters these vectors, and calculates similarity between first target vectors within selected clusters, using a similarity calculation method that includes generating a reduced query vector and inverted indices for efficient similarity determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If clustering is performed on high-dimensional target vectors, then similarity calculation accuracy is improved, but clustering time increases significantly
Solution Approach 1:
The patent segments the target vectors into two types: first target vectors (original high-dimensional vectors) and second target vectors (reduced-dimensional vectors obtained through predetermined processing). The second target vectors are used specifically for clustering operations, while the first target vectors are preserved for accurate similarity calculation. This segmentation allows clustering to be performed on lower-dimensional data, reducing clustering time while maintaining the ability to calculate accurate similarities using the original high-dimensional vectors.
2Productivity
If the number of dimensions of target vectors is reduced, then clustering speed is improved, but similarity calculation accuracy may deteriorate
Solution Approach 1:
The patent maintains both high-dimensional first target vectors and reduced-dimensional second target vectors separately. The second target vectors are used for clustering to improve speed, while the first target vectors are retained and used for the final similarity calculation to ensure accuracy. This dual-vector approach allows the system to benefit from both fast clustering and accurate similarity measurement without compromising either aspect.
Data Source
AI summary
To calculate quickly a similarity to a query vector even when the number of dimensions of a target vector is large. The similarity calculation system is configured to: acquire a plurality of first target vectors; generate a plurality of second target vectors having a smaller number of dimensions than the plurality of first target vectors by subjecting each of the plurality of first target vectors to predetermined processing; acquire, based on the plurality of second target vectors, a plurality of clusters each including at least one of the plurality of first target vectors and at least one of the plurality of the second target vectors, each of the at least one second target vector corresponding to one of the at least one first target vector; and calculate a similarity between each of the at least one first target vector belonging to one or more clusters selected based on a given query vector having the same number of dimensions as the at least one first target vector, and the given query vector.