Random Document Embedding for Text Feature Matrix
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning methods face high computational complexity and memory consumption when processing text data due to the need for evaluating positive definite kernels using KNN-WMD techniques, which limits their efficiency in feature embedding for tasks like document classification and clustering.
Innovation Solution
The approach generates a feature matrix for text data by transforming distance measurements between random text sequences and raw text data into a low-dimensional Euclidean space, reducing computational complexity and memory requirements through the use of random document embedding, specifically by sampling random text sequences of varying lengths from a distribution and employing Word Mover's Distance for document distance measurements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If KNN-WMD techniques are used to evaluate positive definite kernels for text data, then measurement precision is improved, but computational complexity increases to O(N^2L^3 log(L)) and memory consumption increases to O(NL+N)
Solution Approach 1:
The patent transforms the kernel evaluation problem from O(N^2L^3 log(L)) complexity to O(NRLD^2L log(L)) by changing parameters: using random document embeddings (R random documents of maximum length D) instead of full KNN-WMD computation. This parameter transformation maintains measurement precision while reducing computational burden through dimensionality reduction and randomization techniques.
Solution Approach 2:
The patent creates simplified copies of the original text data by generating random document embeddings that capture essential features without requiring full computation. These random embeddings serve as approximate representations that preserve distance measurement accuracy while reducing computational complexity from O(N^2L^3 log(L)) to O(NRLD^2L log(L)).
2Measurement precision
If KNN-WMD techniques are used to evaluate positive definite kernels for text data, then measurement precision is improved, but memory consumption increases to O(NL+N)
Solution Approach 1:
The patent reduces memory consumption from O(NL+N) to O(NR) by changing the representation parameters. Instead of storing and processing full text sequences of length L for N documents, the method uses random document embeddings with maximum length D and R random documents, significantly reducing the memory footprint while preserving measurement precision through the kernel approximation.
Solution Approach 2:
The patent creates compact copies of text data in the form of random document embeddings. These embeddings capture the essential semantic information needed for distance measurement while requiring only O(NR) memory instead of O(NL+N), making the system scalable to larger datasets.
3Productivity
If random document embedding is used to reduce computational complexity, then productivity is improved, but device complexity increases due to the need for generating and managing random text sequences
Solution Approach 1:
The patent performs preliminary actions by pre-generating R random documents and their embeddings before the main processing task. This preliminary setup, which includes sampling random words from the vocabulary according to a distribution and creating random text sequences of varying lengths, enables the subsequent kernel evaluation to run efficiently in O(NRLD^2L log(L)) time without repeating the random generation process.
Solution Approach 2:
The patent introduces random document embeddings as an intermediary between the original text data and the kernel evaluation process. These random embeddings serve as a mediator that simplifies the computation: instead of directly comparing all document pairs (O(N^2) complexity), the system computes distances between documents and random embeddings (O(NR) complexity), then uses these intermediate results to approximate the kernel values.
Data Source
AI summary
Embodiments of the present invention provide a computer-implemented method for performing unsupervised feature representation learning for text data. The method generates reference text data having a set of random text sequences, in which each text sequence of set of random text sequences is of a random length and comprises a number of random words, and in which each random length is sampled from a minimum length to a maximum length. The random words of each text sequence in the set are drawn from a distribution. The method generates a feature matrix for raw text data based at least in part on a set of computed distances between the set of random text sequences and the raw text data. The method provides the feature matrix as an input to one or more machine learning models.