Category Prediction Model Using Co-occurrence Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in accurately predicting categories of documents, information, or items responsive to user search queries, particularly due to class imbalance issues in training data and difficulties in understanding queries with limited user feedback.
Innovation Solution
A method involving a machine learning algorithm trained on a dataset of query-category pairs, utilizing co-occurrence data structures and self-attention mechanisms to calculate word and category embeddings, and minimizing loss functions to predict responsive categories for new queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional machine learning models are used for category prediction, then the system is simpler to implement, but prediction accuracy deteriorates due to class imbalance and limited query understanding
Solution Approach 1:
The patent segments the category prediction problem into two distinct components: (1) query-category co-occurrence modeling that captures direct associations between queries and categories, and (2) category-category co-occurrence modeling that captures relationships between categories themselves. This segmentation allows each component to be optimized independently, improving overall prediction accuracy while managing complexity through modular architecture.
Solution Approach 2:
The patent introduces a second dimension of analysis by incorporating category-category co-occurrence relationships in addition to query-category relationships. This transforms the prediction from a single-dimensional query-category mapping to a two-dimensional problem that also considers inter-category relationships, enabling the model to leverage contextual information from related categories to improve prediction accuracy.
2Measurement precision
If training data is used as-is without addressing class imbalance, then data processing is simpler, but prediction accuracy for underrepresented categories deteriorates
Solution Approach 1:
The patent applies preliminary action by computing category embeddings and establishing category-category co-occurrence relationships before the actual prediction process. This pre-processing step creates a rich contextual framework that helps the model generalize better to underrepresented categories, as the pre-computed relationships provide additional signals that compensate for limited training examples.
Solution Approach 2:
The patent implements feedback mechanisms through the loss function design, which incorporates both query-category prediction errors and category-category relationship errors. This dual-feedback approach guides the model to not only predict categories accurately but also maintain consistent category relationships, thereby improving performance on underrepresented categories through iterative refinement.
3Adaptability or versatility
If more training data is collected to improve coverage, then category coverage increases, but class imbalance problems worsen
Solution Approach 1:
The patent introduces category embeddings as an intermediary representation that mediates between queries and categories. These embeddings capture semantic relationships and co-occurrence patterns, serving as a bridge that allows the model to generalize from well-represented to underrepresented categories. The intermediary layer enables transfer of knowledge across categories, improving prediction accuracy even with limited data for certain categories.
Solution Approach 2:
The patent changes the parameter representation from direct query-category counting to continuous embedding vectors that capture co-occurrence statistics. This parameter transformation allows the model to represent category relationships in a continuous space where similarity and relationships can be inferred even when direct training examples are scarce, thereby maintaining accuracy while achieving broader category coverage.
Data Source
AI summary
A method for determining a category responsive to a user query is disclosed. The method includes receiving a training data set comprising a plurality of data pairs, each data pair including: (i) a query; and (ii) an associated one or more categories that are responsive to the query, wherein the one or more categories in the training data set defines a plurality of categories. The method includes training a machine learning algorithm, according to the training data set, to create a trained model, wherein training the machine learning algorithm includes: creating a first co-occurrence data structure defining co-occurrence of respective word representations of the queries with the plurality of categories, and creating a second co-occurrence data structure defining co-occurrence of respective categories in respective data pairs. The method also includes deploying the trained model to return one or more categories in response to a new query input.