Relevance Classifier for Hierarchical User Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning model training methods fail to accurately classify the relevance of training data to different levels of users within a hierarchy, leading to inefficiencies and reduced accuracy in model performance.
Innovation Solution
A computer-implemented method that uses labeled data from users to train a relevance classifier, which classifies data into groups based on user relationships and provenance, allowing for the selection of relevant training samples specific to each user's group.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If all labeled data from users is used as training data for machine learning models, then the quantity of training data increases, but the relevance of training data to specific user groups decreases leading to data pollution
Solution Approach 1:
The patent segments the user base into hierarchical groups (individual users, teams, departments, enterprises) and partitions training data accordingly. The relevance classifier divides training data into multiple relevance groups, allowing the system to select only the subset of data relevant to each user's specific group, thereby maintaining data relevance while managing quantity effectively.
Solution Approach 2:
The patent implements local quality by assigning different relevance characteristics to different portions of training data based on user group membership. Each user receives training data tailored to their specific group context (e.g., team-specific, department-specific), ensuring that the training data has high relevance quality for that local user context rather than using uniform data for all users.
2Reliability
If training data is classified into multiple user groups, then the relevance of training data to specific users improves, but the complexity of the training process increases
Solution Approach 1:
The patent applies preliminary action by pre-classifying training data into relevance groups using the relevance classifier before actual model training occurs. This pre-processing step organizes data by user group relevance in advance, so that during training, the system only needs to retrieve and use the pre-sorted relevant data subsets, reducing the computational complexity during the actual training phase.
Solution Approach 2:
The relevance classifier serves as an intermediary component between the raw training data and the machine learning model training process. It mediates by filtering and routing training data to appropriate user groups based on relevance, thereby simplifying the overall training process by handling the complexity of data selection and grouping in a dedicated intermediate layer.
3Measurement precision
If a relevance classifier is trained on labeled user data, then the accuracy of selecting relevant training samples improves, but the additional training step increases time consumption
Solution Approach 1:
The relevance classifier is trained in advance as a preliminary step, so that during actual model training and inference, the system can quickly retrieve pre-computed relevance classifications without re-evaluating data relevance. This preliminary classification cache reduces time consumption during subsequent training operations.
Solution Approach 2:
The relevance classifier serves itself by learning from labeled user data to automatically identify and classify relevant training samples without requiring manual intervention for each data point. Once trained, it autonomously performs the relevance assessment for new training data, reducing the time and resources needed for manual data selection and validation.
Data Source
AI summary
A computer-implemented method, a computer program product, and a computer system for classifying relevance of training data. A computer uses labeled data from users as training data to train a relevance classifier. A computer classifies, by the relevance classifier, the labeled data from the users into a set of groups. A computer generates, by the relevance classifier, relevant training data partitioned by the set of groups. In response to receiving a query from a user, a computer selects, from the relevant training data, relevant training samples for the user, where the relevant training samples are in one or more groups to which the user belongs. A computer selects, from relevant training samples for the user, top relevant training samples for the user. A computer uses the top relevant training samples for the user to generate a prompt of a machine learning model.