Blocking-Based Batches for ML Training Hard Negatives
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning model training methods waste computational resources by excluding useful information from similar non-matches, leading to inefficient training and inaccurate differentiation between similar samples, as they focus on positive matches while neglecting hard negatives.
Innovation Solution
The approach involves constructing blocking-based batches that include positive matches, hard negatives, and easy negatives, allowing machine learning models to differentiate between similar samples by grouping data based on shared heuristics, thereby improving training efficiency and reducing resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional machine learning training methods focus only on positive matches, then the training process is simpler, but the model accuracy in differentiating similar samples deteriorates
Solution Approach 1:
The patent segments the training data into distinct categories: positive matches, hard negatives (similar non-matches), and easy negatives (dissimilar non-matches). This segmentation allows the model to learn from different types of examples separately, improving its ability to differentiate similar samples while maintaining a structured training process.
Solution Approach 2:
The patent applies local quality by treating different types of training samples with different weights or attention. Hard negatives receive special focus in the blocking-based batching process, allowing the model to pay more attention to difficult differentiation cases while still processing easier examples efficiently.
2Productivity
If traditional training methods exclude similar non-matches, then computational resources are used more efficiently, but the model's ability to differentiate similar samples deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-computing blockings and identifying hard negatives before the main training process. This preliminary organization of data into blocking-based batches allows the model to efficiently access and learn from difficult examples without wasting computational resources during training inference.
Solution Approach 2:
The blocking structure acts as an intermediary between the raw training data and the model training process. It organizes and filters samples into meaningful groups (positive matches, hard negatives, easy negatives), allowing efficient processing while ensuring important difficult examples are not excluded.
3Measurement precision
If blocking-based batches including hard negatives are constructed, then model differentiation accuracy is improved, but device complexity increases
Solution Approach 1:
The blocking mechanism segments the training process into distinct phases: blocking construction, hard negative identification, and batch assembly. This segmentation makes the complex process more manageable and systematic, allowing automated construction of informative batches without overwhelming complexity.
4Measurement precision
If blocking-based batches with multiple data types are used, then training accuracy is improved, but computational resource consumption increases
Solution Approach 1:
The patent applies partial action by selectively including only the most informative samples in each blocking-based batch. Rather than processing all possible samples, it focuses on positive matches, hard negatives, and easy negatives that provide the most learning signal, reducing unnecessary computational waste while maintaining training accuracy.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture are disclosed to improve model training efficiency comprising block circuitry to: generate a first blocking corresponding to first ones of first data samples retrieved from a first data source, the first ones of the first data samples including a first heuristic; and generate a second blocking corresponding to second ones of the first data samples that include a second heuristic; match circuitry to: retrieve a second data sample from a second data source and determine a match of the first blocking or the second blocking; and assign respective ones of the first data samples from the match one of a first designation type or a second designation type; and batch circuitry to: combine the first designation type and the second designation type into a machine learning input batch.