Irrelevancy Score Generation for Text Classification Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for text classification in the machine learning field, particularly in binary text classification tasks, suffer from high misclassification rates, which is problematic for applications like sorting tax law articles into relevant and irrelevant categories.
Innovation Solution
A computing system utilizing a multi-layer bidirectional transformer, trained with binary cross-entropy loss, generates token sequences and computes relevance and irrelevance probabilities through linear transformations and sigmoid functions to produce an irrelevancy score for input text, effectively categorizing text as relevant or irrelevant.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current binary text classification methods are used, then classification speed is maintained, but misclassification rate increases significantly
Solution Approach 1:
The patent segments the classification task into multiple independent classification models, each trained to identify specific types of irrelevant content. Instead of relying on a single binary classifier, the system divides the classification space into multiple specialized classifiers that work together to reduce misclassifications of relevant content as irrelevant.
Solution Approach 2:
The patent introduces an intermediary scoring mechanism that combines outputs from multiple classification models. Rather than directly binary-classifying text as relevant or irrelevant, the system uses intermediate relevance scores from multiple models to make a final determination, reducing the risk of misclassification.
2Reliability
If multiple classification models are used to reduce misclassification, then classification accuracy improves, but system complexity increases
Solution Approach 1:
The patent designs classification models with universal architectures that can be applied across different domains and text types. The models use common training procedures and can be adapted to different classification tasks, reducing the overall system complexity despite using multiple models.
Solution Approach 2:
The patent adjusts model parameters and thresholds dynamically based on the specific classification task and data characteristics. By optimizing parameters for each model type and using adaptive thresholding, the system maintains high accuracy while managing complexity through parameter optimization rather than structural complexity.
3Manufacturing precision
If traditional text classification approaches are used, then computational resources are conserved, but classification quality becomes insufficient for specialized applications
Solution Approach 1:
The patent applies partial classification by focusing computational resources on identifying and filtering out irrelevant content rather than achieving perfect classification of all content. The system performs excessive classification on potential irrelevant items while using simpler criteria for obviously relevant content, optimizing the balance between quality and resource usage.
Data Source
AI summary
Systems and methods are provided that include a processor executing a program to generate token sequences based on input text, generate an encoder output by inputting the token sequences into a multi-layer bidirectional transformer, linearly transform the encoder output and output a transformed relevant output and transformed irrelevant output, respectively, compute a relevance probability and a first irrelevance probability, respectively, by inputting the transformed relevant output and the transformed irrelevant output into a sigmoid function, the relevance probability being a probability that the token sequence belongs in the relevant classification group, the first irrelevance probability being a probability that the token sequence belongs in the irrelevant classification group, compute a second irrelevance probability by inputting the relevance probability and the first irrelevance probability into a tensor product formula, and generate and output an irrelevancy score for the input text based on the second irrelevance probability.