ML Document Evaluation for Adverse Subject Matter Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in efficiently reviewing large numbers of documents to identify potentially adverse subject matter, which is impractical for humans to accomplish in a timely manner.
Innovation Solution
A system and method utilizing Machine Learning (ML) to evaluate electronic documents by training a document evaluation ML model with labeled text samples for specific subject matter categories, allowing for the identification of risk levels and annotation of text segments within these categories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review of documents is performed to identify adverse subject matter, then accuracy of identification can be maintained, but productivity decreases significantly when large numbers of documents need to be reviewed
Solution Approach 1:
The patent introduces an ML model as an intermediary between the document and the reviewer. The model processes documents automatically to identify adverse subject matter, providing preliminary results that reviewers can then verify. This intermediary handles the high-volume processing while maintaining accuracy through the reviewer's final validation, thus resolving the contradiction between automated speed and human accuracy.
Solution Approach 2:
The system applies ML model processing to all documents (excessive action for automation) but only requires human review for documents that meet certain risk thresholds or show low confidence scores (partial human action). This approach maximizes productivity for clear-cut cases while maintaining accuracy for ambiguous cases, effectively resolving the speed-accuracy tradeoff.
2Loss of time
If ML models are trained with limited labeled text samples, then training time and resource requirements decrease, but measurement precision of risk level identification may worsen
Solution Approach 1:
The system performs preliminary action by using unsupervised learning and clustering algorithms to pre-process and organize the training data before applying supervised learning. This preliminary organization of data structures and feature extraction reduces the amount of labeled data needed while maintaining model accuracy, thus resolving the contradiction between training efficiency and model performance.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the number of clusters, dimensionality reduction parameters, and learning rates based on the available training data size. When labeled samples are limited, the system adapts its parameters to optimize performance with fewer samples, thereby maintaining measurement precision while accepting the constraint of limited training data.
Data Source
AI summary
Apparatus and method of evaluating electronic documents. In an embodiment, the method comprises obtaining a dataset comprising labeled text samples associated with subject matter categories and having a respective label comprises a risk level. The method comprises applying a machine learning (ML) model to the labeled text samples to vectorize the labeled text samples, and determining a representative vector for groups of text sample vectors associated with a same risk level in each subject matter category. The method comprises displaying an electronic document, extracting a text segment, applying the ML model to the text segment to vectorize the text segment, mapping the text segment vector to a subject matter category, determining the risk level associated with the text segment based on a relation between the text segment vector and representative vectors associated with the subject matter category, and annotating the text segment with an annotation based on the risk level.