Document Review System Using Generative Models for E-Discovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for reviewing large document sets, particularly in e-discovery, require extensive human interaction and lead to significant false-positives due to the need for initial setup time and keyword searches, which are inefficient and labor-intensive.
Innovation Solution
A system utilizing a topic-related generative model to build a document map, generate a control set, and select machine-call-responsive documents based on predictive responsiveness, with a task/queue framework and messaging framework for efficient data processing and user feedback integration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If keyword searches and search filters are used for assisted review, then document retrieval speed is improved, but false-positives increase and significant initial setup time is required
Solution Approach 1:
The system implements feedback loops where reviewer decisions on control set documents are used to retrain and refine the machine learning model. The model continuously learns from reviewer feedback, adjusting its predictions to reduce false-positives while maintaining retrieval speed. This iterative feedback process allows the system to improve reliability without sacrificing the speed benefits of automated retrieval.
Solution Approach 2:
The system dynamically adjusts search parameters and model thresholds based on performance metrics and reviewer feedback. By changing parameters such as confidence thresholds, weighting schemes, and search criteria, the system optimizes the balance between retrieval speed and accuracy, reducing false-positives while maintaining efficient document retrieval.
2Measurement precision
If extensive human interaction is required for document analysis, then review accuracy is improved, but labor intensity and time consumption increase significantly
Solution Approach 1:
The system segments the review process into distinct phases: automated preprocessing and initial sorting by the machine learning model, followed by focused human review of only the most relevant or uncertain documents. This segmentation allows human reviewers to concentrate their expertise on critical decisions while the automated system handles routine classification, thereby maintaining accuracy while significantly improving productivity.
Solution Approach 2:
The machine learning model acts as an intermediary between the document set and human reviewers. It pre-processes and ranks documents, presenting only the most relevant candidates to human reviewers. This intermediary layer filters out clearly irrelevant documents, allowing human expertise to be applied more efficiently and reducing overall labor intensity while maintaining review accuracy.
3Reliability
If manual document review processes are used, then thorough analysis is achieved, but hours of labor and extensive training are required
Solution Approach 1:
The system performs preliminary actions by automatically preprocessing documents, extracting key features, and generating initial classifications before human review. The machine learning model pre-sorts documents and identifies likely candidates for further review, so that when human reviewers do engage with the material, they can focus on thorough analysis of only the relevant documents, significantly reducing total review time while maintaining analysis quality.
Solution Approach 2:
The system replaces the mechanical process of manual document screening and initial analysis with an automated machine learning-based system. This substitution handles the bulk of document processing, feature extraction, and initial classification, freeing human reviewers to perform more sophisticated analysis on a smaller subset of documents, thereby reducing overall time investment while maintaining or improving analysis thoroughness.
Data Source
AI summary
Embodiments may provide a document system that receives a responsiveness call from a user through the task/queue framework regarding a machine call document. Theses responsiveness calls may be used to refining the scoring algorithm used by the document system of to generate a desired confidence score for the document system.