Prompt-Based Document Classification Explainability for Consistent eDiscovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing eDiscovery processes face inefficiencies and inconsistencies due to the need for manual review of thousands of documents to train machine learning classifiers and varying interpretations by attorneys, leading to conflicting document classifications.
Innovation Solution
A prompt-based classification model using a generative AI model is employed, where prompt criteria are defined to classify documents, and explanations are generated for the classifications, reducing the manual review required and aligning interpretations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional machine learning classifiers are used, then document classification can be automated, but thousands of documents must be manually reviewed to train the classifier
Solution Approach 1:
The patent uses prompt-based copying where instead of training a model on thousands of labeled documents, the system copies the reasoning patterns from a few example documents and case law to generate classifications. The prompt engineering approach allows the AI to replicate legal reasoning without requiring extensive training data, dramatically reducing manual annotation requirements while maintaining classification quality.
Solution Approach 2:
The patent changes the fundamental parameter of how classification is achieved - from statistical pattern matching through trained models to rule-based reasoning through prompts. This parameter change allows the system to achieve high accuracy with minimal training data by using natural language instructions that encode legal standards and reasoning frameworks.
2Productivity
If different attorneys use machine learning models, then document review can be performed, but conflicting indications of responsiveness result
Solution Approach 1:
The patent applies homogeneity by using a single, standardized prompt framework that all attorneys can apply consistently. Instead of each attorney training their own model with subjective criteria, the system uses a unified prompt structure based on legal standards that ensures all classifications follow the same reasoning logic, eliminating variability caused by different attorney interpretations.
Solution Approach 2:
The patent introduces an intermediary layer of prompt-based reasoning that mediates between the attorney's search query and the document classification. This intermediary framework translates diverse attorney intentions into a standardized classification process, ensuring consistent results across different users while maintaining the ability to address specific case needs.
3Measurement precision
If manual document review is performed, then accurate classification can be achieved, but the process is cumbersome and inefficient
Solution Approach 1:
The patent replaces the mechanical process of manual document review with an AI-based system that uses prompt engineering to achieve classifications. Instead of requiring attorneys to read and analyze each document individually, the system uses natural language prompts that encode legal reasoning to automatically generate accurate classifications, maintaining precision while dramatically improving efficiency.
Data Source
AI summary
The following relates generally to using generative AI to: (i) classify documents; (ii) generate prompts (and/or criteria for prompts) to classify documents; (iii) explain document classifications; and/or (iv) explain updates to prompts (and/or prompt criteria). In some embodiments, one or more processors: obtain at least one prompt criteria defining context for classifying a corpus of documents using a generative AI model; generate a first prompt based upon the at least one prompt criteria; input the first prompt and a first document of the corpus of documents into the generative AI model to generate a classification of the first document; and generate an explanation of why the generative AI model generated the classification based on an output of the generative AI model.