Expert-Guided Document Analysis System with Graduated Relevance Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computerized systems for analyzing electronic documents are inefficient in identifying relevant documents and ranking their relevance, relying heavily on manual keywords and binary scoring, which leads to high costs, errors, and inconsistent results due to the lack of graduated relevance scores and automated prioritization.
Innovation Solution
An expert-guided system that uses machine learning to compute graduated relevance scores for documents, iteratively improving accuracy by feedback from experts, and providing automated prioritization and keyword generation to enhance the efficiency of document review processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual keyword-based classification is used, then document analysis can be performed, but the system suffers from high costs, errors, and inconsistent results due to lack of automated prioritization and graduated scoring
Solution Approach 1:
The patent transforms binary classification (relevant/not relevant) into multi-level graduated scoring (e.g., 1-5 scale), changing the parameter of classification granularity. This enables more nuanced document prioritization and reduces inconsistency by providing intermediate categories rather than forced binary choices, directly addressing the reliability issue while maintaining manageable system complexity through iterative machine learning.
Solution Approach 2:
The system implements iterative feedback loops where expert reviews of classified documents feed back into retraining the classification algorithms. This continuous feedback mechanism improves consistency over time by correcting errors and refining the graduated scoring system, while the automation reduces the manual effort required compared to purely manual classification.
2Productivity
If all documents are reviewed manually, then complete accuracy can be achieved, but the process becomes extremely time-consuming and costly
Solution Approach 1:
The patent segments the document review process into multiple stages: automated pre-filtering using machine learning classifiers, followed by graduated scoring to prioritize documents, and finally focused manual review only on high-priority or ambiguous cases. This segmentation dramatically increases productivity by eliminating the need for complete manual review of all documents while maintaining accuracy through targeted expert assessment of the most relevant subset.
Solution Approach 2:
The system performs preliminary automated classification and graduated scoring before manual review, pre-sorting documents by relevance level. This preliminary action filters out clearly irrelevant documents and prioritizes those needing expert attention, thereby increasing overall process speed while preserving accuracy by ensuring experts focus on documents where their judgment is most valuable.
3Measurement precision
If binary scoring is used for document classification, then the process is simple, but the system cannot provide graduated relevance scores needed for effective prioritization
Solution Approach 1:
The patent implements dynamic graduated scoring where documents receive multi-level scores (e.g., 1-5 scale) that can be adjusted iteratively through machine learning retraining based on expert feedback. This dynamic approach provides the necessary granularity for effective prioritization while managing complexity through automated algorithms that learn and adapt, rather than requiring complex manual scoring guidelines.
Solution Approach 2:
The classification system performs self-service through automated machine learning algorithms that generate graduated scores without requiring manual intervention for each document. The system automatically retrains and refines its scoring mechanism based on feedback, reducing the complexity burden on operators while maintaining high measurement precision through iterative self-improvement.
4Reliability
If multiple experts review documents independently, then diverse perspectives are obtained, but discrepancies and disagreements increase review time and costs
Solution Approach 1:
The patent introduces an automated graduated scoring system as an intermediary between multiple experts. Each expert reviews documents independently, but their assessments are compared against the automated graduated scores, which serve as a mediator to identify and resolve discrepancies. This intermediary approach maintains the benefits of multiple expert perspectives while reducing time loss by providing an objective reference framework for quick resolution of disagreements.
Data Source
AI summary
A system including an electronic repository having a multiplicity of accesses to a respective multiplicity of electronic documents and metadata; a document rater using a processor to run a first computer algorithm on the multiplicity of electronic documents which yields a score which rates each of the multiplicity of electronic documents to an issue; and a metadata-based document discriminator to run a second computer algorithm on at least some of the metadata which yields leads, each lead having at least one metadata value for at least one metadata parameter, whose value correlates with the score of the electronic documents to the issue, typically used in combination with an electronic document analysis method receiving N electronic documents pertaining to a case encompassing a set of issues including at least one issue and establishing relevance of at least the N documents to at least one individual issue in the set of issues.