E-Discovery Notification via Dual Queue Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional eDiscovery systems with passive schedulers face significant delays in notifying clients of processing work inability, particularly when no secondary computer can handle tasks like language translation, due to lengthy time-out periods, which is undesirable in eDiscovery projects.
Innovation Solution
Implementing a system where requests are placed in a first queue and a copy in a second queue, with worker computers rapidly scanning the second queue to determine capability; if the request remains after a brief time, the master computer notifies the client that the worker computers are unable to perform the task, reducing notification delays from hours to seconds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a passive scheduler is used in a conventional eDiscovery system, then the system can handle a large number of secondary computers with varying performance, but the notification delay to clients increases significantly when processing work cannot be completed
Solution Approach 1:
The master computer proactively monitors the job queue for tasks that have been waiting beyond a threshold duration and sends notification messages to client computers before the timeout period elapses. This preliminary action prevents the lengthy waiting period that occurs in conventional systems where clients only learn of processing failures after extended timeouts.
Solution Approach 2:
The system implements a feedback mechanism where the master computer continuously monitors the state of the job queue and sends status updates to client computers. When a job remains in the queue beyond a certain duration, the system automatically notifies the client, creating a closed-loop feedback system that keeps clients informed of processing status without requiring them to wait for timeout periods.
2Loss of time
If an active scheduler is used to distribute processing work directly to secondary computers, then the notification speed improves, but the system complexity increases significantly
Solution Approach 1:
The master computer serves as an intermediary between the job queue and client computers. Instead of implementing a complex active scheduler that directly manages secondary computer assignments and monitoring, the master computer simply monitors the job queue for overdue tasks and notifies clients. This intermediary approach simplifies the scheduling architecture while still achieving timely client notification.
3Duration of action of moving object
If a long timeout period is used to allow secondary computers to complete processing, then the system gives workers adequate time to finish tasks, but client notification is significantly delayed
Solution Approach 1:
The system sends notification messages to clients in advance of the timeout period expiring. By monitoring the job queue and identifying tasks that have been waiting beyond a threshold duration, the system proactively notifies clients before the full timeout period elapses, thereby reducing the effective waiting time for clients while still allowing adequate processing time for complex tasks.
Data Source
AI summary
Improved techniques of fulfilling a request to perform a task involve a master computer placing the request in a first queue and a copy of the request in a second queue, the second queue being frequently accessed by a set of worker computers which rapidly scans the second queue for requests to fulfill. If, during the scanning, a worker computer determines that it has a capability to fulfill the request, the worker computer removes the copy of the request from the second queue. Furthermore, if the copy of the request remains in the second queue after a brief time period, it is clear that the set of worker computers is unable to perform the task. In this case, the master computer takes a remedial action such as notifying a client computer which sent the request that the worker computers, as currently configured, are unable to perform the task.