Retry Buffer Prioritization for Non-Convergent Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in re-processing data sets that fail to converge during standard processing, leading to errors that may not be recoverable, especially when extensive and spanning multiple data sets.
Innovation Solution
The implementation of a data processing system that includes a data processing circuit, an input buffer, and a kick out circuit, which selectively maintains and prioritizes higher quality data sets for re-processing by comparing attributes such as unsatisfied checks and global iterations, allowing higher quality data sets to be processed first, thereby reducing average latency and increasing the likelihood of successful re-processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data sets are re-processed in the order they are received, then processing simplicity is maintained, but latency increases and successful re-processing probability decreases for lower quality data sets
Solution Approach 1:
The patent changes the processing parameter from FIFO order to quality-based priority order. The kick out circuit compares quality metrics (such as error counts, convergence characteristics) of different data sets and reorders them accordingly, allowing higher quality data sets to be processed first while maintaining systematic control
Solution Approach 2:
The system implements feedback by monitoring processing results and quality metrics of data sets. The kick out circuit uses this feedback information to dynamically adjust the processing order, creating a closed-loop system that continuously optimizes re-processing success probability
2Reliability
If all data sets are maintained in the input buffer for retry, then re-processing opportunity is maximized, but buffer resource consumption increases
Solution Approach 1:
The patent applies discarding and recovering by selectively removing lower quality data sets from the buffer when resources are constrained. The kick out circuit identifies and kicks out data sets with lower re-processing success probability, making buffer space available for higher quality data sets while maintaining overall re-processing effectiveness
Solution Approach 2:
The system applies local quality by treating different data sets differently based on their individual quality characteristics. Instead of uniform buffer allocation, the kick out circuit assigns different retention priorities to different data sets based on their quality metrics, optimizing buffer utilization
3Measurement precision
If extensive errors spanning multiple data sets are detected, then error detection accuracy is improved, but recoverability decreases
Solution Approach 1:
The patent applies segmentation by dividing the error analysis into individual data set level assessments. The kick out circuit evaluates each data set's quality metrics separately, allowing the system to identify which specific data sets are recoverable versus those with extensive errors, enabling selective re-processing of recoverable segments
Data Source
AI summary
Systems, circuits, devices and/or methods related to systems and methods for data processing, and more particularly to systems and methods for re-processing data sets not successfully processed during standard processing.