Hybrid Active Learning for Non-Stationary Streaming Data

The patent divides the selection strategy into two independent segments: pool-based selection for periodic batch processing to maintain high selection quality, and stream-based selection for continuous real-time processing to reduce delays. Each segment operates with its own selection criteria and confidence thresholds, allowing them to function independently while complementing each other's strengths and weaknesses.

The patent merges pool-based and stream-based selection strategies into a hybrid system where both operate simultaneously on the same data stream. The pool-based strategy processes batches periodically to ensure high-quality selections, while the stream-based strategy continuously processes individual instances to minimize processing delays. The system integrates their outputs through a unified data structure that tracks selections from both strategies.

The patent segments the processing into real-time stream-based decisions for immediate annotation needs and periodic pool-based batch processing for quality optimization. The stream-based segment operates continuously with lower confidence thresholds to capture urgent annotation opportunities, while the pool-based segment periodically refines selections with more sophisticated evaluation.

The patent implements periodic execution of the pool-based selection strategy at predetermined intervals or when confidence threshold conditions are met. This periodic batch processing allows the system to periodically recalibrate and improve selection quality without compromising continuous real-time processing capabilities. The frequency of periodic execution can be adjusted based on data stream characteristics and annotation resource availability.

The patent dynamically adjusts the confidence threshold parameter based on the ratio of pool-based to stream-based selections and the availability of annotation resources. When annotation resources are abundant, the threshold lowers to include more uncertain instances. When resources are constrained, the threshold raises to focus only on the most critical instances, optimizing the balance between accuracy improvement and resource consumption.

The patent applies partial annotation by selectively annotating only the most critical data instances identified by the hybrid selection strategy, rather than annotating all uncertain instances. The stream-based strategy identifies immediate priorities, while the pool-based strategy identifies high-value targets, allowing the system to achieve significant accuracy improvements with a fraction of the annotation resources that would be required for comprehensive annotation.

The patent implements periodic model updates triggered by accumulating a threshold number of annotated instances or reaching predetermined time intervals. Instead of continuous retraining, the system accumulates annotated data from both pool-based and stream-based selections, then performs batch model updates at periodic intervals. This reduces computational overhead by avoiding frequent small-scale retraining while still maintaining accuracy through regular updates.

The patent performs preliminary filtering and selection of high-value training instances through the hybrid active learning strategy before model updates. By pre-selecting only the most informative annotated instances through combined pool-based and stream-based criteria, the system prepares optimized training datasets in advance, reducing the computational burden during actual model retraining while maximizing the impact of each update.