Streaming Text Classification Decision Point Determination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text classification systems face challenges in classifying streaming text data, particularly in contact centers, where non-stationary behavior and conversational dynamics make it difficult to determine the optimal time for classification decisions, as traditional classifiers are designed for complete documents and not suited for real-time, incremental learning in scripted conversations.
Innovation Solution
A system and method that use certainty calculations based on accumulated conversational data to determine a decision point for classifying streaming text, employing an entropy-based approach to select the appropriate text segment for classification, allowing for real-time classification and incremental updates, and retraining on partial call segments to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional classifiers designed for complete documents are used, then classification can be performed on full text, but real-time classification and incremental learning are not supported
Solution Approach 1:
The patent segments the complete document into multiple text segments that can be processed incrementally. The classifier is applied to individual segments or portions of the document in real-time, allowing incremental learning while maintaining classification accuracy through selective segment processing.
Solution Approach 2:
The system dynamically adjusts the classification process by determining optimal decision points during streaming data processing. Rather than static batch processing, the classifier adapts to incoming data streams in real-time, making classification decisions at dynamically determined points based on certainty calculations.
2Loss of information
If the whole call is collected before classification, then complete information is available, but real-time decision making is delayed
Solution Approach 1:
The system performs preliminary classification actions at intermediate decision points during the conversation rather than waiting for complete data. Certainty calculations are performed at multiple time instances to determine when sufficient information has been gathered to make an accurate classification decision, enabling early termination and real-time decision-making.
Solution Approach 2:
The patent replaces the mechanical approach of collecting all data before classification with an entropy-based certainty calculation system. This computational approach dynamically determines when classification can be performed by measuring information sufficiency, substituting the traditional sequential collection process with a parallel certainty assessment mechanism.
3Ease of manufacture
If a fixed window size is used for incremental learning, then implementation is simple, but adaptability to non-stationary data streams is reduced
Solution Approach 1:
The system transitions from static fixed-window incremental learning to dynamic decision point determination. Instead of processing data in fixed time or size windows, the classifier dynamically determines decision points based on certainty calculations that adapt to the actual information content and non-stationary characteristics of the data stream.
Solution Approach 2:
The patent changes the fundamental parameter from fixed window size to dynamic certainty threshold. By using entropy-based certainty calculations, the system adapts its processing parameters based on the actual state of the data stream, allowing it to respond to non-stationary conditions while maintaining implementation feasibility through standardized calculation procedures.
4Productivity
If incremental learning with sliding windows is used, then online data stream classification is enabled, but classification accuracy on partial data deteriorates
Solution Approach 1:
The system incorporates feedback through certainty calculations at multiple decision points. The entropy-based certainty measure provides feedback on whether sufficient information has been gathered for accurate classification, allowing the system to terminate processing early when accuracy requirements are met while maintaining high productivity through avoided unnecessary processing.
Solution Approach 2:
The patent applies partial action by performing classification only when certainty thresholds are met, rather than processing all possible data. This selective partial processing maintains classification accuracy by avoiding decisions on insufficient data while improving productivity by terminating processing early when sufficient information is available.
Data Source
AI summary
A method for determining a decision point in real-time for a data stream from a conversation includes receiving streaming conversational data; and determining when to classify the streaming conversational data, using a measure of certainty, by performing certainty calculations at a plurality of time instances during the conversation and by selecting a decision point in response to the certainty calculations, the decision point not being based on a fixed window of conversational data but being based on accumulated conversational data available at different ones of the plurality of time instances. Systems and computer program products are also provided.