Lexical Change Detection in Text Data Streams

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The vast volume of text data streams makes it impractical for human analysts to detect and summarize changes in lexical items over time, as existing methods are either too expensive or impossible to implement comprehensively.

Innovation Solution

A system and method for detecting and coordinating changes in lexical items within data streams by using a lexical occurrence model, applying significance and interestingness tests, and grouping changes across lexical and metavalue vocabularies to summarize synchronous events, which can be implemented in both retrospective and stream analysis modes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If comprehensive human inspection of text data streams is performed to detect changes in lexical items, then measurement precision is improved, but productivity deteriorates due to the vast volume of data

Engineering Contradiction:

Improvedetection accuracy of lexical changesVSAvoidprocessing speed of text data

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent introduces an automated change detection system that acts as an intermediary between the text data streams and human analysts. The system processes lexical items, applies statistical models, and generates structured reports automatically, eliminating the need for comprehensive human inspection while maintaining high detection precision through algorithmic analysis of lexical occurrences and change-point detection.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical process of human reading and analysis with automated computational methods. Statistical models, probability calculations, and algorithmic change-point detection substitute for human cognitive processes, enabling rapid processing of large text volumes while preserving measurement precision through systematic quantitative analysis.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If automated change detection systems are implemented to improve productivity, then processing speed is improved, but measurement precision deteriorates due to loss of nuanced understanding

Engineering Contradiction:

Improveprocessing speed of text dataVSAvoiddetection accuracy of lexical changes

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent employs sophisticated statistical parameters and models to maintain precision in automated detection. By using probability distributions, significance thresholds, and structured change-point detection algorithms, the system transforms qualitative lexical analysis into quantitative measurements that preserve nuanced understanding while enabling high-speed automated processing.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If detailed analysis of each document in the corpora is performed to determine changes, then measurement precision is improved, but loss of time increases making the process expensive and difficult

Engineering Contradiction:

Improveaccuracy of change detectionVSAvoidtime required for comprehensive analysis

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent extracts only the essential information needed for change detection from the text corpora. By focusing on lexical occurrences, metadata, and structured patterns rather than analyzing each document in full detail, the system achieves accurate change detection while dramatically reducing the time required for analysis.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies partial analysis by selectively examining specific lexical items and metadata relevant to change detection rather than performing exhaustive analysis of all document content. This targeted approach maintains measurement precision for critical changes while minimizing time loss through efficient sampling and focused statistical modeling.

Inventive Principle:

Principle #16Partial or excessive action

4Ease of operation

If summarization of changes is provided without structured output, then ease of operation is improved, but loss of information increases reducing analytical value

Engineering Contradiction:

Improvesimplicity of change summaryVSAvoidcontextual information about changes

Core Design Contradiction:

Ease of operationVSLoss of information

Solution Approach 1:

The patent segments change information into structured components including change types, lexical items, metadata, and temporal patterns. This segmentation organizes complex change data into manageable, easily interpretable units while preserving contextual information through systematic categorization and structured output formats that maintain analytical value.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11023676B2Systems and methods for detecting and coordinating changes in lexical items

Publication Date: 2021.06.01 AT&T INTELLECTUAL PROPERTY I L P

AI summary

Systems and methods for efficiently detecting and coordinating step changes, trends, cycles, and bursts affecting lexical items within data streams are provided. Data streams can be sourced from documents that can optionally be labeled with metadata. Changes can be grouped across lexical and/or metavalue vocabularies to summarize the changes that are synchronous in time. The methods described herein can be applied either retrospectively to a corpus of data or in a streaming mode.