Real-time keyword detection using single-sweep incidence database

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current content filtering technologies are inadequate for real-time differentiation between desired and undesired online content due to limitations in processing speed, memory usage, and the inability to efficiently evaluate a large number of keywords and rules, leading to obtrusive delays and inefficiencies in classifying textual content on the internet.

Innovation Solution

A single-sweep content analysis system utilizing an incidence database and detection tree structure that updates scores in real-time by incrementing counters and adjusting scores based on keyword occurrences, allowing for simultaneous analysis of multiple texts without resetting the database, thereby enabling fast and efficient classification of content.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional text content analysis and data mining techniques are used to classify Internet content, then the filtering can be comprehensive enough to recognize undesired content, but the processing time becomes too long causing obtrusive delays

Engineering Contradiction:

Improvecontent classification accuracyVSAvoidprocessing delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system pre-compiles a detection tree structure containing keywords and classification rules before actual content analysis. This preliminary organization of filtering criteria into a hierarchical tree structure with nodes representing keywords and branches representing rule relationships enables rapid real-time querying without reprocessing the entire rule set during content classification

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The content analysis system is divided into independent modular components: keyword detection modules, rule evaluation modules, and score calculation modules. Each component operates independently and can be processed in isolation, allowing parallel execution and reducing overall processing time while maintaining comprehensive classification capability

Inventive Principle:

Principle #1Segmentation

2Productivity

If URL-based filtering is used to block undesired content, then the implementation is simple and fast, but it cannot differentiate between desired and undesired content within the same URL

Engineering Contradiction:

Improvefiltering speedVSAvoidcontent differentiation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system transitions from uniform URL-level filtering to granular content-level filtering by analyzing specific textual elements within pages. Different portions of content are evaluated against different keywords and rules, allowing precise identification and classification of desired versus undesired content segments within the same URL based on their local characteristics

Inventive Principle:

Principle #3Local quality

3Measurement precision

If a large number of keywords and rules are evaluated for accurate content classification, then the classification becomes more precise, but the processing time increases causing delays

Engineering Contradiction:

Improvecontent classification precisionVSAvoidevaluation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The detection tree structure pre-organizes keywords and rules hierarchically before content analysis, with common keywords positioned at higher levels and more specific keywords at lower levels. This preliminary structuring enables the system to evaluate fewer keywords in many cases by matching content against the hierarchical structure, reducing evaluation time while maintaining comprehensive classification precision

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Multiple keyword matching operations and rule evaluations are merged into a single integrated detection tree traversal process. Instead of separately evaluating each keyword and then each rule, the system combines these operations into one unified pass through the tree structure, reducing redundant processing and improving efficiency

Inventive Principle:

Principle #5Merging (Combining)

4Reliability

If off-line content analysis is performed to build comprehensive filtering databases, then the classification coverage is complete, but it cannot keep up with real-time Internet content delivery

Engineering Contradiction:

Improveclassification coverageVSAvoidreal-time processing speed

Core Design Contradiction:

ReliabilityVSSpeed

Solution Approach 1:

The system performs preliminary organization of filtering criteria into a compact detection tree structure that can be efficiently queried in real-time. This pre-processing step creates an optimized data structure that maintains comprehensive classification coverage while enabling fast online evaluation without requiring continuous off-line database building

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9519704B2Real time single-sweep detection of key words and content analysis

Publication Date: 2016.12.13 NETABPARK

AI summary

A system and method are provided for real-time analysis of text. During a single sweep through the text, a detection tree is used to identify a sequence of characters in the text from a large dictionary of keywords. When a keyword is detected a rule tally database is updated. An intermediate score may be available during the sweep and a final score of the text may be available substantially immediately upon finishing the single sweep. A second text may be analyzed immediately using the same score buffer and rule tally database without updating the rule tally database.