Automated Lexicon Construction from Unlabeled Event Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current approaches to constructing lexicons for detecting entity behavior are manual, time-consuming, and prone to errors, often failing to include all relevant terms, especially for anomalous or malicious behavior, which can lead to inefficiencies and inaccuracies in identifying security risks.

Innovation Solution

A method and system for constructing a lexicon by identifying a corpus of training events, grouping terms into topic clusters, analyzing these clusters to derive learned lexicons, which automates the addition of relevant terms and improves the efficiency and accuracy of identifying anomalous or malicious entity behavior.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual approaches are used to construct lexicons for detecting entity behavior, then the process allows human oversight and customization, but it is time-consuming and prone to errors, reducing productivity and reliability

Engineering Contradiction:

Improveaccuracy of lexicon constructionVSAvoidtime consumption in lexicon construction

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent replaces the manual mechanical process of lexicon construction with an automated computer-implemented system. The system automatically identifies candidate terms from event data, groups them into topic clusters using algorithms, and derives lexicons without human intervention, thereby eliminating time consumption and human error while maintaining reliability through systematic processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system performs self-service by automatically constructing lexicons from available event data without requiring manual curation. The automated clustering algorithms and term identification processes enable the system to generate and update lexicons independently, significantly improving productivity while maintaining consistent quality through repeatable computational processes

Inventive Principle:

Principle #25Self-service

2Reliability

If manual lexicon construction is used, then human expertise can guide the process, but relevant terms for anomalous or malicious behavior are often missed, reducing the completeness and detection accuracy

Engineering Contradiction:

Improvedetection accuracy of security risksVSAvoidmissing relevant terms in lexicon

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The system performs preliminary action by automatically identifying and capturing candidate terms from event data before security analysis is needed. The automated process continuously builds and updates lexicons with relevant terms including those for anomalous or malicious behavior, ensuring comprehensive coverage without relying on human anticipation of future security threats

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system incorporates feedback mechanisms where event data and clustering results continuously inform lexicon construction. The automated process analyzes patterns in security events and adjusts term selection and grouping accordingly, ensuring that relevant terms for detecting anomalous or malicious behavior are consistently included based on actual data patterns rather than human assumption

Inventive Principle:

Principle #23Feedback

3Productivity

If automated processes are implemented for lexicon construction, then productivity and consistency are improved, but the complexity of the system increases

Engineering Contradiction:

Improveefficiency of lexicon constructionVSAvoidsystem complexity for automated lexicon construction

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the lexicon construction process into distinct modular components: event data ingestion, candidate term identification, topic clustering, and lexicon derivation. Each module performs a specific function and can be independently implemented or adjusted, reducing overall system complexity while maintaining high productivity through automated processing of each segment

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11568136B2Automatically constructing lexicons from unlabeled datasets

Publication Date: 2023.01.31 EVERFOX HOLDINGS LLC

AI summary

A system, method, and computer-readable medium are disclosed for performing a lexicon construction operation. The lexicon construction operation includes: identifying a corpus, the corpus comprising a plurality of training events, each of the plurality of training events comprising a term; grouping terms from the plurality of training events into topic clusters; analyzing the plurality of topic clusters, the analyzing providing a plurality of classified clusters; and, deriving a plurality of learned lexicons from the plurality of classified clusters.