Dynamic Content Characterization for URL Classification Transparency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing machine learning classification systems for URL content in cloud-based security systems lack transparency, leading to misclassifications and a need for understanding why predictions are made to improve model accuracy and efficiency.

Innovation Solution

Implementing Dynamic Content Characterization (DCC) that uses machine learning techniques such as SHapley Additive explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) to analyze and explain predictions, focusing on data labeling, preprocessing, feature extraction, and building a Light Gradient Boosted Machine (LightGBM) model for accurate URL classification and categorization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If machine learning classification is used for URL content categorization, then automation and productivity are improved, but transparency and understanding of predictions deteriorate

Engineering Contradiction:

ImproveURL classification throughputVSAvoidPrediction transparency

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent introduces explanation mechanisms (SHAP, LIME) as intermediary components between the machine learning model and users. These intermediaries translate complex model predictions into interpretable insights about why URLs are classified into certain categories, preserving automation while restoring transparency through intermediate explanation layers.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Manufacturing precision

If complex machine learning models are deployed for accurate URL classification, then manufacturing precision is improved, but device complexity increases

Engineering Contradiction:

ImproveClassification accuracyVSAvoidModel complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent segments the complex classification task into multiple components: feature extraction, model prediction, and explanation generation. By dividing the system into these modular segments, the patent maintains high classification accuracy while making the overall system more manageable and interpretable through separate explanation components.

Inventive Principle:

Principle #1Segmentation

3Adaptability or versatility

If machine learning models are used for dynamic content classification, then adaptability is improved, but reliability deteriorates due to misclassifications

Engineering Contradiction:

ImproveDynamic URL categorizationVSAvoidPrediction accuracy

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent implements feedback mechanisms where explanation insights are used to identify and correct misclassifications. By continuously analyzing prediction explanations and using that feedback to refine the model, the system improves reliability while maintaining adaptability to new URL categories.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20220121984A1Explaining internals of Machine Learning classification of URL content

Publication Date: 2022.04.21 ZSCALER INC

AI summary

Systems and methods include obtaining Uniform Resource Locator (URL) transactions that were either undetected by a machine learning model or mischaracterized by the machine learning model; filtering the URL transactions based on any of size and transaction count; utilizing one or more techniques to determine words that provide an explanation for a category of a plurality of categories of the filtered URL transactions; and utilizing a label for the filtered URL transactions and the determined words for each as training data to update the machine learning model.