Associated Web Resource Crawling for Latent Threat Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cybersecurity techniques are ineffective in detecting immediate or latent threats associated with malicious web content, failing to extend the evaluation of a single identified web resource threat to encompass associated web resources.
Innovation Solution
Utilizing specialized web crawling techniques to identify a first set of malicious and/or potentially malicious resource identifiers, evaluate associated web resources, and employ a threat detection component to classify and record resource identifiers exceeding a safety or trustworthiness level, facilitating corrective actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional cybersecurity techniques are used to identify malicious web content, then particular web content can be identified as malicious, but immediate or latent threats associated with the malicious web content remain undetectable
Solution Approach 1:
The patent segments the threat detection process into multiple phases: initial malicious content identification, extraction of associated resource identifiers through web crawling, and subsequent threat evaluation of those identifiers. This segmentation allows the system to detect both the primary malicious content and previously undetectable latent threats by systematically breaking down the evaluation into manageable stages.
Solution Approach 2:
The system performs preliminary web crawling to extract associated resource identifiers before conducting the actual threat evaluation. By preparing the list of related identifiers in advance through crawling activities, the system enables comprehensive threat detection without missing latent threats that would otherwise remain undetected during direct analysis.
2Productivity
If conventional techniques evaluate only identified malicious web content, then analysis is focused and efficient, but associated web resources and latent threats are not encompassed
Solution Approach 1:
The patent introduces web crawling as an intermediary process between identifying malicious content and evaluating threats. The crawler acts as a mediator that extracts associated resource identifiers from the malicious content, bridging the gap between focused analysis and comprehensive threat assessment, thereby enabling efficient yet thorough detection of associated web resources.
Solution Approach 2:
The system transitions from a single-dimension analysis of identified malicious content to a multi-dimensional approach by incorporating web crawling that explores associated resources, linked pages, and related identifiers. This dimensional expansion allows the system to maintain analysis efficiency while comprehensively encompassing associated web resources that would otherwise be missed.
3Reliability
If comprehensive web crawling is performed to detect all associated threats, then threat detection is thorough, but computational resources and time increase
Solution Approach 1:
The patent applies partial action by performing web crawling to extract associated resource identifiers without necessarily evaluating every single identifier in exhaustive detail. The system strategically selects and prioritizes identifiers for evaluation based on their relevance and risk level, achieving thorough threat detection for critical threats while reducing overall computational time and resources through selective analysis.
4Measurement precision
If iterative threat analysis is performed on all identified resource identifiers, then detection accuracy improves, but computational resources increase
Solution Approach 1:
The system applies local quality by performing iterative threat analysis with varying intensity on different resource identifiers based on their local characteristics and risk profiles. High-priority identifiers receive more intensive iterative analysis for higher detection accuracy, while lower-priority identifiers receive streamlined evaluation, thereby maintaining overall detection accuracy while reducing total computational resource consumption through differentiated analysis intensity.
Data Source
AI summary
Examples of the present disclosure describe systems and methods for evaluating malicious web content for associated threats using specialized web crawling techniques. A seed resource identifier is evaluated to determine a second resource identifier associated with the seed resource identifier. A resource corresponding to the second resource identifier is scanned to identify a third resource identifier. The third resource identifier is processed with a machine learning model to classify the third resource identifier according to a classification representing a predicted level of threat. The machine learning model trained to classify resource identifiers into a plurality of classifications. A corrective action can be executed based on the classification of the third resource identifier.