Unstructured Text Filtering for Software Vulnerability Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional web monitoring techniques for detecting security vulnerabilities in computer programs are expensive, limited in scalability, and ineffective against cryptic language used by malicious entities on the dark web, often failing to cover a sufficient number of sources and accurately identifying relevant discussions.
Innovation Solution
A machine learning model is trained to identify keywords corresponding to a computer program and security vulnerabilities within unstructured text, filtering user-generated posts to enhance security by automating the detection of potential or existing attacks, using a combination of product and vulnerability corpora to determine keyword frequencies and applying neural networks for pattern recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If human analysts are employed to monitor web sources for security vulnerabilities, then detection accuracy is improved, but cost and scalability deteriorate
Solution Approach 1:
The patent replaces the mechanical system of human analysts with an automated machine learning-based text classification system. The system uses trained models to analyze web content, extract keywords, and identify security vulnerability discussions automatically, eliminating the need for human labor while maintaining detection capabilities and enabling scalable monitoring of numerous web sources simultaneously.
Solution Approach 2:
The system enables self-service by automatically performing the entire security monitoring workflow without human intervention. The machine learning models autonomously classify text, extract relevant keywords, identify vulnerability discussions, and generate reports, allowing the system to serve its own security monitoring function continuously and independently.
2Ease of manufacture
If conventional text classification based on entire sentences is used, then implementation simplicity is improved, but detection reliability deteriorates due to cryptic language
Solution Approach 1:
The patent segments the text analysis process into multiple levels: first dividing content into sentences, then further segmenting sentences into individual words and phrases for keyword extraction. This hierarchical segmentation allows the system to analyze cryptic language at the word level while maintaining the contextual structure of sentences, improving detection reliability without significantly complicating implementation.
Solution Approach 2:
The patent adds a new dimension to text analysis by incorporating keyword frequency analysis and vector space modeling alongside traditional sentence-level classification. This multi-dimensional approach allows the system to detect security vulnerabilities through multiple pathways (sentence context, keyword presence, frequency patterns), improving reliability while building upon conventional classification methods.
3Reliability
If comprehensive web monitoring is implemented to cover all sources, then detection coverage is improved, but resource consumption deteriorates
Solution Approach 1:
The patent applies partial action by focusing monitoring efforts on the most relevant web sources and content types rather than uniformly monitoring all available sources. The system prioritizes high-value targets such as security forums, vulnerability databases, and relevant news sites, achieving comprehensive coverage of critical sources while consuming fewer resources than universal monitoring would require.
Solution Approach 2:
The system applies local quality by allocating monitoring resources differently across various web sources based on their security relevance. High-priority sources receive more intensive analysis with higher keyword matching thresholds, while lower-priority sources receive lighter monitoring, optimizing resource consumption according to the local quality and importance of each source.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Techniques are described herein that are capable of increasing security of a computer program using unstructured text. Unstructured text is received from web-based sources. The unstructured text includes user-generated posts. A machine learning model is trained by determining each keyword of a plurality of keywords in the unstructured text that corresponds to a computer program and further by determining each keyword of the plurality of keywords in the unstructured text that corresponds to a security vulnerability. The user-generated posts that are included in the unstructured text are filtered, using the machine learning model, to provide a subset of the user-generated posts such that each user-generated post in the subset includes a keyword that corresponds to the computer program and a keyword that corresponds to the security vulnerability. An action is performed based at least in part on the subset of the user-generated posts.