Example-Guided Security Query Generation for Proprietary Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Security analysts face challenges in understanding the proprietary log structures of multiple security products, requiring proficiency in Kusto Query Language (KQL) for effective security investigations, which is often unfamiliar to junior analysts.
Innovation Solution
A computer-implemented method using a trained machine learning model to select examples and generate metadata for prompts, which are input to a large language model to produce accurate security language queries, such as KQL, by training the model with a probing procedure to assess the impact of included examples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If junior analysts use proprietary security log structures directly, then query accuracy may be maintained, but the learning curve and time required for proficiency increase significantly
Solution Approach 1:
The patent introduces a natural language interface as an intermediary between the analyst and the proprietary KQL query system. Analysts can submit queries in plain English without needing to learn KQL syntax, while the system automatically translates these into accurate KQL queries, thus maintaining query accuracy while eliminating the learning curve
Solution Approach 2:
The patent replaces the mechanical process of learning and applying complex query syntax (KQL) with an automated natural language processing system. Instead of analysts manually constructing queries according to proprietary log structures, the system automatically generates the appropriate KQL from natural language inputs
2Adaptability or versatility
If multiple security products with proprietary log structures are used, then security coverage is improved, but the complexity of understanding and querying these logs increases
Solution Approach 1:
The patent creates a universal natural language interface that works across multiple security products with different proprietary log structures. The system handles various log formats (Windows Event Logs, Linux syslog, Apache access logs, etc.) through a single unified interface, allowing analysts to query diverse security data sources without needing to understand each proprietary structure
Solution Approach 2:
The natural language processing system acts as a mediator that translates between diverse proprietary log structures and a unified query interface. Analysts interact with a single standardized natural language interface regardless of which security product or log format they are querying, while the system handles the complexity of mapping to appropriate log structures
3Productivity
If analysts learn proprietary KQL syntax, then effective security investigation is enabled, but the barrier to entry for junior analysts increases
Solution Approach 1:
The patent replaces the need for analysts to manually learn and apply complex KQL syntax with an automated natural language processing system. The system handles query construction, optimization, and execution automatically, maintaining security investigation effectiveness while making the process accessible to analysts with no prior knowledge of proprietary query languages
Data Source
AI summary
A computer-implemented method of generating a security language query from a user input query includes receiving, at a computer system, an input security hunting user query indicating a user intention; selecting, using a trained machine learning model and based on the input security hunting query, an example user security hunting query and corresponding example security language query; generating, using the trained machine learning model, query metadata from the input security hunting query; generating a prompt, the prompt comprising: the input security hunting user query; the selected example user security hunting query and the corresponding example security language query; and the generated query metadata; inputting the prompt to a large language model; receiving a security language query from the large language model corresponding to the input security hunting query reflective of the user intention.