Network security text-to-kql query generation method and system based on pattern screening and semantic verification, and storage medium

Abstract

The application provides a network security Text-to-KQL query generation method and system based on pattern screening and semantic verification and a storage medium, and the method comprises the following steps: S1, data input and task definition; receiving a user natural language query Q and a whole database pattern S, wherein Q is a user natural language retrieval / analysis intention, and S represents a set of all tables and fields in the system; S2, pattern optimization based on a hybrid attention model; S3, candidate KQL generation based on few-shot adaptation; and S4, KQL verification and selection based on execution signature and reasoning. The application has the beneficial effects that the application solves the problem of how to accurately and efficiently convert a natural language into a KQL query statement, reduces the technical threshold of a security analyst in a data query process, improves the efficiency of data query and analysis, and enhances the ability and response speed of network security protection.

Description

Technical Field

[0001] This invention relates to the intersection of network security and data query technology, and in particular to a network security Text-to-KQL query generation method, system, and storage medium based on pattern filtering and semantic verification. Background Technology

[0002] In the field of cybersecurity, security analysts often need to process large amounts of security log data, querying this data to perform critical tasks such as threat hunting, incident investigation, and behavioral analysis. However, traditional structured query languages ​​such as KQL, due to their complex syntax and high learning curve, pose a significant barrier to use for non-technical personnel.

[0003] Research on Text-to-QL (Text-to-QL) is relatively mature in the field of general-purpose databases. Based on executable, cross-domain, large-scale benchmarks, it has driven advancements in models regarding complex semantic understanding, cross-database generalization, and robustness to noise. Unlike relational SQL, cybersecurity analysis widely uses Kusto Query Language (KQL), which focuses on pipeline operations, time series analysis, text parsing, and semi-structured (JSON / dynamic) processing. It is particularly suitable for threat hunting and incident investigation on platforms such as Sentinel / Log Analytics using massive amounts of telemetry and log data.

[0004] In the direction of Text-to-KQL, the paper NL2KQL[1] has made preliminary explorations and adopted strategies such as schema screening, few-sample driven generation and KQL refinement to migrate the general Text-to-SQL method to the security scenario. The patent "Query statement generation method, device and equipment based on Text2SQL" performs inverse normalization reconstruction of the database through the schema decomposition evaluation module, obtains the reconstructed schema set, and then matches it with the problem features to generate SQL, solving the problem of poor SQL generation quality and low efficiency in complex database patterns. However, this method is still a static and one-time structural reorganization, rather than the dynamic and hierarchical screening based on query intent in your invention. At the same time, it does not involve semantic consistency verification after execution.

[0005] Despite the foundation laid by the above work, Text-to-QL for cybersecurity analytics still faces the following key technical bottlenecks:

[0006] (1) Redundancy of schema and excessively long context: Security platforms typically contain a large number of event types and fields, and different log sources are highly heterogeneous in terms of field naming, type and semantics. Directly exposing the complete schema to the model can easily lead to excessive context and injection of irrelevant noise, which significantly reduces the generation quality and stability. There is an urgent need for efficient schema refinement and correlation localization mechanisms to focus on a small number of related tables / columns and context knowledge.

[0007] (2) Insufficient domain adaptation and lack of syntactic robustness: General-purpose LLMs have limited coverage of specialized syntax and operation habits such as KQL. When faced with pipelined operations, time windows, dynamic fields and parsing functions, they are prone to syntax errors, misreferences of fields and inefficient query patterns, which affect their usability in practical scenarios (threat hunting, source tracing, lateral movement analysis).

[0008] (3) Semantic-Execution Mismatch Risk: Using "executability" as the optimization goal alone may overlook semantic correctness. In security analysis that emphasizes behavioral patterns and temporal correlation, semantic bias can lead to misleading results, false positives / false negatives, and increased investigation costs. Therefore, it is necessary to combine domain semantic consistency verification with alignment verification based on normalized view (ASIM). Summary of the Invention

[0009] To address the problems in existing technologies, this invention provides a network security Text-to-KQL query generation method based on pattern filtering and semantic verification, comprising:

[0010] Step S1: Data input and task definition; Receive the user's natural language query Q and the entire database schema S, where Q is the user's natural language retrieval / analysis intent, and the database schema S represents the collection of all tables and fields in the system;

[0011] Step S2: Pattern optimization based on a hybrid attention model; first, table-level attention filtering is performed, then column-level attention refinement is performed within the selected table, and the two are combined for the final column selection, forming the final pattern subset S. * ;

[0012] Step S3: Generate candidate KQL based on few-sample adaptive methods; filter from the pre-built example library E and Q, S * Related examples were provided, and the final prompt words were constructed based on the concept of thought chain to guide the model to generate multiple candidate KQLs;

[0013] Step S4: KQL verification and selection based on execution signature and reasoning.

[0014] As a further improvement of the present invention, step S2 further includes:

[0015] Step S21: For each table With column Computational domain semantic embedding, where It is a table The vector representation is obtained by concatenating the table name, table description, and example rows and inputting them into the semantic encoder; This is a list The vector representation is obtained by concatenating the column name, column type, column description, and sample value and then inputting them into SecBERT;

[0016] Step S22: Calculate the query embedding , This represents a parameterized vector embedding function, where Θ represents the learnable parameters of the model, and different vector models correspond to different values ​​of Θ.

[0017] Step S23: Calculate the table-level attention score:

[0018] ,

[0019] Indicates query With table The table-level relevance, i.e., the attention distribution value; The inner product represents similarity; Indicates the embedding dimension; Indicates indexes on all tables Perform normalization to obtain the table-level attention probability distribution;

[0020] Step S24: Column-level attention score, including table-level information as prior:

[0021] ,

[0022] Indicates query With column Column-level relevance; Indicates an index on all columns k After normalization, we obtain a table-level attention probability distribution, where q is the query vector, representing the embedding representation of the user-input query. This is a list k The embedded vector representation contains the semantic information of that column. Represents the query vector q and the column vector The dot product of these numbers measures their similarity; log represents...

[0023] Introduce table-level priors into column-level scoring;

[0024] Step S25: Calculate the query dependency gating weight and merge it with the column score:

[0025] ,

[0026] ,

[0027] σ() represents the Sigmoid function, which maps the input to the (0,1) interval so that the output can be used as a weight; and Indicates trainable parameters, Represents a vector. Indicates parameters; indicates average pooling of the query embedding; indicates scalar gating weights, determined by query features, used to balance the importance of table-level and column-level attention. This represents the final column-related score, used for sorting and selection; Column-level attention score, representing the relationship between query q and column. Direct correlation; : Table-level attention score, representing the relationship between query q and column Relevance of the table to which it belongs; Represents column Parent table index ;

[0028] Step S26: Hierarchical selection strategy;

[0029] First select Each table, based on Sort, This indicates a threshold or upper limit for the number of tables to be selected.

[0030] Next, for each selected table, select... Each column, based on Sort, This represents a threshold indicating the number of columns to select in each selected table;

[0031] Final output sub-pattern ,in T For the set of selected tables, C For the set of selected columns, K This refers to KQL-specific knowledge fragments.

[0032] As a further improvement of the present invention, step S3 further includes:

[0033] Step S31, Example Library Construction and Indexing: Before calculating relevance, the example library E is structurally constructed and preprocessed;

[0034] Step S32: Dynamic example retrieval and filtering;

[0035] Step S33: Construct parallel thought chain prompts; construct two independent prompt templates, each corresponding to one of the two thought chain strategies;

[0036] Step S34: Generate a candidate KQL set.

[0037] As a further improvement of the present invention, step S31 includes:

[0038] Step S311, Example Structure Definition: Each example It is a quintuple: ,in, It is a natural language query text; This refers to the original complete or sub-database schema corresponding to the query; It is a correct KQL code that has been manually verified or generated and audited by a top-tier model; This is a problem type label used for coarse screening; These labels indicate difficulty levels and are categorized based on query complexity.

[0039] Step S312, Example generation method: First, based on the de-identified real security operation data, security experts write seed question-answer examples. Then, a pre-trained large language model with advanced natural language understanding and code generation capabilities is called to generate examples in batches with task description + schema as input. The task description includes KQL syntax specifications. Then, it is cross-reviewed and corrected by security experts.

[0040] Step S313, Preprocessing and Indexing: For all examples and The text, which includes table name, column name, and description, uses [specific formatting information]. Calculate the embedding vector and store it in a vector database for subsequent fast similarity retrieval.

[0041] As a further improvement of the present invention, step S32 includes:

[0042] Step S321, Type-based coarse screening: Use a lightweight text classifier or keyword matching to perform preliminary type identification on user query Q to obtain the predicted type. Filter the type tags from sample library E. or related sample subsets This significantly narrows down the search scope;

[0043] Step S322, Multi-granularity similarity calculation: For For each example e in the query, calculate its relationship with the current query. Overall related score:

[0044] ,

[0045] Indicate Q and Q e Vector cosine similarity;

[0046] Pattern similarity Represented as:

[0047] ,

[0048] in, The set of table names for the current sub-pattern. This is the set of table names in the historical example pattern. For S * The set average or CLS vector of all column names and their described semantic embedding vectors. for This function calculates the set average or CLS vector of the semantic embedding vectors of all column names and their descriptions. `Jaccard()` calculates the Jaccard similarity coefficient for the set of table names, and `Cosine()` calculates the cosine similarity between the vectors of the two sets. This is a weighting coefficient that controls the balance between exact match of table names and semantic similarity of columns. These are weighting coefficients used to balance the importance of pattern similarity and query semantic similarity. , which are adjustable hyperparameters or learned through a small validation set;

[0049] Sim_query calculates Q and Q's semantic similarity using the same embedding model. e Vector cosine similarity

[0050] Degree, expressed as:

[0051] ,

[0052] in This indicates the calculation of the cosine similarity between vectors from two sets.

[0053] Step S323, Sampling by difficulty ratio: Based on the comprehensive relevant score from step S322 For the example subset of step S321 The examples in the dataset are sorted according to a preset ratio. From each difficulty partition (Deasy, Dmed, Dhard), the top-keasy, top-kmed, and top-khard examples are selected respectively to form a few-shot example set D. few=e1,e2,…,ek,k=keasy+kmed+khard, where Deasy, Dmed, and Dhard represent the easy, medium, and hard difficulty partitions, respectively, and top-keasy, top-kmed, and top-khard represent the candidate examples in the easy, medium, and hard difficulty partitions, ranked by score. The number of examples sorted from largest to smallest and then filtered from front to back.

[0054] As a further improvement of the present invention, step S33 includes:

[0055] Building basic instructions Includes system roles, task objectives, and the current database sub-schema S. * Detailed description;

[0056] Strategy 1, based on problem decomposition: Decompose complex problems into sub-problems, generate sub-queries step by step and merge them into a complete KQL;

[0057] Strategy 2, based on query planning hints: start by determining the main table and joins, then define filtering conditions, plan aggregation and grouping, and finally design the sorting output and translate it into KQL.

[0058] As a further improvement of the present invention, each example e in the example library E needs to be pre-annotated manually or automatically to generate corresponding decomposition ideas or query plan text based on its characteristics, for use in prompt word filling.

[0059] As a further improvement of the present invention, step S34 includes:

[0060] Step S341, Parallel Invocation: Q, S * and Fill in the two prompt word templates from step S33 respectively, and call the large model in parallel;

[0061] Step S342, Multiple Sampling: For each strategy, call the algorithm m times, each time using the set temperature parameter to introduce diversity, to obtain a candidate set. and , This represents the candidate set obtained based on Strategy 1 (problem decomposition and suggestion method). This represents the candidate set obtained based on strategy two (a suggestion method based on query planning);

[0062] Step S343, Preliminary filtering: Filtering the merged candidate set Perform basic cleaning, including:

[0063] Syntax checking: A fast check is performed using the syntax parser of the KQL interpreter, discarding candidates that cannot be parsed;

[0064] Schema Existence Check: Quickly scans the query statement to check if the table names and column names referenced in the query exist in the provided subschema S. * In the process, candidates that reference non-existent objects are marked or discarded;

[0065] Time field check: For queries that clearly require a time range, check whether the candidate string contains S. * The filtering conditions for the time field identified in the text;

[0066] Step S344, Output: After preliminary filtering, a set of valid candidate KQLs is obtained. , , N This represents the number of candidate KQLs in the final valid KQL set.

[0067] As a further improvement of the present invention, step S4 includes:

[0068] Step S41: Execute and generate a result signature in the sandbox environment;

[0069] Step S42: Fast deduplication and sorting based on signature;

[0070] Step S43: Construct a unified verification prompt for LLM arbitration; design a comprehensive prompt word so that LLM can act as a verification arbitrator;

[0071] Step S44: Analyze the decision and execute subsequent actions;

[0072] Step S45: Final output; output a structured result object.

[0073] As a further improvement of the present invention, step S41 specifically includes:

[0074] Execute each [task] in an isolated, read-only data sandbox. , It is the set of valid candidate KQLs in step S344. This refers to KQL. i The value range is [1, N], where N represents the number of candidate KQLs in the final valid KQL set. To avoid performance impact, the maximum number of rows returned by the query and the execution timeout period for the generated result signature are limited. Represented as:

[0075] ,

[0076] Time range : The minimum and maximum values ​​of the actual time field in the query results. If the query does not have a time field, then null is returned.

[0077] Output column role A list that records the name of each column in the result set and its inferred semantic role, inferred from the column name, data type, and context in the query;

[0078] Aggregate type List of aggregate functions and aggregate fields used in the query;

[0079] Grouping granularity The list of fields in the GROUP BY clause represents the grouping dimensions of the data;

[0080] Output shape : A pair representing the number of rows and columns in the result set;

[0081] Summary Statistics / Examples The first k rows of the result set, or the statistical values ​​of the key metric columns, can be anonymized;

[0082] Content Validation Calculate a hash value for the result set content to quickly determine whether two query results are completely identical;

[0083] Step S42 includes:

[0084] Step S421, Deduplication: Compare all candidates If multiple candidates have the same checksum, they are considered to have completely consistent execution results, and only the one with the shortest execution time is retained.

[0085] Step S422, Sorting: Sort the deduplicated candidate set according to the following priority to obtain the sorted candidate set. :

[0086] Primary sorting key: Whether the number of rows in the query results (row_count) is within a reasonable range;

[0087] Secondary sorting key: Query execution time, prioritizing faster queries;

[0088] The third sorting key: the complexity of the query statement, favoring more concise queries;

[0089] Step S44 includes:

[0090] Step S441: Call LLM to process the prompt constructed in step S43 and obtain structured output;

[0091] Step S442: Parse the output:

[0092] Accept: The process ends, and the accepted KQL is returned;

[0093] Regenerate: The improved instructions given by the LLM are fed back as new constraints to the generator in step S3. * Without changing the underlying principles, generate a new batch of candidate KQLs;

[0094] Re-select Schema: The missing element is fed back to the schema selector in step S2, the attention weight or filtering threshold is adjusted, and a new, complete sub-schema is selected. Then with

[0095] Re-enter step S3.

[0096] The present invention also discloses a network security Text-to-KQL query generation system based on pattern filtering and semantic verification, comprising: a memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0097] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0098] The beneficial effects of this invention are: This invention aims to overcome the shortcomings of the existing technology and provide a high-accuracy and high-reliability method for generating and verifying natural language to query language in the field of network security. The core objective is:

[0099] Precise pattern filtering: Through a hybrid attention mechanism, tables, fields, and domain knowledge fragments that are highly relevant to user queries are dynamically filtered from massive and heterogeneous network security data patterns, effectively solving the problems of context redundancy and noise;

[0100] Controllable query generation: Combining few-shot learning and parallel thinking chain strategies, the large model is guided to reason within the professional domain, generating grammatically correct and semantically close candidate queries that closely match the user's security analysis intent, reducing illusions;

[0101] Semantic consistency verification: Design a joint verification mechanism based on execution signature and large model inference. Without returning all result data, automatically evaluate whether the execution results of candidate queries truly answer the user's question at the semantic level, ensuring the correctness of the analysis results.

[0102] Enhance system robustness: Improve the system's fault tolerance to complex queries and potential erroneous inputs through mechanisms such as sandbox execution and iterative repair, and provide interpretable evidence for the output. Attached Figure Description

[0103] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0104] Parameter explanation:

[0105] SecBERT is a pre-trained model based on BERT, specifically designed for text in the cybersecurity field, aiming to improve the performance of related tasks by learning cybersecurity knowledge.

[0106] like Figure 1 As shown, this invention discloses a network security Text-to-KQL query generation method based on pattern filtering and semantic verification, including the following steps:

[0107] Step S1: Data input and task definition;

[0108] Receive user natural language query Q and the entire database schema S.

[0109] Explanation: Q represents the user's natural language search / analysis intent (e.g., "Which IPs triggered a large number of failed login attempts in the past 24 hours?"); S represents the collection of all tables and fields in the system (including table descriptions, field names, field types, sample values, and other metadata).

[0110] Step S2: Pattern optimization based on a hybrid attention model; first, perform table-level attention filtering, then refine column-level attention within the selected table, and combine the two for the final column selection to form the final pattern subset.

[0111] Schema optimization avoids including the entire schema in the prompt, which would lead to excessively long and noisy context, by filtering database subschemas that are strongly related to the problem. It also solves the problems of schema redundancy and context overload by accurately filtering a subset of schemas that are relevant to natural language queries.

[0112] Step S2 includes:

[0113] Step S21: For each table With column The semantic embedding of the computation domain is calculated using the following formula:

[0114] ,

[0115] in It is a table The vector representation is obtained by concatenating the table name, table description, and example rows and inputting them into the semantic encoder; This is a list The vector representation is obtained by concatenating the column name, column type, column description, and sample value and then inputting them into SecBERT; This represents a parameterized vector embedding function, where Θ represents the learnable parameters of the model (such as the weight matrix and attention mechanism parameters). Different vector models (SecBERT, LLM-embedder, etc.) correspond to different values ​​of Θ, reflecting the flexibility of the model.

[0116] Step S22: Calculate the query embedding ;

[0117] Step S23: Calculate the table-level attention score:

[0118] ,

[0119] Indicates query (There is only one query Q here) and table The table-level relevance, i.e., the attention distribution value (attention distribution value); The inner product represents similarity; Indicates the embedding dimension (used to scale the inner product to stabilize the softmax). Indicates indexes on all tables Perform normalization to obtain the table-level attention probability distribution;

[0120] Step S24: Column-level attention score (including table-level information as prior):

[0121] ,

[0122] Indicates query With column Column-level relevance; Indicates an index on all columns k After normalization, we obtain a table-level attention probability distribution, where q is the query vector, representing the embedding representation of the user-input query. This is a list k The embedded vector representation contains the semantic information of that column. Represents the query vector q and the column vector The dot product of these numbers is used to measure their similarity. Represents column Parent table index Therefore, log Introducing table-level priors into column-level scoring; this form is equivalent to amplifying the column scores from highly correlated tables based on column similarity (the probability values ​​are numerically fused by adding logarithms).

[0123] Step 25: Calculate query dependency gating weights and merge column scores:

[0124] ,

[0125] ,

[0126] σ represents the scalar gating weight, determined by the query features, used to balance the importance of table-level and column-level attention; σ() represents the Sigmoid function, which maps the input to the (0,1) interval, making the output usable as a weight. and Represents trainable parameters (vector and bias); This indicates that average pooling is performed on the query embedding. If (q) is a sequence representation, a fixed-dimensional vector is obtained. This indicates the final column-related score (used for sorting and selection); Column-level attention score, representing the relationship between query q and column. Direct correlation; : Table-level attention score, representing the relationship between query q and column The relevance of the table to which it belongs.

[0127] Step S26: Hierarchical selection strategy; (two-stage)

[0128] First select Each table (based on) (Sort), This indicates a threshold or upper limit for the number of tables to be selected.

[0129] Next, for each selected table, select... Columns (based on) (Sort);

[0130] Final output sub-pattern ,in T C is the set of selected tables, C is the set of selected columns, and K is a KQL-specific knowledge fragment (such as time field priority, commonly used aggregate function templates, etc., which can be read from documents or expert knowledge bases and appended to the prompt words of the large model).

[0131] Step S3: Generate candidate KQL based on few-sample adaptive testing;

[0132] Problems and sub-patterns S *Directly feeding KQL data into a large model can lead to significant illusions. Large models excel at online learning, meaning they generate examples through learning during the inference phase, which greatly reduces these illusions. This stage involves filtering from a pre-built example library and... S * The relevant examples are used, and the final prompt words are constructed based on the concept of CoT (Concept of Thought), which guides the model to generate multiple candidate KQLs;

[0133] Step S3 includes:

[0134] Step S31, Example Library Construction and Indexing: Before calculating relevance, the example library E is structurally constructed and preprocessed;

[0135] Step S31 includes:

[0136] Step S311, Example Structure Definition: Each example It is a quintuple: ,in, It is a natural language query text; It is the original complete or sub-database schema (tables, columns, and descriptions) corresponding to the query. It is a correct KQL code that has been manually verified or generated and audited by a top-tier model; These are issue type tags (such as: login anomaly, traffic statistics, threat tracing, data pivot, etc.) used for coarse screening; The difficulty level is indicated by labels (Easy, Medium, Hard), which are based on the complexity of the query (such as the number of tables involved, JOIN operations, nested subqueries, window functions, etc.).

[0137] Step S312, Example generation method: Based on real security operation data (after anonymization), security experts then write seed question-answer examples, and then use top models such as GPT-4 and Claude-3 to generate them in batches with "task description (including KQL syntax specification) + schema" as input, and then cross-review and correction by security experts.

[0138] The size of the example library (e.g., |E|≥500) can improve coverage, but quality (correctness, diversity) takes far greater priority than sheer quantity.

[0139] Step S313, Preprocessing and Indexing: For all examples and The text (table name, column name, description), which includes the table name, column name, and description, respectively using... Calculate the embedding vector and store it in a vector database for subsequent fast similarity retrieval.

[0140] Step S32: Dynamic example retrieval and filtering;

[0141] Step S32 includes:

[0142] Step S321, Type-based coarse screening: First, a lightweight text classifier (or keyword matching) is used to perform preliminary type identification on the user query Q to obtain the predicted type. Quickly filter out type tags from example library E. or related sample subsets This significantly narrows down the search scope;

[0143] Step S322, Multi-granularity similarity calculation: For For each example e in the query, calculate its relationship with the current query. Overall related score:

[0144] ,

[0145] Indicate Q and Q e Vector cosine similarity;

[0146] Pattern similarity Represented as:

[0147] ,

[0148] The symbol is interpreted as follows:

[0149] The set of table names for the current sub-pattern. This is the set of table names in the historical example pattern. For S * The set average or CLS vector of all column names and their described semantic embedding vectors. for This function calculates the set average or CLS vector of the semantic embedding vectors of all column names and their descriptions. `Jaccard()` calculates the Jaccard similarity coefficient for the set of table names, and `Cosine()` calculates the cosine similarity between the vectors of the two sets. This is a weighting coefficient that controls the balance between exact match of table names and semantic similarity of columns. These are weighting coefficients used to balance the importance of pattern similarity and query semantic similarity. , which are adjustable hyperparameters or learned through a small validation set;

[0150] Sim_query calculates Q and Q's semantic similarity using the same embedding model. e The vector cosine similarity is expressed as:

[0151] ,

[0152] in This indicates the calculation of the cosine similarity between vectors from two sets.

[0153] Step S323, Sampling by Difficulty Ratio: Based on the comprehensive relevant score from step S322 For the example subset of step S321 The examples in the dataset are sorted according to a preset ratio (e.g., easy : kmed : khard = 2 : 2 : 1). From each difficulty partition Deasy, Dmed, and Dhard, the top-keasy, top-kmed, and top-khard examples are selected respectively to form a few-shot example set D. few =e1,e2,…,ek,k=keasy+kmed+khard, with a suggested total of 3-8, where Deasy, Dmed, and Dhard represent the easy, medium, and hard difficulty partitions respectively, and top-keasy, top-kmed, and top-khard represent the candidate examples in the easy, medium, and hard difficulty partitions, ranked by score. The number of examples sorted from largest to smallest and then filtered from front to back.

[0154] Step S33: Constructing CoT (Content of Parallel Thought Chains) prompts;

[0155] Construct two independent prompt word templates, each corresponding to one of the two CoT strategies;

[0156] Step S33 includes:

[0157] First, construct the basic instructions. This includes the system role ("You are a cybersecurity data analysis expert, proficient in KQL"), the task objective ("Translate natural language questions into correct and efficient KQL queries"), and the current database subschema S. * Detailed description;

[0158] Strategy 1, Problem Decomposition Hint: Decompose complex problems into subproblems, generate subqueries step by step, and merge them into a complete KQL. A sample hint template is shown below:

[0159] {I base}

[0160] Please follow these steps of thinking:

[0161] 1. Understanding the problem: Paraphrase the user's intent in one sentence.

[0162] 2. Decompose sub-goals: Break down complex problems into several logical sub-goals that are executed sequentially.

[0163] 3. Mapping Data: Map each subtarget to a specific table and field in `S*`.

[0164] 4. Write subqueries: Write a piece of KQL code for each sub-target.

[0165] 5. Combine complete queries: Combine all subqueries into the final KQL.

[0167] Reference example:

[0168] {e1.query} ->{e1.CoT_analysis} ->{e1.KQL}

[0169] {e2.query} ->{e2.CoT_analysis} ->{e2.KQL} ...

[0172] Now, please analyze and generate the following:

[0173] Question: {Q}

[0174] Please begin your step-by-step thinking:

[0175] Strategy 2: Query planning-based suggestion method

[0176] {I base}

[0177] Please follow these steps of thinking:

[0178] 1. Determine the main table and joins: Identify the core fact table and determine which dimension tables need to be joined.

[0179] 2. Define filtering conditions: Extract filtering conditions such as time range, IP address, and status code from the question.

[0180] 3. Planning Aggregation and Grouping: Clarify which metrics (count, sum, average, etc.) need to be calculated and the grouping dimensions.

[0181] 4. Design Sorting and Output: Determine the sorting method for the results and the columns to be output.

[0182] 5. Translate to KQL: Translate the above plan into KQL syntax section by section.

[0184] Reference example:

[0185] {e3.query} ->{e3.plan} ->{e3.KQL}

[0186] {e4.query} ->{e4.plan} ->{e4.KQL} ...

[0189] Now, please analyze and generate the following:

[0190] Question: {Q}

[0191] Please begin your query planning and generation:

[0192] Note: Each example e in the example library E needs to be manually or automatically annotated in advance to generate a corresponding decomposition idea or query plan text based on its characteristics, which will be used for prompt word filling.

[0193] Step S34: Generate a candidate KQL set.

[0194] Step S34 includes:

[0195] Step S341, Parallel Invocation: Q, S * and Fill in the two prompt word templates from step S33 respectively, and call the large model in parallel;

[0196] Step S342, Multiple Sampling: For each strategy, call the algorithm m times (e.g., m=3), each time using a certain temperature parameter (temperature=0.7) to introduce diversity, and obtain the candidate set. and , This represents the candidate set obtained based on Strategy 1 (problem decomposition and suggestion method). This represents the candidate set obtained based on strategy two (a suggestion method based on query planning);

[0197] Step S343, Preliminary filtering: Filtering the merged candidate set Perform basic cleaning:

[0198] Syntax checking: A fast check is performed using the syntax parser of the KQL interpreter, discarding candidates that cannot be parsed;

[0199] Schema Existence Check: Quickly scans the query statement to check if the table names and column names referenced in the query exist in the provided subschema S. * In the process, candidates that reference non-existent objects are marked or discarded;

[0200] Time field check: For queries that clearly require a time range, check whether the candidate string contains S. * The filtering conditions for the time field identified in the text;

[0201] Step S344, Output: After preliminary filtering, a set of valid candidate KQLs is obtained. , , N This represents the number of candidate KQLs in the final valid KQL set.

[0202] Step S4: KQL verification and selection based on execution signature and reasoning.

[0203] The goal of this stage is to safely and efficiently select the KQL with the most correct semantics from C_final, avoiding the overhead and risks of executing all queries and comparing the complete results.

[0204] Step S4 includes:

[0205] Step S41: Execute and generate a result signature in the sandbox environment;

[0206] Execute each [task] in an isolated, read-only data sandbox. , It is the set of valid candidate KQLs in step S344. This refers to KQL. i The value range is [1, N], where N represents the number of candidate KQLs in the final valid KQL set. To avoid performance impact, the maximum number of rows returned by the query (e.g., 100 rows) and the execution timeout (e.g., 30 seconds) can be limited. The generated result signature... Represented as:

[0207] ,

[0208] The symbols are explained as follows:

[0209] (Time Range): The minimum and maximum values ​​of the actual time field in the query results. If the query does not have a time field, it will be null.

[0210] (Output column roles): A list that records the name of each column in the result set and its inferred semantic role (e.g., grouping key, measure, timestamp, filter condition, etc.), inferred from the column name, data type, and context in the query;

[0211] (Aggregation type): A list of aggregate functions used in the query (e.g., count(), sum(bytes)) and the aggregate fields;

[0212] (Grouping granularity): The list of fields in the GROUP BY clause, representing the grouping dimension of the data;

[0213] (Output shape): A tuple (row_count, col_count) representing the number of rows and columns in the result set;

[0214] (Summary Statistics / Example): The contents of the first k rows of the result set (e.g., k=3), or the statistical values ​​of key measure columns (e.g., count). This part can be anonymized.

[0215] (Content checksum): Calculate a hash value for the result set content (such as all values ​​concatenated after being sorted by row) to quickly determine whether two query results are completely identical;

[0216] Step S42: Fast deduplication and sorting based on signature;

[0217] Step S42 includes:

[0218] Step S421, Deduplication: Compare all candidates If multiple candidates have the same checksum, they are considered to have completely consistent execution results, and only the one with the shortest execution time is retained.

[0219] Step S422, Sorting: Sort the deduplicated candidate set according to the following priority to obtain the sorted candidate set. :

[0220] Primary sorting key: Whether the number of rows in the query results (row_count) is within a reasonable range (e.g., non-zero and not too large);

[0221] Secondary sorting key: Query execution time, prioritizing faster queries;

[0222] The third sorting key is the complexity of the query statement (such as length and nesting depth), which tends to favor more concise queries.

[0223] Step S43: Construct a unified verification prompt for LLM arbitration; design a comprehensive prompt word so that the LLM can act as a verification arbitrator, where the input content is:

[0224] Original question from the user: {Q}

[0225] Available tables / columns (subschemas) in the database: {S*}

[0227] Candidate KQLs and their execution summaries (sorted by priority):

[0228] Candidate 1 [Highest Priority]:

[0229] KQL: {c_1}

[0230] Results Summary: Time Range = {T_1}, Output Columns = {Cols_1}, Aggregation = {Agg_1}, Grouping = {Group_1} The resulting shape is {Shape_1}, and the sample is {Sample_1}.

[0231] ---

[0232] Candidate 2:

[0233] KQL: {c_2}

[0234] Results Summary: Time Range = {T_2}, Output Columns = {Cols_2}, Aggregation = {Agg_2}, Grouping = {Group_2} The resulting shape is {Shape_2}, and the sample is {Sample_2}.

[0235] ---

[0236] ... (List the first M items in C_sorted, e.g., M=3-5 items)

[0238] Your task is to act as the final arbitrator and select the KQL that best aligns with the user's intent. Please consider the following steps:

[0239] 1. Understanding Intent: Accurately understand the core needs of the user's question "{Q}".

[0240] 2. Evaluate candidates: Check the "Results Summary" of each candidate KQL in turn.

[0241] - Check if the time range matches the time mentioned in the question (such as "the past 24 hours").

[0242] - Check if the output columns and aggregations directly answer the core question (e.g., the question asks "which IPs", the results...). Does the IP column serve as a grouping key? The question asks for "a large number of attempts," and the results include count aggregation.

[0243] - Check if the grouping granularity is reasonable.

[0244] - Check if the result sample is semantically reasonable (e.g., the count of failed login attempts should not be negative).

[0245] 3. Make a decision:

[0246] - If a candidate fully meets the requirements, output `Decision: Accept` and the candidate's number.

[0247] - If all candidates have serious semantic defects (e.g., completely wrong time windows, missing key aggregations, etc.), (Grouping dimension error), identify the root cause, and output `Decision: Regenerate` and specific improvement instructions (e.g.: "The failure count must be grouped by src_ip, not by dest_port."

[0248] - If all candidates are missing a key table or field, resulting in an inability to correctly express the query, output `resolve`. Solution: Re-select Schema and missing elements (e.g., "need to include elements from the 'user_logon_failure' table"). (The 'reason_code' field).

[0249] 4. Provide an explanation: Explain the reasoning behind the decision in one sentence.

[0250] Step S44: Analyze the decision and execute subsequent actions;

[0251] Step S44 includes:

[0252] Step S441: Call LLM to process the prompts constructed in step S43 and obtain structured output;

[0253] Step S442: Parse the output:

[0254] Accept: The process ends, and the accepted KQL is returned;

[0255] Regenerate: The improved instructions given by the LLM are used as new constraints and fed back to the generator in step S3. * If the process remains unchanged, a new batch of candidate KQLs is generated (usually only one iteration is performed).

[0256] Re-select Schema: The missing element is fed back to the schema selector in step S2, the attention weights or filtering thresholds are adjusted, and a new, complete sub-schema is selected. Then with Re-enter step S3 process.

[0257] Step S45: Final output; output a structured result object.

[0258] {

[0259] "status": "success | regenerated | schema_updated",

[0260] "final_kql": "The selected KQL statement",

[0261] "signature": { ...}, / / The final KQL signature

[0262] "explanation": "The reasons for selection / rejection provided by the LLM, used for interpretability demonstration."

[0263] "alternative_candidates": [...] / / (Optional) Signature digests of other high-quality candidates. For user reference

[0264] }

[0265] This invention aims to solve the problem of how to accurately and efficiently convert natural language into KQL query statements. Through innovative methods and systems, it lowers the technical threshold for security analysts in the data query process, improves the efficiency of data query and analysis, and enhances network security protection capabilities and response speed to cope with increasingly complex network security threats.

[0266] The present invention also discloses a network security Text-to-KQL query generation system based on pattern filtering and semantic verification, comprising: a memory, a processor, and a computer program stored on the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0267] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0268] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for generating network security Text-to-KQL queries based on pattern filtering and semantic verification, characterized in that, include: Step S1: Data input and task definition; Receive the user's natural language query Q and the entire database schema S, where Q is the user's natural language retrieval / analysis intent and the database schema S represents the collection of all tables and fields in the system; Step S2: Pattern optimization based on a hybrid attention model; first, table-level attention filtering is performed, then column-level attention refinement is performed within the selected table, and the two are combined for the final column selection, forming the final pattern subset S. * ; Step S3: Generate candidate KQL based on few-sample adaptive methods; filter from the pre-built example library E and Q, S * Related examples were provided, and the final prompt words were constructed based on the concept of thought chain to guide the model to generate multiple candidate KQLs; Step S4: KQL verification and selection based on execution signature and reasoning; Step S3 further includes: Step S31, Example Library Construction and Indexing: Before calculating relevance, the example library E is structurally constructed and preprocessed; Step S32: Dynamic example retrieval and filtering; Step S33: Construct parallel thought chain prompts; construct two independent prompt templates, each corresponding to one of the two thought chain strategies; Step S34: Generate a candidate KQL set; Step S34 includes: Step S341, Parallel Invocation: Q, S * and Fill in the two prompt word templates from step S33 respectively, and call the large model in parallel; Step S342, Multiple sampling: For each strategy, call the algorithm m times, each time using the set temperature parameter to introduce diversity, to obtain a candidate set. and , This represents the candidate set obtained based on strategy one. This represents the candidate set obtained based on strategy two; Step S343, Preliminary filtering: Filtering the merged candidate set Perform basic cleaning, including: Syntax checking: A fast check is performed using the syntax parser of the KQL interpreter, discarding candidates that cannot be parsed; Schema Existence Check: Quickly scans the query statement to check if the table names and column names referenced in the query exist in the provided subschema S. * In the process, candidates that reference non-existent objects are marked or discarded; Time field check: For queries requiring a time range, check if the candidate string contains S. * The filtering conditions for the time field identified in the text; Step S344, Output: After preliminary filtering, a set of valid candidate KQLs is obtained. , , N This represents the number of candidate KQLs in the final valid KQL set.

2. The network security Text-to-KQL query generation method according to claim 1, characterized in that, Step S2 further includes: Step S21: For each table With column Computational domain semantic embedding, where It is a table The vector representation is obtained by concatenating the table name, table description, and example rows and inputting them into the semantic encoder; This is a list The vector representation is obtained by concatenating the column name, column type, column description, and sample value and then inputting them into SecBERT; Step S22: Calculate the query embedding , This represents a parameterized vector embedding function, where Θ represents the learnable parameters of the model, and different vector models correspond to different values ​​of Θ. Step S23: Calculate the table-level attention score: ， Indicates query With table The table-level relevance, i.e., the attention distribution value; The inner product represents similarity; Indicates the embedding dimension; Indicates indexes on all tables Perform normalization to obtain the table-level attention probability distribution; Step S24: Column-level attention score, including table-level information as prior: ， Indicates query With column Column-level relevance; Indicates an index on all columns k After normalization, we obtain a table-level attention probability distribution, where q is the query vector, representing the embedding representation of the user-input query. This is a list k The embedded vector representation contains the semantic information of that column. Represents the query vector q and the column vector The dot product of these numbers measures their similarity; log represents... Introduce table-level priors into column-level scoring; Step S25: Calculate the query dependency gating weight and merge it with the column score: ， ， This represents a scalar gating weight, determined by query characteristics, used to balance the importance of table-level and column-level attention. σ() represents the Sigmoid function, which maps the input to the (0,1) interval so that the output can be used as a weight; and Indicates trainable parameters, Represents a vector. Indicates parameters; This indicates that the query embedding is averaged. This represents the final column-related score, used for sorting and selection; Column-level attention score, representing the direct relevance of query q to the column; : Table-level attention score, representing the relationship between query q and column Relevance of the table to which it belongs; Represents column Parent table index ; Step S26: Hierarchical selection strategy; First select Each table, based on Sort, This indicates a threshold or upper limit for the number of tables to be selected. Next, for each selected table, select... Each column, based on Sort, This represents a threshold indicating the number of columns to select in each selected table; Final output sub-pattern ,in T For the set of selected tables, C For the set of selected columns, K This refers to KQL-specific knowledge fragments.

3. The network security Text-to-KQL query generation method according to claim 1, characterized in that, Step S31 includes: Step S311, Example Structure Definition: Each example It is a quintuple: ,in, It is a natural language query text; This refers to the original complete or sub-database schema corresponding to the query; It is a correct KQL code that has been manually verified or generated and audited by a top-tier model; This is a problem type label used for coarse screening; These labels indicate difficulty levels and are categorized based on query complexity. Step S312, Example generation method: First, based on the de-identified real security operation data, security experts write seed question-answer examples. Then, a pre-trained large language model with advanced natural language understanding and code generation capabilities is called to generate examples in batches with task description + schema as input. The task description includes KQL syntax specifications. Then, it is cross-reviewed and corrected by security experts. Step S313, Preprocessing and Indexing: For all examples and The text, which includes table name, column name, and description, uses [specific formatting information]. Calculate the embedding vector and store it in a vector database for similarity retrieval. This represents a parameterized vector embedding function, where Θ represents the learnable parameters of the model, and different vector models correspond to different values ​​of Θ.

4. The network security Text-to-KQL query generation method according to claim 1, characterized in that, Step S32 includes: Step S321, Type-based coarse screening: Use a lightweight text classifier or keyword matching to perform preliminary type identification on user query Q to obtain the predicted type. Filter the type tags from sample library E. or related sample subsets Narrowing the search scope; Step S322, Multi-granularity similarity calculation: For For each example e in the query, calculate its relationship with the current query. Overall related score: , Indicate Q and Q e Vector cosine similarity; Pattern similarity Represented as: ， in, The set of table names for the current sub-pattern. This is the set of table names in the historical example pattern. For S * The set average or CLS vector of all column names and their described semantic embedding vectors. for This function calculates the set average or CLS vector of the semantic embedding vectors of all column names and their descriptions. `Jaccard()` calculates the Jaccard similarity coefficient for the set of table names, and `Cosine()` calculates the cosine similarity between the vectors of the two sets. This is a weighting coefficient that controls the balance between exact match of table names and semantic similarity of columns. It is a weighting coefficient used to balance the importance of pattern similarity and query semantic similarity. , which are adjustable hyperparameters or learned through a small validation set; Query semantic similarity Q and Q are computed using the same embedding model. e Vector cosine similarity Degree, expressed as: ， in This indicates the calculation of the cosine similarity between vectors from two sets. Step S323, Sampling by difficulty ratio: Based on the comprehensive relevant score from step S322 For the example subset of step S321 The examples in the dataset are sorted according to a preset ratio. From each difficulty partition (Deasy, Dmed, Dhard), the top-keasy, top-kmed, and top-khard examples are selected respectively to form a few-shot example set D. few =e1,e2,…,ek,k=keasy+kmed+khard, where Deasy, Dmed, and Dhard represent the easy, medium, and hard difficulty partitions, respectively, and top-keasy, top-kmed, and top-khard represent the candidate examples in the easy, medium, and hard difficulty partitions, ranked by score. The number of examples sorted from largest to smallest and then filtered from front to back.

5. The network security Text-to-KQL query generation method according to claim 1, characterized in that, Step S33 includes: Building basic instructions Includes system roles, task objectives, and the current database sub-schema S. * Detailed description; Strategy 1, based on problem decomposition: Decompose complex problems into sub-problems, generate sub-queries step by step and merge them into a complete KQL; Strategy 2, based on query planning hints: start by determining the main table and joins, then define filtering conditions, plan aggregation and grouping, and finally design the sorting output and translate it into KQL.

6. The network security Text-to-KQL query generation method according to claim 3, characterized in that, Each example e in the example library E needs to be pre-annotated manually or automatically to generate a corresponding decomposition idea or query plan text based on its characteristics, which will be used for prompt word filling.

7. The network security Text-to-KQL query generation method according to claim 1, characterized in that, Step S4 includes: Step S41: Execute and generate a result signature in the sandbox environment; Step S42: Fast deduplication and sorting based on signature; Step S43: Construct a unified verification prompt for LLM arbitration; design a comprehensive prompt word so that LLM can act as a verification arbitrator; Step S44: Analyze the decision and execute subsequent actions; Step S45: Final output; output a structured result object.

8. The network security Text-to-KQL query generation method according to claim 7, characterized in that, Step S41 specifically involves: Execute each [task] in an isolated, read-only data sandbox. , It is the set of valid candidate KQLs in step S344. This refers to KQL. i The value range is [1, N], where N represents the number of candidate KQLs in the final valid KQL set. To avoid performance impact, the maximum number of rows returned by the query and the execution timeout period for the generated result signature are limited. Represented as: , Time range : The minimum and maximum values ​​of the actual time field in the query results. If the query does not have a time field, then null is returned. Output column role A list that records the name of each column in the result set and its inferred semantic role, inferred from the column name, data type, and context in the query; Aggregate type List of aggregate functions and aggregate fields used in the query; Grouping granularity The list of fields in the GROUP BY clause represents the grouping dimensions of the data; Output shape : A pair representing the number of rows and columns in the result set; Summary Statistics / Examples The first k rows of the result set, or the statistical values ​​of the key metric columns, can be anonymized; Content Validation Calculate a hash value for the result set content to quickly determine whether two query results are completely identical; Step S42 includes: Step S421, Deduplication: Compare all candidates If multiple candidates have the same checksum, they are considered to have completely consistent execution results, and only the one with the shortest execution time is retained. Step S422, Sorting: Sort the deduplicated candidate set according to the following priority to obtain the sorted candidate set. : Primary sorting key: Whether the number of rows in the query results (row_count) is within a reasonable range; Secondary sorting key: Query execution time, prioritizing faster queries; The third sorting key: the complexity of the query statement, favoring more concise queries; Step S44 includes: Step S441: Call LLM to process the prompt constructed in step S43 and obtain structured output; Step S442: Parse the output: Accept: The process ends, and the accepted KQL is returned; Regenerate: The improved instructions given by the LLM are fed back as new constraints to the generator in step S3. * Without changing the underlying principles, generate a new batch of candidate KQLs; Re-select Schema: The missing element is fed back to the schema selector in step S2, the attention weight or filtering threshold is adjusted, and a new, complete sub-schema is selected. Then with Re-enter step S3.

9. A network security Text-to-KQL query generation system based on pattern filtering and semantic verification, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the method of any one of claims 1-8 when invoked by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the method according to any one of claims 1-8 when invoked by a processor.

Images