Generating secure language queries

Abstract

A computer-implemented method of generating a secure language query from a user input query includes receiving, at a computer system, an input secure search user query indicating a user intent; selecting, using the trained machine learning model and based on the input secure search query, an example user secure search query and a corresponding example secure language query; generating query metadata from the input secure search query using the trained machine learning model; generating a cue word, wherein the cue word comprises: inputting a security search user query; a selected example user secure search query and a corresponding example secure language query; and the generated query metadata; inputting the cue word into a large language model; a secure language query corresponding to an input secure search query reflecting a user intent is received from a large language model.

Description

Background Art

[0001] Security hunting or threat hunting involves actively searching for security threats to computer systems. One challenge for security analysts is to understand the structure and format of security data in order to be effective when performing security investigations. On average, companies use 20 different security products to protect and defend their assets and intellectual property. Most of these security products use their own proprietary log structure, which is difficult to decipher and understand, and requires a lot of time for analysts to become proficient in interrogation. Safety products (e.g. ) uses a query language called Kusto Query Language (KQL) to query these logs, which is often unfamiliar to junior analysts who have limited understanding of the relevant tables and schema definitions. Summary of the invention

[0002] According to one aspect disclosed herein, a computer-implemented method is provided, including: receiving an input safe search user query; selecting an example user safe search query and a corresponding example safe language query using a trained machine learning model and based on the input safe search query; generating query metadata from the input safe search query using the trained machine learning model; generating prompt words, the prompt words including: the input safe search user query; the selected example user safe search query and the corresponding example safe language query; and the generated query metadata; inputting the prompt words into a large language model; and receiving a safe language query corresponding to the input safe search query from the large language model.

[0003] Using a trained machine learning model to select examples (or “shots”) to include in prompt words and generate appropriate metadata (e.g., table schema data) helps provide reliable safe language queries that are closely aligned with user intent.

[0004] According to another aspect disclosed herein, a computer-implemented method is provided, comprising: receiving a training data set, the training data set comprising a plurality of user safety search queries, corresponding truth-value safety language queries, and corresponding query metadata; generating a plurality of detection prompt words from the training data set, wherein generating each detection prompt word comprises: selecting one of the plurality of user safety search queries as a subject of the detection prompt word; randomly selecting a plurality of user safety search queries and corresponding truth-value safety language queries as examples of the detection prompt word; inputting the generated detection prompt words into a large language model; receiving a safety language query corresponding to the subject of each detection prompt word as an output from the large language model; and inputting the received safety language query for the detection prompt word into a large language model. The query is compared with the corresponding true-value safe language query; based on the comparison, a detection score is calculated for each detection prompt word; based on the calculated detection score and the training data set, a machine learning model is trained, and the machine learning model is trained to: receive an input user safe search query; generate an output score that reflects the utility of each user safe search query and the corresponding true-value safe language query of the training data set to the input user safe search query; and based on the output score, select the user safe search query and the corresponding true-value safe language query of the training data set as examples of prompt words for input to a large language model; and generate query metadata for the prompt words for the large language model from the input safe search user query.

[0005] The generation of multiple probe cues is an effective way to evaluate the impact of including specific samples in the cue, which may be opaque given the nature of large language models. The probes enable the model to be trained to select samples for inclusion in the cue.

[0006] According to another disclosed aspect, a system is provided that includes a processor and a storage device including computer readable instructions that, when executed, cause the system to perform any of the methods discussed herein.

[0007] This summary is provided to introduce in simplified form a selection of concepts further described below in the detailed description. This summary is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. The claimed subject matter is also not limited to implementations that address any or all of the shortcomings noted herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] To aid understanding of the present disclosure and to show how embodiments may be implemented, reference is made to the accompanying drawings by way of example, in which:

[0009] Figure 1 is a schematic block diagram of an environment in which examples of the present disclosure operate;

[0010] Figure 2 is a schematic diagram of example prompt words for submitting to a large language model to generate a safe language query;

[0011] Figure 3 is a flow chart of a method for training a machine learning model to select samples and generate query metadata;

[0012] Figure 4 is a schematic block diagram illustrating a technique for forming a training data set;

[0013] Figure 5 is a schematic block diagram illustrating a technique for probing a large language model to generate training data for selecting samples;

[0014] Figure 6 It is shown that the use Figure 5 A graph of detection scores generated by the detection techniques;

[0015] Figure 7 is a schematic block diagram of an example machine learning model;

[0016] Figure 8 It is shown for training Figure 7 A schematic block diagram of an example technique for selecting samples using a machine learning model;

[0017] Fig. 9 It is shown for training Figure 7 A schematic block diagram of an example technique for using a machine learning model to generate query metadata;

[0018] Fig.10 is a schematic flow chart of an example method for generating a secure language query from a user input query using a trained machine learning model;

[0019] Fig.11 is a schematic block diagram of an example technique for selecting a sample;

[0020] Fig.12 is a schematic flow chart of an example process for generating a prompt word;

[0021] Fig.13 is a schematic flow chart of an example process for generating a plurality of cue words;

[0022] Fig.14 is a schematic block diagram illustrating an example technique for coordinating the generation of multiple cue words; and

[0023] Fig.15 is a schematic diagram of an example computing system. DETAILED DESCRIPTION

[0024] In general, examples of the present disclosure relate to generating queries in a safe query language (ie, a structured code-like language such as KQL) from unstructured user input (eg, in the form of a natural language query).

[0025] Recently, large language models (LLMs) have been developed using transformer architectures. Such LLMs are trained on very large amounts of data, including a variety of different datasets. For example, by Open GPT-3 (Generative Pre-trained Transformer 3) developed by AI has 175 billion parameters and is trained on 499 billion tokens. BERT (Bidirectional Encoder Representation from Transformers) developed by AI is an example of another LLM.

[0026] The diverse training and larger size of LLMs have led to some emerging properties and features that were not possible with previous models. One of these aspects is the concept of using natural language cues to ask the LLM to solve a task in a general way. These generally fall into the category of zero-shot, one-shot, or few-shot learning, where a “sample” is a number of labeled examples provided to the LLM as part of a natural language cue.

[0027] Thus, the present disclosure relates to generating prompt words for an LLM that include one or more examples (or "samples") and additional query metadata (e.g., an indication of a table schema and related tables). The samples and query metadata are generated by a trained machine learning model. The resulting prompt words can then be passed to the LLM, which returns a corresponding safe query language query. In some examples, multiple prompt words are generated and passed to the LLM, and a selection is made from the resulting safe language queries. Using a trained machine learning model to select samples and query metadata results in an accurate safe language query that closely corresponds to the intent of the original user input.

[0028] In one example, a machine learning model that generates samples (i.e., "single-shot" or "few-shot") for a cue word is trained using a detection process, where the detection cue words are generated from a training set. The detection cue words include various permutations of example samples that are passed to the LLM. The results for each detection cue word are compared to a known true value to generate a score. The model is then trained to select the example sample, or in examples where multiple samples are used, multiple samples. For example, the model can be trained to rank the example samples and select samples to include in the cue word accordingly. Query metadata is also learned from the training set.

[0029] The LLM provides a general application programming interface (API) for performing tasks including completion (i.e., completing prompt words to provide answers such as safe language queries). The API, and to some extent the LLM, is a black box, and it can be difficult to determine why the LLM returns the results it does, making it difficult to reliably program prompt words to return results in a predictable and expected manner. The use of the detection process results in a trained machine learning model that takes into account the characteristics of the LLM.

[0030] Figure 1 An example environment 1 in which examples of the present disclosure operate is shown to provide an overview of the components of the present disclosure.

[0031] Environment 1 includes a large language model (LLM) 301. LLM 301 is a trained language model based on a transformer deep learning network. LLM 301 is trained on a very large corpus (e.g., on the order of billions of tokens) and is a generative model that can generate text or data in response to receiving a prompt word. In particular, LLM 301 is capable of generating code in response to a prompt word. "Code" in this context includes query language.

[0032] An example of a suitable LLM 301 is the Open AI Codex model (https: / / openai.com / blog / openai-codex / ). The Codex model is a version of the GPT-3 model that is fine-tuned for code generation. However, various LLMs 301 may be employed in alternatives, which may or may not be specifically tuned for code generation. The techniques discussed herein effectively learn the characteristics of the underlying LLM 301 and are therefore particularly suitable for use with different LLMs 301.

[0033] The LLM 301 operates in a suitable computer system 300. For example, the LLM 301 is stored in a suitable data center, and / or as part of a cloud computing environment or other distributed environment. The LLM 301 is accessible via a suitable API, such as through a network connection.

[0034] Environment 1 includes a system 100 configured to train a machine learning model 130 to generate elements for inclusion in a prompt word input to LLM 301. The prompt word includes a user query, discussed in more detail below, which is converted by LLM 301 into a query in a safe language. For convenience, reference is made throughout this disclosure to a KQL query as an example of a query in a safe language. However, KQL is merely an example of a suitable safe language.

[0035] System 100 includes a processor 101 and a storage device 102. Processor 101 is configured to execute instructions stored in storage device 102 to perform the training method discussed herein. Storage device 102 also stores a training data set 110. The training operation of system 100 is represented by training module 120, which will be discussed in more detail below.

[0036] The system 100 for training the machine learning model 130 includes any suitable computer system. In one example, the system 100 can be a suitable high-performance computer or computer cluster. In other examples, the system 100 can be a server computer located in a data center, for example. Similarly, the system 100 can be a desktop or laptop computer, etc.

[0037] Environment 1 also includes system 200, which is configured to generate prompt words for LLM 301 using trained model 130. In other words, system 200 performs the reasoning time activities discussed herein. System 200 can also submit prompt words to LLM 301 to generate corresponding KQL. Accordingly, this means that KQL reflects the intention of the user who input the query. In other words, KQL will return results in response to the user's input query when executed.

[0038] The system 200 includes a processor 201 and a storage device 202. The processor 201 is configured to execute instructions stored in the storage device 202 to perform the reasoning method discussed herein. The storage device 202 also stores the trained model 130. The reasoning operation of the system 200 is represented by the reasoning module 210 and will be discussed in more detail below.

[0039] The system 200 also includes an access interface 220. In one example, the access interface 220 can take the form of a suitable API for receiving user queries from a user device (500, discussed below) and returning corresponding KQLs. In another example, the access interface 220 is a web interface configured to serve a web page via which a user can enter a user query and receive a corresponding KQL.

[0040] The environment also includes a system 500, which is a user system operated by an end user. The system 500 includes a processor 501 and a storage device 502. The system 500 has a user interface 503, which is configured to receive user input and display data to the user. The system 500 also includes a security system 504, which is configured to receive and execute KQL queries. For example, the security system 504 may include or The user can interact with the system 500 via the user interface 503 to input a user query. In some examples, the system 500 displays the corresponding KQL on the user interface 503. In some examples, the system 500 executes the corresponding KQL using the security system 504.

[0041] In some examples, system 100 and system 200 are the same system. That is, the same system can be used to train the model and for reasoning. In some examples, system 200 and system 500 are the same system, so that the system performing reasoning is the same system that includes security system 504 and receives user input.

[0042] Figure 2 An example prompt word 400 is shown that is submitted to the LLM 301 to generate a security query in the examples herein.

[0043] The prompt 400 includes a preamble 401, which generally summarizes the tasks required of the LLM 301. The preamble 401 indicates that the LLM 301 should generate a KQL query based on the input query (ASK).

[0044] In one example, preamble 401 is static. That is, it can be predetermined rather than dynamically generated. In other examples, preamble 401 is dynamically generated, for example, some variability can be introduced into preamble 401 by selecting preamble 401 (or an element thereof) from a plurality of predetermined options, such as by random chance or according to some other distribution.

[0045] Hint 400 also includes table schema data 402. This is an example of query metadata, which is information derived from an input query. Query metadata is additional information that acts as a hint or pointer to the LLM 301 about the KQL query to be generated. Figure 2 In the example of , the table schema data 402 lists specific tables (imAuthentication) and specific columns (EventResult, SrcDvclpAddr, etc.) related to the KQL query to be generated.

[0046] The prompt word 400 also includes samples 403, which are example input queries and corresponding KQL queries. In the example shown, the prompt word includes two samples 403, which are separated by a row of pound signs used as separators.

[0047] The prompt word 400 also includes an input query 404 for which a corresponding KQL query is sought.

[0048] Finally, the prompt 400 includes ideographic data 405, which is another example of query metadata. The ideographic data 405 states which tables the resulting KQL query should use.

[0049] The samples 403 and query metadata 402 , 405 are dynamically generated by the system 200 using the trained model 130 .

[0050] Figure 2 The illustrated prompt word 400 is merely an example of the structure of a suitable prompt word to aid in understanding the example systems and methods discussed herein. The arrangement of the elements of the prompt word 400 and the number of samples 403 included may vary.

[0051] Additionally, other types of query metadata may be included in prompt 400. In some examples, the query metadata includes an indication of the length or complexity of the resulting KQL query. For example, the query metadata may include statements indicating that the resulting query may be short (e.g., less than a certain number of rows) or long (e.g., more than a certain number of rows). The query metadata may give an indication of the types of statements to be included in the KQL query (e.g., the query should include a join statement).

[0052] Figure 3 The process of training a model to generate elements of a prompt word for LLM 301 is generally shown. The process may be performed by the training system 100.

[0053] In step S301, the process includes forming a training dataset for training the model 130. The training dataset may include manually labeled training data and / or synthetically generated examples.

[0054] In step S302, the process includes probing the LLM 301 using probe cues generated from the training data. The probe cues include samples selected from the training data. By evaluating the response of the LLM 301 to different probe cues including different sample selections, a ranking of sample usefulness is obtained.

[0055] In step S303 , the process includes training the model 130 to select samples to include in the prompt word using the ranking obtained in step S202 .

[0056] In step S304 , the process includes training the model 130 to generate query metadata using the training data to include in the prompt term.

[0057] This process results in a trained model 130 that is configured to generate samples and query metadata for prompt words for submission to the LLM 301 .

[0058] Figure 4 An example process of forming training data set 110 is shown in more detail.

[0059] The system 100 is provided with receiving an initial training data set 111. The initial training data set 111 includes data similar to Figure 2 The example prompt words shown in , which are ground truth examples of prompt words for a specific input query.

[0060] The system 100 is also provided with a set of KQL queries 112, each query having a corresponding description. The description explains the purpose of the corresponding KQL query. Figure 4 An example KQL query and description 112a is shown in FIG.

[0061] In one example, KQL queries and descriptions are manually created. For example, they can be harvested from suitable manual training materials, code examples, etc., where the description is a comment in the code. In other examples, KQL queries and descriptions can be synthetically generated, for example, by using a roundtrip filtering technique similar to that discussed in Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, and Michael Collins, Synthetic QA Corpora Generation with Roundtrip Consistency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6168–6173, Florence, Italy. Association for Computational Linguistics, 2019.

[0062] Initial training set 111 and KQL query set 112 are used to generate example user queries corresponding to KQL queries in query set 112. In particular, examples from initial training set 111 are used as samples in prompt words 113 for generating corresponding user queries.

[0063] Once generated, each prompt word 113 is provided to LLM 114. In one example, LLM 114 is not the same LLM as LLM 301 used to generate KQL queries. Instead, LLM 114 is an LLM intended for natural language generation, such as the Davinci GPT-3 model provided by Open AI. LLM 114 accordingly returns a synthetic user query 115 corresponding to a KQL query in KQL query set 112.

[0064] This approach allows a relatively small initial training dataset 111 comprising user queries and corresponding KQL queries to be expanded using a larger labeled dataset of KQL queries 112 accompanied by textual descriptions.

[0065] In addition, by changing the samples included in the prompt word 113, multiple user queries of different styles corresponding to the same underlying KQL can be generated. This is based on the fact that the LLM 114 will respond with different variations of the user query (e.g., different grammatical structures or writing styles) according to the samples included in the prompt word. The result of this part of the process is a corpus including KQL queries, descriptions, and corresponding user queries.

[0066] To further expand the training data, the synthetic query is then enhanced. In other words, natural language processing techniques or tools are used to generate additional queries that substantially correspond to the query 115 in meaning.

[0067] In one example, a user query is reverse translated to generate a new query. Reverse translation is the process of converting a query from English to a different language and back again using a suitable trained machine translation model. The result of the reverse translation can simply be used as a new query, or the result can be combined with the original query to expand the query. Queries can be reverse translated via a variety of different languages ​​to generate more queries.

[0068] The reverse transformed and original query 115 is enhanced to generate an enhanced query 116 by replacing one or more words of the query with synonyms using a subject term or word embedding model, or by inserting words in the query based on a suitable word embedding model. An example library suitable for performing this data enhancement is the NLP Enhancement Library (Edward Ma, see https: / github.com / makcedward / nlpaug ).

[0069] This results in a relatively large corpus 110 of example natural language queries, each corresponding to a KQL query. In one example, an initial training dataset 111 of 62 examples and approximately 1200 KQL queries 112 was used to generate 219 synthetic questions, each expressed in 6 different ways, for a total of 1314 natural language queries. Back-transformation expanded the total natural language queries to 9200, and applying augmentation resulted in 28300 user queries.

[0070] Figure 5 An example process of detecting LLM 301 using training data set 110 is shown.

[0071] The training data 110 is split into two parts for the detection process—the first part is a plurality of validation training data items 501, and the second part is a plurality of example samples 502. The validation training items 501 each include a user query 501a and a true value KQL query 501b. The samples 502 also include a user query and a corresponding true value KQL.

[0072] The user query 501a is selected to form the subject of the probe cue word 504. In other words, in order to obtain the KQL corresponding to the selected user query 501a, the probe cue word 504 is generated.

[0073] The system 100 selects a random arrangement of samples from the example samples 502, as shown in box 503. For example, a random subset of 4 samples can be selected, where the order is also randomized. The selected samples are included in the detection cue word 504.

[0074] Subsequently, the probe cue word 504 is submitted to the LLM 301 , and the KQL 505 corresponding to the probe cue word is returned by the LLM 301 .

[0075] The LLM generated KQL 505 is then compared with the true value KQL 501b corresponding to the user query 501a. The system calculates a score reflecting the quality of the LLM generated KQL 505 based on the true value 501b. Figure 5 denoted by box 506.

[0076] In one example, the score is a weighted score that includes a combination of metrics. For example, one metric can be the edit distance (e.g., Levenshtein distance or the like) between the KQL 505 generated by the LLM and the true value 501b. Other example metrics reflect the semantic validity of the KQL 505, and whether the correct table is included in the KQL 505. The score can be thought of as a distance, where a higher score indicates a KQL output 505 that is less reflective of the true value 501b, while a lower score reflects a KQL output 505 that is closer to the true value 501b.

[0077] The process of generating randomly arranged cue words including samples for a given user query 501 is repeated multiple times. For example, 100 probe cue words may be generated for a given user query.

[0078] Figure 6 The output of this part of the process is shown. Figure 6 Each row in the table of represents a specific probe cue word for a given user query 501. The rightmost column of the table represents the score associated with the probe cue word. The other columns of the table reflect which example samples in the set of example samples are included in the probe cue word, where "1" indicates that the sample is included and "0" indicates that the sample is not included. For illustration purposes, Figure 6 The table in includes only 6 possible samples and 4 probe cues, and in reality the set of samples and probe cues is significantly larger.

[0079] Next, the linear model (507, Figure 5 ) is fitted to the scored probe prompt words. The linear model 507 reflects the contribution of each sample to the score, that is, the degree to which each sample affects the score and how much. The coefficients output by the linear model are then used to generate the ranking of the samples 508. The ranking reflects the usefulness of each sample for generating the correct KQL 505.

[0080] This process is repeated for each user query 501 in order to generate a ranked list of samples associated with each query.

[0081] Return to Figure 3 , the machine learning model 130 is then trained to select samples for inclusion in the prompt words for the LLM 301 based on the ranked list associated with the training data user query. The samples selected and included in the prompt words may be referred to as "a sample" or "several samples" (as the case may be) or "selected samples" to distinguish them from the corpus of samples 502 from which they were extracted.

[0082] Figure 7 The structure of an example machine learning model 130 is shown in . The machine learning model 130 includes a base pre-trained LLM 131. For example, LLM 131 is a BERT model or a variant thereof, such as a distilBERT model. The machine learning model 130 also includes a pooling layer 132, which receives a pool of the outputs of the BERT model 131. On top of the pooling layer, the model 130 includes two task heads: a ranking task head 133 and a classification task head 134. The ranking task head 133 is configured to determine which samples are included in the prompt word and the order in which they are included. The classification task head 134 is configured to output query metadata to be included in the prompt word.

[0083] Figure 8 An example process of training a machine learning model 130 to select samples is shown.

[0084] For a given training query 501, two example samples 508a, 508b are selected from the corresponding ranked list of samples 508. Two inputs 509a, 509b are generated to the model 130, from each of the selected samples 508b, 508b. In the example using the BERT model, the input may be of the form "[CLS] example sample [SEP] training query [SEP]".

[0085] The input is passed through the base LLM 131, the pooling layer 132, and the ranking task head 133 to generate corresponding output scores 510a and 510b. Then, a pairwise cross entropy loss 511 is calculated from the output scores 510a and 510b and the true rankings of the samples 508a, 508b. The training of the model 130 attempts to minimize the loss function 511. In one example, the loss function 511 is a pairwise contrastive cross entropy loss.

[0086] In more detail, calculating the cross entropy 511 may involve passing the output scores 510a and 510b through a SoftMax layer to obtain probability values ​​corresponding to the output scores 510a, 510. In addition, the probability values ​​are obtained based on the true value ranking 508. The two sets of probability values ​​are compared to generate a cross entropy loss 511.

[0087] Figure 8 The process is shown for a pair of samples 508a, 508b for one query 501. The process is repeated for each pair of samples extracted from the ranked list of samples 508. The process is then repeated for each training query 501. As a result of the training process, the model 130 is trained to generate output scores that reflect the utility of the samples 508 for unseen input queries.

[0088] Return to Figure 3 , and then train the machine learning model 130 to generate query metadata using the training data to include in the prompt terms.

[0089] Fig. 9 An example of a process of training a model to generate query metadata is shown. In particular, the model receives input training data 501 in the form of a user query 501a with corresponding query metadata 501c. The query metadata 501c can be extracted from a true value KQL query 501b associated with the user query 501a. For example, the KQL query 501b can be parsed to extract a list of tables or columns included in the query, or can be evaluated to determine its length or complexity, or to determine whether the KQL query 501b includes a particular operation (e.g., "join" or "merge").

[0090] The model 130 is then trained so that the classification head 134 predicts query metadata for unseen input queries. Techniques for further training BERT or other such LLM models for specific classification tasks include those discussed in: https: / / towardsdatascience.com / multilabel-classification-with-pytorch-in-5-minutes-a4fa8993cbc7 and https: / / curiousilyl.com / posts / multi-label-text-classification-with-bert-and-pytorch-lighting / .

[0091] The discussion will now turn to using the model 130 to generate cue words at inference time.

[0092] Fig.10 An example process for generating cue words for LLM 301 is generally shown. Fig.10 The process can be performed by system 200.

[0093] In step S1001 , the process includes receiving an input secure search user query. The input secure search user query may be received via the access interface 220 .

[0094] For example, the access interface 220 takes the form of a web application. In other words, the access interface 220 takes the form of a web server configured to provide web pages. A user can provide input, for example, by viewing and interacting with the served web pages using the system 500. In particular, the access interface 220 includes an input element (e.g., a text box) in which a user can enter the input secure search user query.

[0095] In step S1002, the process includes selecting samples using the trained machine learning model 130. In particular, the process includes inputting an input safe search user query to the trained model 130, and in response, receiving selected samples, for example from the ranking head 133. The samples include an example user safe search query and a corresponding example safe language query. In some cases, multiple samples may be selected, as described below.

[0096] In step S1003, the process includes generating query metadata from an input safe search query using the trained machine learning model 130. This involves inputting an input safe search user query to the trained model 130, and in response, receiving query metadata, for example from the classification head 134.

[0097] In step S1004, the process includes generating a prompt word for input to LLM 301. The prompt word is used to generate a safe language query corresponding to the input safe search user query. The prompt word includes the input safe search user query, the selected sample, and the generated query metadata. The prompt word may also include static data, such as a preamble.

[0098] In step S1005, the generated prompt word is input to LLM 301. In particular, the prompt word can be input to LLM 301 through the above-mentioned Figure 1The appropriate API in question submits the prompt words to the LLM 301. In response, in step S1006, the process includes receiving from the LLM 301 a safe language query corresponding to the input safe search input user query.

[0099] In step S1007, an output is generated based on the generated secure language query. In one example, the generated secure language query is then displayed to the user, for example, on the user interface 503. The user can then copy the query and execute it on the secure system 504 to which they have access. In another example, step S1007 includes executing the query in the secure system 504.

[0100] Go to Fig.11 , showing in more detail an example process of selecting samples using a trained machine learning model 130.

[0101] Each of the samples 502-1 to 502-N is combined with the user query 601 to generate a corresponding input 601-1 to 601-N to the model 130. Each of the inputs 601-1 to 601-N is input to the model 130. The ranking head 133 outputs an output score 602-1 to 602-N indicating the usefulness of the corresponding sample 502 to the user query 601. The ranking of the usefulness of each sample 502 to the input query 601 is then obtained by sorting the samples based on the score 602.

[0102] The ranking is then used to select samples. In one example, the k highest scoring samples are selected 502. k can be preset. For example, k is 3. In other examples, k is 1 (so the prompt word is an example of single-sample learning of LLM).

[0103] In other examples, the samples 502 are randomly sampled based on the scores 602. Random sampling will be discussed in more detail below.

[0104] Fig.12 An example process of generating a prompt word including selected samples and query metadata is shown. The process generally involves filling a template or "skeleton" prompt word with samples and metadata from a trained model and other predetermined or statically generated information.

[0105] First, in step S1201, a preamble of a prompt word is generated. The preamble includes predetermined text that serves as a context for the LLM 301 in order to generate a correct output. In addition, the preamble includes query metadata in the form of table schema information.

[0106] Next, in step S1202, the selected sample 502 (for example, as described above with respect to Fig.10 and Fig.11The selected sample discussed above is inserted into the prompt word. In the case of including multiple samples, the samples can be included in a predetermined order. For example, in some cases, it has been found that the final sample in the prompt word has a greater impact on the output of the LLM 301 than the earlier prompt words. Therefore, the samples 502 can be sorted so that the highest scoring sample 502 is placed last. In the case of inserting multiple samples 502, they can be separated by a suitable separator.

[0107] In step S1203, sample metadata corresponding to each selected sample 502 is generated. The sample metadata is similar to the query metadata discussed herein. However, since the KQL for each selected sample is known, the sample metadata can be determined from the KQL itself. For example, the KQL in the sample 502 can be parsed to extract the tables and columns used. The KQL can also be processed to provide a measure of the complexity or length of the query.

[0108] In step S1204, the sample metadata is inserted into the prompt word.

[0109] In step S1205, final information is generated and inserted into the prompt word. The final information is the final part of the prompt word. The final information includes query metadata in the form of ideographic information. The final information also includes the user query. The completed prompt word is in the form of text data (i.e., a string of characters).

[0110] Fig.13 A process according to another example of the present disclosure is shown.

[0111] In step S1301, multiple prompt words are generated for a given input query. The multiple prompt words are different from each other. For example, the prompt words may include different sample selections or different query metadata. Therefore, generating each prompt word may include selecting a corresponding sample and generating query metadata.

[0112] This may be accomplished by randomly sampling the output of model 130 or varying parameters or settings associated with the trained model 130 to introduce variability into the output of model 130 .

[0113] In some examples, step S1301 includes coordinating the generation of multiple prompt words so as to effectively obtain a spread of prompt words with some spread or distribution of different sample or query metadata. Coordinating the generation of multiple prompt words in this way can increase the chance that LLM 301 returns an effective KQL query that closely reflects the user's intent.

[0114] Fig.14 An example technique for coordinating the generation of multiple prompt words is shown. Fig.14, the ranking head 133 and the classification head 134 each have a corresponding prior probability distribution 133P, 134P. The prior distributions 133P, 134P shown reflect the distribution of the subject matter of the user query - in this case, whether they are related to a particular product. In other examples, the distribution can reflect the intent in the user query, i.e., table names or schema data, or whether a particular operation (join or merge) is included.

[0115] In this example, the model 130 also includes a prior sampler 135 that combines the prior distributions 133P, 134P from the head into a central prior distribution 135P. The prior sampler 135 then samples the central prior distribution to output a plurality of prior distributions 136P-1 to 136P-4. For example, a truncated discrete sampling step can be used to generate different distributions 136P.

[0116] The sampling distributions 136P-1 to 136P-4 are then each used as a parameter for the ranking head 133 and the classification head 134 to generate the relevant output (i.e., sample selection or query metadata). In the example shown, 4 distributions will be used to generate 4 prompt words, but it should be understood that the number of different prompt words generated can vary. In one example, 10 prompt words are generated.

[0117] Back to Fig.13 In step S1302, the multiple generated prompt words are respectively passed to LLM 301, and the corresponding KQL query is returned.

[0118] Next, in step S1303, the KQL query is scored. The score reflects the likelihood that the KQL query corresponds to the input user security search query. In one example, the score is based on the probability or confidence returned by LLM 301. For example, the probability returned by LLM 301 is weighted using a priori sampling probability to generate a score.

[0119] Next, in step S1304, the KQL queries returned by the LLM 301 are validated. In other words, the KQL queries are parsed to ensure that they are syntactically valid. In step S1305, the highest scoring valid query is selected as the KQL query corresponding to the user input.

[0120] Fig.15 A non-limiting example of a computing system 1200 that can perform one or more of the above methods and processes is schematically shown. The computing system 1200 is shown in simplified form. The computing system 1200 can embody the above and Figure 1Any of the computing systems 100, 200, 300, or 500 shown. The computing system 1200 can take the form of one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phones), and / or other computing devices, as well as wearable computing devices (such as smart watches and head-mounted augmented reality devices).

[0121] The computing system 1200 includes a logic processor 1202, a volatile memory 1204, and a non-volatile storage device 1206. The computing system 1200 may optionally include a display subsystem 1208, an input subsystem 1210, a communication subsystem 1212, and / or Fig.12 Other components not shown.

[0122] Logical processor 1202 includes one or more physical devices configured to execute instructions. For example, a logical processor may be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform tasks, implement data types, transform the state of one or more components, implement technical effects, or otherwise achieve a desired result.

[0123] The logical processor may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logical processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of the logical processor 1202 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel and / or distributed processing. The various components of the logical processor may optionally be distributed in two or more separate devices, which may be remotely located and / or configured for coordinated processing. Various aspects of the logical processor may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration. In this case, these virtualized aspects are run on different physical logical processors of various different machines, which should be understood.

[0124] The non-volatile storage device 1206 includes one or more physical devices configured to hold instructions executable by a logical processor to implement the methods and processes described herein. When implementing such methods and processes, the state of the non-volatile storage device 1206 may be changed, for example, to hold different data.

[0125] The non-volatile storage device 1206 may include a removable and / or built-in physical device. The non-volatile storage device 1206 may include an optical memory (e.g., CD, DVD, HD-DVD, Blu-ray disc, etc.), a semiconductor memory (e.g., ROM, EPROM, EEPROM, flash memory, etc.), and / or a magnetic memory (e.g., a hard disk drive, a floppy disk drive, a tape drive, MRAM, etc.), or other mass storage device technology. The non-volatile storage device 1206 may include a non-volatile, dynamic, static, read / write, read-only, sequential access, location addressable, file addressable, and / or content addressable device. It should be understood that the non-volatile storage device 1206 is configured to retain instructions even when the non-volatile storage device 1206 is powered off.

[0126] The volatile memory 1204 may include a physical device that includes random access memory. The volatile memory 1204 is typically used by the logical processor 1202 to temporarily store information during the processing of software instructions. It should be understood that when the volatile memory 1204 is powered off, the volatile memory 1204 typically does not continue to store instructions.

[0127] Aspects of the logic processor 1202, volatile memory 1204, and non-volatile storage device 1206 may be integrated together into one or more hardware logic components. For example, such hardware logic components may include field programmable gate arrays (FPGAs), program and application specific integrated circuits (PASIC / ASIC), program and application specific standard products (PSS / ASSP), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0128] The terms "module", "program", and "engine" may be used to describe aspects of a computing system 1200 that is typically implemented in software by a processor to use portions of volatile memory to perform a specific function that involves a transformation process that specifically configures the processor to perform the function. Thus, a module, program, or engine may be instantiated using portions of volatile memory 1204 via a logical processor 1202 that executes instructions held by a non-volatile storage device 1206. It should be understood that different modules, programs, and / or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, and the like. Likewise, the same module, program, and / or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, and the like. The terms "module", "program", and "engine" may cover individuals or groups of executable files, data files, libraries, drivers, scripts, database records, and the like.

[0129] When included, the display subsystem 1208 can be used to present a visual representation of the data held by the non-volatile storage device 1206. The visual representation can take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data held by the non-volatile storage device and thus transform the state of the non-volatile storage device, the state of the display subsystem 1208 can also be transformed to visually represent the changes in the underlying data. The display subsystem 1208 can include one or more display devices utilizing almost any type of technology. Such a display device can be combined with the logical processor 1202, the volatile memory 1204, and / or the non-volatile storage device 1206 in a shared housing, or such a display device can be a peripheral display device.

[0130] When included, the input subsystem 1210 may include or interface with one or more user input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may include or interface with selected natural user input (NUI) components. Such components may be integrated or peripheral, and the conversion and / or processing of input actions may be processed on-board or off-board. Example NUI components may include microphones for speech and / or speech recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing components for assessing brain activity; and / or any other suitable sensors.

[0131] When included, the communication subsystem 1212 can be configured to communicatively couple the various computing devices described herein to each other and to communicatively couple to other devices. The communication subsystem 1212 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network. In some embodiments, the communication subsystem can allow the computing system 1200 to send messages to other devices and / or receive messages from other devices via a network such as the Internet.

[0132] Various modifications may be made to the examples discussed herein within the scope of the present disclosure.

[0133] Throughout this disclosure, reference is made to KQL as an example security query language. KQL is merely an example of such a security query language, and the techniques disclosed herein can be applied to other query languages, such as the query languages ​​used in Splunk and Google security products.

[0134] The probing techniques discussed herein for evaluating the impact of samples on output KQLs can also be applied to probing other aspects of LLM 301. For example, a probe cue word can be generated to evaluate the impact of a particular hyperparameter or model setting on the output KQL. For example, a regression head can be added to a model 130 that is trained based on the results of a probe cue word in which a hyperparameter of the model is randomly selected.

[0135] The examples herein utilize a single underlying pre-trained model with multiple heads to perform sample selection and query metadata generation, respectively. However, in other examples, separate underlying pre-trained models may be used for these purposes. In such examples, the combination of models may be considered a "trained model" as defined herein.

[0136] The specific training datasets discussed herein are examples and may vary. The techniques used to synthesize training data may be employed in a different order, for example, back-transformation may occur after augmentation.

[0137] Although the examples described herein relate to user input queries in the form of natural language (i.e., speech), user input queries are not limited to this form. In some examples, the input query may include structured data. For example, the input query may include a graph showing the relationship between entities or table data.

[0138] The measurement of "distance" or similarity during the detection process is an example, and any suitable method of evaluating the correspondence between the true query and the results returned by LLM 301 during the detection period may be used. Likewise, the use of a linear model to rank the influence of samples on a particular user query is an example, and another suitable model may be used.

[0139] At inference time, the order of the steps discussed herein can be changed. For example, a prompt word template can be created, and query metadata and sample selection can be generated subsequently. Similarly, in examples involving the generation of multiple KQL queries, the KQL queries can be validated before being scored.

[0140] It should be understood that the processor or processing system or circuit referred to herein may actually be provided by a single chip or integrated circuit or multiple chips or integrated circuits, optionally as a chipset, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a graphics processing unit (GPU), etc. One or more chips may include circuits (and possible firmware) for embodying at least one or more of one or more data processors, one or more digital signal processors, baseband circuits, and radio frequency circuits, which are configurable to operate according to exemplary embodiments. In this regard, exemplary embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by a processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware).

[0141] Reference is made herein to data storage devices (e.g., storage devices 102 and 202) for storing data. This may be provided by a single device or multiple devices. Suitable devices include, for example, hard disks and non-volatile semiconductor memories (e.g., solid-state drives or SSDs).

[0142] Although at least some aspects of the embodiments described herein with reference to the accompanying drawings include computer processes executed in a processing system or processor, the invention also extends to computer programs suitable for practicing the invention, in particular computer programs on or in a carrier. The program may be in the form of non-transitory source code, object code, code intermediate source and object code, such as in a partially compiled form, or in any other non-transitory form suitable for use in the implementation of the process according to the invention. The carrier may be any entity or device capable of carrying the program. For example, the carrier may include a storage medium, such as a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, such as a CD ROM or a semiconductor ROM; a magnetic recording medium, such as a hard disk; a conventional optical memory device; and the like.

[0143] Additional example features of the disclosure are set forth below.

[0144] The following features are defined with respect to the first aspect listed in the Summary.

[0145] User queries entered into SafeSearch can be natural language queries.

[0146] The query metadata may include table schema data indicating tables or columns relevant to the safe language query.

[0147] The method may also include generating example metadata from the selected example safe language query, wherein the prompt word also includes the example metadata.

[0148] The hint word may also include a preamble that indicates that the large language model should return an input safe search query corresponding to the input safe search user query.

[0149] The method may further include: selecting a plurality of example user safety search queries and corresponding example safety language queries, the prompt word including the selected plurality of example user safety search queries.

[0150] The method may include: storing a collection of example user security search queries and corresponding example security language queries; using a trained machine learning model to generate a score indicating the relevance of each example user security search query and corresponding example security language query in the collection to an input user security search query; and selecting multiple example security search queries and corresponding example security language queries from the collection based on the score.

[0151] The method may include: generating multiple prompt words, each prompt word including a different combination of a selected example user safe search query and a corresponding example safe language query and generated query metadata; inputting each of the multiple prompt words into a large language model; receiving a safe language query corresponding to each of the multiple prompt words from the large language model; scoring each safe language query; and selecting the safe language query with the highest score.

[0152] The method may include: sampling the prior probability distribution of a trained machine learning model to generate a sampled prior probability distribution for each of a plurality of prompt words; and selecting an example user security search query and a corresponding example security language query based on the sampled prior probability distribution for each prompt word, and generating query metadata.

[0153] The following features are defined with respect to the second aspect listed in the Summary.

[0154] The trained model may include: a base large language model; a pooling layer; a ranking head; and a classification head. The ranking head may be trained to select user safe search queries and corresponding truth safe language queries. The classification head may be trained to generate query metadata.

[0155] The method may include generating a training data set by: receiving an initial training data set, the initial training data set including multiple user safety search queries, corresponding true-value safety language queries and corresponding query metadata; receiving a safety query set, the safety query set including safety language queries and corresponding text descriptions of the safety language queries; generating prompt words, the prompt words including: user safety search queries and corresponding true-value safety language queries extracted from the initial training data set, and safety language queries and corresponding text descriptions from the safety query set; inputting the prompt words into a natural language generation large language model; receiving user safety search queries corresponding to the safety language queries from the safety query set, and including the received user safety search queries and safety language queries from the safety query set in the training data set.

[0156] The method may include: reverse-transforming the user safe search query to generate a reverse-transformed safe search query; and including the reverse-transformed safe search query in a training data set.

[0157] The method may include: fitting a model to a detection score; generating a ranking from the fitted model, the ranking indicating the usefulness of each example included in the detection cue word to the topic of the detection cue word; and training a machine learning model based on the ranking to select user safe search queries and corresponding true value safe language queries of a training data set.

[0158] Training a machine learning model based on the rankings includes: selecting a pair of examples from the rankings; inputting the pair of examples into the machine learning model; and training the model to minimize a pairwise cross entropy loss calculated from the pair of examples.

[0159] The method may include: generating multiple hyperparameter detection prompts, wherein generating each hyperparameter detection prompt includes: selecting one of multiple user safety search queries as the subject of the hyperparameter detection prompt; randomly selecting a value for a hyperparameter for a machine learning model; inputting the generated hyperparameter detection prompt into a large language model; receiving a safety language query corresponding to the subject of each hyperparameter detection prompt as an output from the large language model; comparing the received safety language query for the hyperparameter detection prompt with the corresponding true value safety language query; calculating a hyperparameter detection score for each hyperparameter detection prompt based on the comparison; and training the machine learning model based on the calculated hyperparameter detection score to select a value for the hyperparameter.

[0160] The methods of the aspects discussed in the overview may be combined.The trained model used in the first aspect may be trained according to the method of the second aspect.

[0161] Another example computer-implemented method provided by the present disclosure includes:

[0162] The present disclosure extends to a system comprising a processor and a storage device storing computer readable instructions which, when executed by the processor, perform any of the methods described herein.

[0163] The examples described herein are to be understood as illustrative examples of embodiments of the present invention. Additional embodiments and examples are contemplated. Any feature described with respect to any one example or embodiment may be used alone or in combination with other features. In addition, any feature described with respect to any one example or embodiment may also be used in combination with one or more features of any other example or embodiment or any combination of any other example or embodiment. In addition, equivalents and modifications not described herein may also be adopted within the scope of the present invention as defined in the claims.

Claims

1. A computer-implemented method comprising: receiving, at a computer system, an input secure search user query indicative of user intent; selecting, using the trained machine learning model and based on the input safe search query, an example user safe search query and a corresponding example safe language query; generating query metadata from the input secure search query using the trained machine learning model; Generate prompt words, the prompt words include: The input safe search user query; the selected example user safe search query and the corresponding example safe language query; and the query metadata generated; Inputting the prompt word into a large language model; A safe language query corresponding to the input secure search query reflecting the user intent is received from the large language model. 2 . The method of claim 1 , wherein the input secure search user query is a natural language query.

3. The method according to claim 1 or 2, wherein the query metadata includes table schema data, the table schema data indicating tables or columns related to the safe language query.

4. The method according to any of the preceding claims, further comprising: generating example metadata from the selected example safe language query, The prompt word also includes the example metadata.

5. A method according to any preceding claim, comprising: Select multiple example user safe search queries and corresponding example safe language queries, The prompt words include the selected plurality of example user safe search queries.

6. The method according to claim 5, comprising: storing a collection of example user safe search queries and corresponding example safe language queries; generating, using the trained machine learning model, a score indicating relevance of each example user safe search query and corresponding example safe language query in the collection to the input user safe search query; Based on the scores, the plurality of example safe search queries and corresponding example safe language queries are selected from the collection.

7. A method according to any preceding claim, comprising: Generate a plurality of prompt words, each prompt word including a different combination of the selected example user security search query and the corresponding example security language query and the generated query metadata; Inputting each of the plurality of prompt words into the large language model; receiving, from the large language model, a safe language query corresponding to each of the plurality of prompt words; Score each safe language query; Select the highest-rated safe language query.

8. The method according to claim 7, comprising: Sampling the prior probability distribution of the trained machine learning model to generate a sampled prior probability distribution for each cue word in the plurality of cue words; as well as For each prompt word, based on the sampled prior probability distribution, the example user safety search query and the corresponding example safety language query are selected, and the query metadata is generated.

9. A computer-implemented method comprising: receiving a training data set, the training data set comprising a plurality of user secure search queries, corresponding truth-valued secure language queries, and corresponding query metadata; Generate a plurality of detection cue words from the training data set, wherein generating each detection cue word comprises: Selecting one of the plurality of user safe search queries as a subject of the detection prompt word; Randomly selecting a plurality of user security search queries and corresponding truth-value security language queries as examples of the detection prompt word; Inputting the generated detection cue words into a large language model; receiving, as output from the large language model, a safe language query corresponding to the subject of each probe cue word; comparing the received secure language query for the detection cue word with the corresponding true secure language query; Based on the comparison, calculating a detection score for each detection cue word; A machine learning model is trained based on the calculated detection score and the training data set, wherein the machine learning model is trained to: receiving an input user secure search query; generating an output score reflecting the utility of each user safe search query and the corresponding truth-valued safe language query of the training data set to the input user safe search query; and Based on the output score, selecting a user safe search query and a corresponding truth-value safe language query of the training data set as examples of prompt words for input to the large language model; and Query metadata for prompt terms for the large language model is generated from the input secure search user query.

10. The method of claim 9, comprising generating the training data set by: receiving an initial training data set, the initial training data set comprising a plurality of user secure search queries, corresponding truth-valued secure language queries, and corresponding query metadata; receiving a set of safe queries, the set of safe queries comprising safe language queries and corresponding text descriptions of the safe language queries; Generate prompt words, the prompt words include: User safe search queries and corresponding truth-valued safe language queries extracted from the initial training dataset, and a safe language query and a corresponding text description from the safe query set; Inputting the prompt word into a natural language generation large language model; A user safe search query corresponding to the safe language query is received from the safe query set, and the received user safe search query and the safe language query from the safe query set are included in the training data set.

11. The method according to claim 9 or 10, comprising: Reverse transforming the user safe search query to generate a reverse transformed safe search query; as well as The back-transformed safe-search query is included in the training data set.

12. The method according to any one of claims 9 to 11, comprising: fitting a model to the detection scores; generating a ranking from the fitted model, the ranking indicating the usefulness of each example included in the probe cue word to the topic of the probe cue word; The machine learning model is trained based on the ranking to select user safe search queries and corresponding truth-valued safe language queries of the training data set.

13. The method of claim 12, wherein training the machine learning model based on the ranking comprises: Select a pair of examples from said ranking; inputting the pair of examples into the machine learning model; The model is trained to minimize a pairwise cross entropy loss calculated from the pair of examples.

14. The method according to any one of claims 9 to 14, comprising: Generate multiple hyperparameter detection prompt words, wherein generating each hyperparameter detection prompt word includes: Selecting one of the plurality of user safe search queries as a subject of the hyperparameter detection prompt word; randomly selecting values ​​for hyperparameters of the machine learning model; Inputting the generated hyperparameter detection prompt words into the large language model; receiving, as output from the large language model, a safe language query corresponding to the topic of each hyperparameter detection hint word; comparing the received safe language query for the hyperparameter detection prompt word with the corresponding true safe language query; Calculating a hyperparameter detection score for each hyperparameter detection cue word based on the comparison; The machine learning model is trained to select a value for the hyperparameter based on the calculated hyperparameter detection score.

15. A system comprising a processor and a storage device storing computer readable instructions which when executed by the processor perform the method according to any preceding claim.

Images