Generating secure reports

Abstract

In some examples, a method of generating a security report is provided. The method includes receiving a user query and security data, and providing the user query and security data to a semantic model. The semantic model generates one or more first inlays. The method also includes receiving one or more second inserts from the data model. The data model is generated based on historical threat intelligence data. The model further includes generating an execution plan based on the one or more first inserts and the one or more second inserts, and returning a report corresponding to the execution plan.

Description

Background Art

[0001] Security analysts can manually combine security data with threat intelligence data to achieve security objectives. Security analysts typically perform this work as an offline task before or after a security incident. Combining security data with threat intelligence is a fragile process that may not match similar, but not identical, entities across security and threat intelligence data.

[0002] It is with respect to these and other general considerations that the embodiments have been described.Furthermore, although relatively specific problems have been discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the background. Summary of the Invention

[0003] Aspects of the present disclosure relate to methods, systems, and media for generating security reports. In some examples, user queries and security data may be provided. The security data may include raw logs associated with one or more incidents of a computing environment. In addition, the user query may be a query provided by a user to achieve a security goal. The user query and security data may be provided to a semantic model that generates one or more first embeddings. One or more second embeddings may be received from a data model. The data model may be generated based on historical threat intelligence data. An execution plan may be generated based on the one or more first embeddings and the one or more second embeddings. Specifically, the execution plan may be generated based on determining instructions based on corresponding similarities between the second embedding and the first embedding. A report corresponding to the execution plan may be returned.

[0004] This summary is provided to introduce in simplified form a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be set forth in part in the following description and in part will become apparent from the description or may be learned through practice of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0006] Figure 1 An overview of an example system according to some aspects described herein is shown.

[0007] Figure 2 An example process for executing a plan according to some aspects described herein is shown.

[0008] Figure 3 An example process for training a data model according to some aspects described herein is shown.

[0009] Figure 4 An example architecture according to some aspects described herein is shown.

[0010] Figure 5 An example user interface is shown according to some aspects described herein.

[0011] Figure 6 An example user interface is shown according to some aspects described herein.

[0012] Figure 7 An example method for generating a safety report according to some aspects described herein is shown.

[0013] Figure 8A and Figure 8B An overview of an example generative machine learning model that can be used in accordance with aspects described herein is shown.

[0014] Figure 9 A block diagram illustrating example physical components of a computing device in which aspects of the present disclosure may be practiced.

[0015] Figure 10 Shown is a simplified block diagram of a computing device that can be used to practice aspects of the present invention.

[0016] Figure 11 is a simplified block diagram of a distributed computing system in which aspects of the present disclosure may be practiced. DETAILED DESCRIPTION

[0017] In the following detailed description, reference is made to the accompanying drawings which form a part thereof, and in which specific embodiments or examples are shown by way of illustration. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the present disclosure. The embodiments may be practiced as methods, systems, or devices. Therefore, the embodiments may take the form of hardware implementations, fully software implementations, or implementations combining software and hardware aspects. Therefore, the following detailed description should not be considered restrictive, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0018] As mentioned above, security analysts can manually combine security data with threat intelligence data to achieve security goals. Security analysts typically perform this work as an offline task before or after a security incident. Combining security data with threat intelligence is a fragile process that may fail to match similar, but not identical, entities across security and threat intelligence data.

[0019] The mechanisms provided herein may include assistance from artificial intelligence (AI) agents that analyze security data and automatically extract relevant threat intelligence to enrich investigations in real time. Security data may include incidents or entities discovered by security tools. Security data may also include raw logs from endpoints, networks, clouds, security appliances, and applications. The AI agent can help find relevant entities from security data; automatically generate queries to retrieve threat intelligence (TI) data; extract entities from TI data; and standardize entities across security data and TI data to form a consistent view of security context.

[0020] The AI agent can also perform fuzzy and / or semantic matching to find related entities that have been specified differently across different data sources. The AI agent can combine data in real time, or can be proactively executed to find similarities between security data and threat intelligence for further investigation. The mechanism provided herein may not require parsers for different types of data. The AI agent can automatically generate reports in natural language to record security incidents or potential security investigations. The reports generated by the AI can then be consumed by the AI agent as security data to improve the process (e.g., by training itself based on iterations of the methods provided herein).

[0021] Figure 1 An example of a system 100 according to some aspects of the disclosed subject matter is shown. System 100 can be a system for generating security reports and / or generating execution plans. System 100 includes one or more computing devices 102, one or more servers 104, a secure data source 106, an input data source 107, and a communication network or network 108.

[0022] Computing device 102 may receive security data 110 from security data source 106, which may be, for example, an endpoint, a network, a cloud, a security appliance, a program executed by a computer generating the security data, and / or a memory having data corresponding to the security data stored therein. Security data 111 may be, for example, an error file, a log file, threat intelligence data, computer terminal records, network records, or some other security data recognizable by one of ordinary skill in the art. Additionally or alternatively, network 108 may receive security data 110 from security data source 107.

[0023] The computing device 102 may receive input data 111 from an input data source 107, which may be, for example, a camera, a microphone, a computer-executed program generating the input data, and / or a memory having data corresponding to the input data stored therein. The input data 111 may be, for example, user input (such as a voice query, a text query, etc.), an image, an action performed by a user and / or the device, a computer command, a program evaluation, or some other input data recognizable by one of ordinary skill in the art. Additionally or alternatively, the network 108 may receive the input data 111 from the input data source 107.

[0024] Computing device 102 may include a communication system 112, a goal generation engine or component 114, and / or an execution plan generation engine or component 116. In some examples, computing device 102 may execute at least a portion of goal generation component 114 to generate an embedding corresponding to a prompt word. For example, a user may provide a security-related prompt word to address a security issue, which is represented by the embedding generated by goal generation component 114.

[0025] In some examples, computing device 102 can execute at least a portion of execution plan generation component 116 to generate a plan based on historical threat intelligence data, security data, and a user-provided query. For example, the plan can be similar to one or more historical plans associated with one or more stored embeddings to which the embedding generated by goal generation component 114 is compared. In some examples, the plan can include one or more executable skills.

[0026] Server 104 may include a communication system 118, a goal generation engine or component 120, and / or an execution plan generation engine or component 122. In some examples, server 104 may execute at least a portion of goal generation component 120 to generate an embedding corresponding to a prompt word. For example, a user may provide a security-related prompt word to address a security issue, which is represented by the embedding generated by goal generation component 120.

[0027] In some examples, server 104 can execute at least a portion of execution plan generation component 122 to generate a plan based on historical threat intelligence data, security data, and a user-provided query. For example, the plan can be similar to one or more historical plans associated with one or more stored embeddings to which the embedding generated by goal generation component 120 is compared. In some examples, the plan can include one or more executable skills.

[0028] Additionally or alternatively, in some examples, computing device 102 may transmit data received from input data source 107 and / or secure data source 106 to server 104 via communication network 108, which may execute goal generation component 114 and / or at least a portion of execution plan generation engine 116. In some examples, goal generation component 114 and / or 120 may perform the following in conjunction with Figure 7 Furthermore, in some examples, the execution plan generation engine 116 and / or the execution plan generation engine 122 may perform the following in conjunction with Figure 7 One or more portions of the method / process 700 are described.

[0029] In some examples, computing device 102 and / or server 104 can be any suitable computing device or combination of devices, such as a desktop computer, a vehicle-mounted computer, a mobile computing device (e.g., a laptop, a smartphone, a tablet computer, a wearable computer, etc.), a server computer, a virtual machine executed by a physical computing device, a web server, etc. Furthermore, in some examples, there can be multiple computing devices 102 and / or multiple servers 104. One of ordinary skill in the art will recognize that security data 110 and / or input data 111 can be received at one or more of the multiple computing devices 102 and / or one or more of the multiple servers 104, such that the mechanisms described herein can generate a plan based on the security data 110 and / or input data 111.

[0030] In some examples, secure data source 106 can be any suitable secure data source (e.g., a microphone, a camera, a sensor, etc.). In a more specific example, secure data source 106 can include a memory that stores secure data (e.g., local memory of computing device 102, local memory of server 104, cloud memory, portable memory connected to computing device 102, portable memory connected to server 104, privately accessible memory, publicly accessible memory, etc.). In another more specific example, secure data source 106 can include an application configured to generate secure data. In some examples, secure data source 106 can be local to computing device 102. Additionally or alternatively, secure data source 106 can be remote from computing device 102 and can transmit secure data 111 to computing device 102 (and / or server 104) via a communication network (e.g., communication network 108).

[0031] In some examples, input data source 107 can be any suitable source of input data (e.g., a microphone, a camera, a sensor, etc.). In a more specific example, input data source 107 can include a memory that stores input data (e.g., local memory of computing device 102, local memory of server 104, cloud memory, a portable memory connected to computing device 102, a portable memory connected to server 104, a privately accessible memory, a publicly accessible memory, etc.). In another more specific example, input data source 107 can include an application configured to generate secure data. In some examples, input data source 107 can be local to computing device 102. Additionally or alternatively, input data source 107 can be remote from computing device 102 and can transmit input data 111 to computing device 102 (and / or server 104) via a communication network (e.g., communication network 108).

[0032] In some examples, communication network 108 can be any suitable communication network or combination of communication networks. For example, communication network 108 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, a 5G network, etc., conforming to any suitable standard), a wired network, etc. In some examples, communication network 108 can be a local area network (LAN), a wide area network (WAN), a public network (e.g., the Internet), a private or semi-private network (e.g., a company or university intranet), any other suitable type of network, or any suitable combination of networks. Figure 1 The communication links (arrows) shown may each be any suitable communication link or combination of communication links, such as a wired link, a fiber optic link, a Wi-Fi link, a Bluetooth link, a cellular link, or the like.

[0033] Figure 2 An example process 200 for executing a plan is shown. In process 200, input incidents 202 and strategies 204 are provided to establish initial goals and sub-goals 206. Input incidents 202 may include safety data, such as information about Figure 1 Described security data 106. Additionally, policies 204 may include intents, prompt words, and / or queries that may be provided by a user and / or the system.

[0034] Establishing the initial goal and sub-goals 206 may include generating text and / or one or more first embeddings, such as first semantic embeddings, based on the provided input incident 202 and strategy 204. The text and / or first embeddings corresponding to the goal may then be provided to the engine to generate an execution plan 208.

[0035] Execution plan 208 can be generated by comparing the target text and / or first embedding from establishing initial target 206 with historical text and one or more second embeddings 210 received from data model 212. Data model 212 can be an embedding object store that stores embeddings. Additionally or alternatively, data model 212 can be an index, a database, and / or a repository. In some examples, data model 212 includes historical threat intelligence data corresponding to second embeddings 210. For example, each second embedding 210 can correspond to a historical input incident (e.g., similar to input incident 202), a historical policy (e.g., similar to policy 204), and / or a historical plan (e.g., similar to generated execution plan 208).

[0036] The generated execution plan 208 can include natural language and / or computer-readable instructions. In some examples, process 200 includes operations for executing the plan 214. In some examples, one or more aspects of the plan can be executed by a user. Additionally or alternatively, one or more aspects of the plan can be executed by a system, such as automatically.

[0037] In some examples, after executing the plan 214, customer insights are presented 216. The customer insights may include one or more indications corresponding to actions performed and / or results achieved in response to executing the plan 214. Additionally or alternatively, presenting the customer insights may include generating a notification to the user indicating that the plan has been executed. In response to presenting the customer insights 216, user feedback 220 may be provided to update the goals and sub-goals 218. The user feedback 220 may be in the form of text, a selection option, a provision gesture, etc. The user feedback 220 may correspond to an indication of whether executing the plan successfully resolved the input incident 202.

[0038] Based on the user feedback 220, one or more updated first embeddings corresponding to the updated goals and sub-goals 218 can be generated. With the updated first embeddings from the updated goals and sub-goals 218, the process 200 can return to generating the execution plan 208. Thus, the execution plan can be updated based on the one or more updated first embeddings and the one or more second embeddings 210 (e.g., corresponding to the updated goals and sub-goals 218).

[0039] User feedback 220 can allow the execution plan to be modified based on supervised learning (e.g., user feedback). In addition, user feedback 220 can allow the execution plan to be personalized (e.g., for a specific user and / or organization). In some examples, the execution plan can be updated based on user feedback from a single user. Alternatively, in some examples, the execution plan can be updated based on user feedback from multiple users.

[0040] In some examples, data model 212 can be summarized 222 to reduce the size of tokens input to a model (e.g., a large language model). For example, if data model 212 is too large to generate embedding 210, the data model can be summarized before generating embedding 210. In some examples, token size may not be a limiting factor for the model into which data model 212 is provided. In such examples, summarizing data model 222 may be optional.

[0041] Figure 3 An example process 300 for training a data model 304 is shown. The data model 304 may be similar to the one described above with respect to Figure 2 Data model 212 discussed. Threat intelligence data 302 may be provided to data model 304. Threat intelligence data 302 may include knowledge, skills, and / or experience-based information regarding the occurrence and assessment of virtual threats, physical threats, and / or threat actors. Threat intelligence data 302 may be intended to help mitigate potential attacks and harmful events that may occur in one or more computing environments. Accordingly, historical threat intelligence data that may be stored in data model 304 may correspond to previously collected threat intelligence data.

[0042] Execution plan 306 can be generated based on data model 304. For example, threat intelligence data 302 can match or be significantly similar to historical threat intelligence data stored in data model 304, and the execution plan corresponds to the threat intelligence data 302. Thus, execution plan 306 can be determined based on the similarity of threat intelligence data 302 to the historical threat intelligence data to which the historical execution plan corresponds.

[0043] The execution plan 306 can then be executed 308 (e.g., by a user and / or by the system), and feedback 310 can be received based on the execution 308 of the execution plan 306. The data model 304 can be updated based on the feedback 310. For example, when the threat intelligence data 302 is received, if there are multiple ranked historical execution plans, the historical execution plans stored in the data model 304 can be ranked based on the feedback previously collected for the historical execution plans. For example, historical execution plans with negative feedback can be ranked lower than historical execution plans with positive feedback.

[0044] Poor feedback can include an indication that the execution plan did not resolve the incident, performed unstable operations, and / or was relatively computationally inefficient. Conversely, good feedback can include an indication that the execution plan did resolve the incident, performed stable operations, and / or was relatively computationally inefficient. Feedback 310 can include a discrete metric that affects the ranking of the execution plan and / or a continuous metric that affects the ranking of the execution plan. In addition, feedback 310 can be quantitative feedback and / or qualitative feedback, and the ranking can be identified from the feedback by a person of ordinary skill in the art using known techniques.

[0045] Figure 4 An example architecture 400 is shown according to some aspects described herein. The example architecture includes data components 402, data prototypes 404, skills or functions 406, planning models 408, orchestration layer 410, agents 412, and agent workers 414.

[0046] Data components 402 may include basic data building blocks, such as type classes and / or phantom types for data. Some example aspects of data components 402 include table data components, entity data components, alert data components, and / or summary data components. In addition, some data components may include an indication of whether the corresponding data is small data, valid data, and / or true positive data. Additional and / or alternative aspects of data components 402 may be recognized by one of ordinary skill in the art.

[0047] Data prototype 404 may include a composite of data components 402. For example, prototype 404 may include a table data component having small data and / or an alarm data component having true positive data. Additional and / or alternative aspects of data prototype 404 may be recognized by one of ordinary skill in the art.

[0048] Skills 406 may include capabilities that can operate on data prototypes 404. Skills 406 may include global skills and / or local skills. Skills 406 may operate on all data of a particular type or local version (e.g., filtered data). Skills 406 may include generating security search data, reasoning, merging reasons, generating insights, splitting data, executing queries, processing search results (e.g., including validation, subsampling, etc.), raw security data understanding (e.g., including summarization, answering questions based on logs / threat intelligence, etc.), extracting entities, adding results to context (e.g., directly or via language transformation), such as using one or more query supplementation models pre-trained and fine-tuned on supplementary data, and / or combining data. Additional and / or alternative skills included in skills 406 may be recognized by one of ordinary skill in the art.

[0049] The plan model 408 includes data types for processing goals, objects, and / or executions. For example, the plan model 408 may include goals or strategies, objectives or customer requirements, and execution diagrams.

[0050] The orchestration layer 410 is responsible for taking a strategy and a set of customer requirements and converting them into a plan and a series of execution graphs. Accordingly, the orchestration layer may include a target and purpose planner component and an execution planner component.

[0051] Agent 412 includes deterministic components and goal-driven (eg, artificial intelligence) agent components. Agent worker 414 can perform one or more aspects of agent 412.

[0052] Figure 5 and Figure 6 Example user interfaces 500 and 600 are shown, respectively, according to some aspects described herein. User interface(s) 500 and / or user interface(s) 600 can be graphical user interfaces (GUIs) displayed on a display screen of a computing device, such as computing device 102. Furthermore, user interface(s) 500 and / or user interface(s) 600 can be generated by at least one of a computing device and / or a server device.

[0053] Go to Figure 5 , user interface 500 includes user input feature 502. User input feature 502 can be a text box, a drop-down menu, and / or a button (e.g., audio, video, gaze, gesture) that activates a sensor. Alternative types of user input with which user input feature 502 can be compatible can be recognized by one of ordinary skill in the art.

[0054] User input feature 502 can be configured to receive prompt words and / or queries about system security. For example, the prompt words can be the type of prompt words that a user will provide to a security analyst to diagnose and / or solve security-related problems (such as for a specific computing environment).

[0055] The user interface 500 also includes a first button 504, a second button 506, and a third button 508. Although three buttons are shown on the user interface 500, one of ordinary skill in the art will recognize that additional, fewer, and / or alternative buttons may be used in alternative examples. In the illustrated example 500, the first button 504 is configured to generate a summary of everyday threats (e.g., security threats) when selected (e.g., by a user or the system). Accordingly, selecting the first button 504 causes a set of operations to be performed that generate a summary of everyday threats (e.g., unauthorized access attempts, untrusted network connections, high network traffic, unusual usage, etc.).

[0056] In the illustrated example 500, the second button 506 is configured to display suggested prompt words. Accordingly, selecting the second button 506 may cause a set of operations to be performed that generates one or more suggested prompt words (such as prompt words that the user and / or system can enter into the user input feature 502). In addition, in the illustrated example, the third button 508 is configured to allow the user to drag and drop log files. For example, a user can provide a raw error log file that the system can analyze for potential security risks. In some examples, the mechanisms disclosed herein can analyze log files based on queries provided by the user. Additionally or alternatively, in some examples, the mechanisms provided herein can analyze log files based on historical knowledge and / or training for performing security analysis without any queries provided by the user.

[0057] Go to Figure 6 , user interface 600 includes a summary of the execution plan 602. Summary 602 may include natural language that a user may understand to determine what steps are proposed by the mechanisms described herein to resolve the provided prompt word.

[0058] The user interface 600 also includes a first button 604, a second button 606, and a third button 608. Although three buttons are shown on the user interface 600, one of ordinary skill in the art will recognize that additional, fewer, and / or alternative buttons may be used in alternative examples. In the illustrated example 600, the first button 604 is configured to run the execution plan. For example, the execution plan may be automatically executed by one or more computing devices (e.g., computing device 102 and / or server 104) in response to the first button 604 being selected.

[0059] In the illustrated example 600, the second button 606 is configured to display the code corresponding to the determined execution plan. The code can be one or more computer languages known to those skilled in the art, such as object-oriented languages, binary languages, procedural languages, high-level languages, low-level languages, etc.

[0060] Additionally, in the example shown, third button 608 is configured to escalate the incident. For example, selecting third button 608 may generate a notification that is transmitted to escalate the incident. This notification may be transmitted to a security analyst, information technology representative, supervisor, etc. Generally, if executing the plan does not satisfactorily resolve the provided prompt and / or if executing the plan's summary leads to further issues that require additional support, third button 608 provides the user with the ability to request further assistance.

[0061] Figure 7 An example method 700 for generating a safety report according to some aspects described herein is shown. In an example, aspects of the method 700 are performed by a device, such as described above with respect to Figure 1 The computing device 102 and / or server 104 is discussed.

[0062] Method 700 begins at operation 702, where a user query and security data are received. In some examples, the security data corresponds to one or more incidents associated with a computing environment. In some examples, the security data includes raw logs associated with one or more incidents (e.g., errors, breaches, failures, unexpected computer actions, etc.). In some examples, the computing environment is one or more of an endpoint, a network, a cloud environment, a security appliance, and / or a computer-executable application.

[0063] In some examples, the user query may be a security prompt, such as a security prompt that a user would provide to a security analyst to investigate a security-related matter. Some example user queries may relate to resetting a password, investigating unauthorized access, or determining the source of a computer error. However, such examples are merely examples. Additional and / or alternative examples of user queries, whether explicitly or implicitly related to security, may be recognized by one of ordinary skill in the art.

[0064] At operation 704, the user query and security data are provided to the semantic model. The semantic model generates one or more first embeddings. The one or more first embeddings can be semantic embeddings. In some examples, the one or more first embeddings can be target embeddings corresponding to the intent provided by the user. In addition, the semantic model can include a generative large language model (LLM). Additional and / or alternative types of semantic models may be recognized by those of ordinary skill in the art, based at least on the teachings provided herein.

[0065] In some instances, generative LLMs may use few-shot hints. For few-shot hints, preprocessing may occur for generating a new plan. For example, there may be a finite number of labeled or summarized data elements (e.g., embeddings), and predictions (e.g., new plans) may be generated based on this finite number of labeled data elements. In some instances, generative LLMs may use zero-shot hints. Examples may also utilize single-shot hints or zero-shot hints. For zero-shot hints, there may be no labels or summaries for the new data elements (e.g., embeddings), so that the algorithm may have to make predictions about the new data elements by using prior knowledge about the relationships that exist between the data elements (e.g., embeddings).

[0066] At operation 706, one or more second embeddings are received from the data model. The one or more second embeddings may be semantic embeddings. In addition, the one or more second embeddings may be template embeddings corresponding to historical execution plans and / or historical inputs. The data model is generated based on historical threat intelligence data. The threat intelligence data may include historical execution plans and / or historical inputs corresponding to the one or more second embeddings. In some examples, the threat intelligence data includes knowledge, skills and / or experience-based information about the occurrence and assessment of virtual threats, physical threats and / or threat actors. The threat intelligence data may be intended to help mitigate potential attacks and harmful events that may occur in one or more computing environments. Accordingly, historical threat intelligence data relates to previously collected threat intelligence data, which may be stored in a memory or otherwise accessible according to the mechanisms provided herein.

[0067] At operation 708, it is determined whether there is an execution plan associated with the one or more first embeddings and the one or more second embeddings. In some examples, source data associated with the first embedding and / or the second embedding can be located (e.g., on the device on which method 700 is being executed and / or remote from the device on which method 700 is being executed), and the plan can be further determined based on the source data. The source data can include one or more of an audio file, a text file, an image file, a video file, threat intelligence data, a security report, a log file, data generated by a particular software application, and the like.

[0068] If it is determined that there is no execution plan associated with the one or more first embeddings and the one or more second embeddings, flow branches "NO" to operation 710, where a default action is executed. For example, an embedding may have an associated preconfigured action. In other examples, method 700 may include determining whether an embedding has an associated default action so that, in some cases, no action may be executed as a result of receiving the embedding. Method 700 may terminate at operation 710. Alternatively, method 700 may return to operation 702 to provide an iterative loop of receiving a user query and security data, generating one or more embeddings, and determining whether there is an execution plan associated with the embedding.

[0069] However, if it is determined that a plan associated with the one or more first embeddings and the one or more second embeddings exists, the process branches "yes" to operation 712 instead, where an execution plan is generated based on the one or more first embeddings and the one or more second embeddings. In some examples, generating the execution plan includes determining a corresponding similarity between the second embeddings and the first embeddings. For example, the similarity can be determined based on a ranking and / or a distance measure (e.g., cosine distance, Euclidean distance, etc.). Generating the execution plan can also include determining instructions based on the similarity between the second embeddings and the first embeddings, and generating the execution plan based on the instructions.

[0070] In some examples, instructions may correspond to skills, such as Figure 4 The skills in question. For example, the skills may include one or more of: generating query language, reasoning, merging reasons, generating insights, splitting data, executing query language, validating query language, finding threat intelligence, extracting entities, and / or combining data. Additionally and / or alternatively, in some examples, the execution plan may include skills that were not previously stored. Accordingly, method 700 may include generating its own skills (e.g., in the form of computer-readable instructions) to perform the desired operations.

[0071] At operation 712, a report corresponding to the execution plan is returned. In some examples, the report includes one or more natural languages corresponding to instructions for resolving the security incident. In some examples, the report includes computer-readable instructions, and method 700 further includes executing instructions to perform a set of operations based on the execution plan. In some examples, method 700 includes automatically executing the execution plan via one or more processors (e.g., of a computing device (such as computing device 102) and / or a server device (such as server 104)).

[0072] In some examples, an execution plan is provided as an output. For example, the plan can be provided as an output to a user, a system on which method 700 is being executed, and / or a system remote from the system on which method 700 is being executed. In addition, in some examples, method 700 can also include adapting a computing device to execute the provided plan. The execution plan can be any of a plurality of different execution plans. For example, the plan can be a plan executed by a user and / or by a system. The plan can include instructions and / or information output to the user.

[0073] In some examples, user feedback is received based on the report. The updated one or more first embeddings can be generated based on the user feedback (e.g., by a model, such as a machine learning model, a large language model, etc.). In addition, the execution plan can be updated based on the updated one or more first embeddings and the one or more second embeddings. User feedback can allow the execution plan to be modified based on supervised learning (e.g., feedback provided by the user). In addition, user feedback can allow the execution plan to be personalized (e.g., for a specific user and / or organization). In some examples, the execution plan can be updated based on user feedback from a single user. Alternatively, in some examples, the execution plan can be updated based on user feedback from multiple users.

[0074] Figure 8A and Figure 8B An overview of an example generative machine learning model that can be used in accordance with aspects described herein is shown. Figure 8A , conceptual diagram 800 depicts an overview of a pre-trained generative model package 804 according to aspects described herein that processes input 802 to generate model output for use in generating an execution plan 806. Examples of pre-trained generative model packages 804 include, but are not limited to, the Megatron-Turing Natural Language Generation model (MT-NLG), Generative Pre-Trained Transformer 3 (GPT-3), Generative Pre-Trained Transformer 4 (GPT-4), BigScience BLOOM (Large Open Science Open Access Multilingual Language Model), DALL-E, DALL-E 2, Stable Diffusion, or Jukebox.

[0075] In an example, the generative model package 804 is pre-trained based on a variety of inputs (e.g., various human languages, various programming languages, and / or various content types) and, therefore, does not need to be fine-tuned or trained for a specific scenario. Instead, the generative model package 804 can be pre-trained more generally, such that the input 802 includes prompt words that are generated, selected, or otherwise designed to induce the generative model package 804 to produce certain generative model outputs 806. For example, the prompt word includes context and / or one or more completion prefixes, thereby pre-loading the generative model package 804 accordingly. As a result, the generative model package 804 is induced to generate an output based on the prompt word, which includes a predicted sequence of tokens associated with the prompt word (e.g., up to the token limit of the generative model package 804). In an example, the predicted sequence of tokens is further processed (e.g., by output decoding 816) to produce the output 806. For example, each token is processed to identify a corresponding word, word fragment, or other content that forms at least a portion of the output 806. It should be understood that the input 802 and the generative model output 806 can each include any of a variety of content types, including but not limited to text output, image output, audio output, video output, programmatic output, and / or binary output, etc. In an example, the input 802 and the generative model output 806 can have different content types, as may be the case when the generative model package 804 includes a generative multimodal machine learning model.

[0076] Thus, generative model package 804 can be used in any of a variety of scenarios, and further, a different generative model package can be used in place of generative model package 804 without substantially modifying other associated aspects (e.g., similar to the examples described herein with respect to Figures 1 to 7 Thus, generative model package 804 operates as a tool with which machine learning processing is performed, with certain inputs 802 to generative model package 804 being programmatically generated or otherwise determined, thereby causing generative model package 804 to produce model output 806, which can then be used for further processing.

[0077] The generative model package 804 can be provided or otherwise used according to any of a variety of paradigms. For example, the generative model package 804 can be provided or otherwise used on a computing device (e.g., Figure 1 The generative model package 804 may be used locally on the computing device 102 in the example, or may be accessed remotely from the machine learning service. In other examples, aspects of the generative model package 804 are distributed across multiple computing devices. In some instances, the generative model package 804 may be accessed via an application programming interface (API), such as may be provided by the operating system of the computing device and / or by the machine learning service in other examples.

[0078] Referring now to the illustrated aspects of the generative model package 804, the generative model package 804 includes input tokenization 808, input embedding 810, model layer 812, output layer 814, and output decoding 816. In the example, the input tokenization 808 processes the input 802 to generate the input embedding 810, which includes a sequence of symbolic representations corresponding to the input 802. Thus, the input embedding 810 is processed by the model layer 812, the output layer 814, and the output decoding 816 to produce the model output 806. Figure 8B An example architecture corresponding to the generative model package 804 is depicted in , which is discussed in further detail below. Even so, it should be understood that the architecture shown and described herein should not be considered limiting, and in other examples, any of a variety of other architectures may be used.

[0079] Figure 8B 850 is a conceptual diagram depicting an example architecture 850 of a pre-trained generative machine learning model that can be used according to aspects described herein. As described above, any of a variety of alternative architectures and corresponding ML models can be used in other examples without departing from the aspects described herein.

[0080] As illustrated, architecture 850 processes input 802 to produce generative model output 806, aspects of which are discussed above with respect to Figure 8A The architecture 850 is depicted as a transformer model comprising an encoder 852 and a decoder 854. The encoder 852 processes an input embedding 858 (which in various aspects may be similar to Figure 8A ), the input embedding 858 includes a sequence of symbolic representations corresponding to the input 856. In the example, the input 856 includes input content 802, which can include user input and / or machine-generated input, such as prompt words, commands, context, etc.

[0081] Furthermore, position encoding 860 may include information about the relative and / or absolute positions of tokens with respect to input embedding 858. Similarly, output embedding 874 includes a sequence of symbolic representations corresponding to output 872, and position encoding 876 may similarly include information about the relative and / or absolute positions of tokens with respect to output embedding 874.

[0082] As shown, the encoder 852 includes an example layer 870. It should be understood that any number of such layers can be used, and the depicted architecture is simplified for illustrative purposes. The example layer 870 includes two sublayers: a multi-head attention layer 862 and a feed-forward layer 866. In the example, residual connections are included around each layer 862, 866, and normalization layers 864 and 868 are included after layers 862, 866.

[0083] The decoder 854 includes an example layer 890. Similar to the encoder 852, any number of such layers can be used in other examples, and for illustrative purposes, the architecture of the decoder 852 depicted is simplified. As shown, the example layer 890 includes three sublayers: a masked multi-headed attention layer 878, a multi-headed attention layer 882, and a feedforward layer 886. The aspects of the multi-headed attention layer 882 and the feedforward layer 886 can be similar to those discussed above with respect to the multi-headed attention layer 862 and the feedforward layer 866, respectively. Additionally, the masked multi-headed attention layer 878 performs multi-headed attention on the output of the encoder 852 (e.g., output 872). In the example, the masked multi-headed attention layer 878 prevents a position from paying attention to subsequent positions. This masking combined with offset embedding (e.g., offsetting one position, as shown in the multi-headed attention layer 882) can ensure that the prediction for a given position depends on the known output of one or more positions smaller than the given position. As shown, residual connections are also included around layers 878, 882, and 886, and normalization layers 880, 884, and 888 are included after layers 878, 882, and 886, respectively.

[0084] The multi-head attention layers 862, 878, and 882 can each use a set of linear projections to linearly project the query, key, and value into corresponding dimensions. Each linear projection can be processed using an attention function (e.g., dot product or additive attention) to produce an n-dimensional output value for each linear projection. The resulting values can be concatenated and projected again so that the value is then as follows Figure 8B is processed as shown (e.g., by corresponding normalization layer 864, 880, or 884).

[0085] Feed-forward layers 866 and 886 can each be a fully connected feed-forward network applied to each position. In an example, feed-forward layers 866 and 886 each include multiple linear transformations with rectified linear unit activations therebetween. In an example, each linear transformation is the same across different positions, while different parameters can be used compared to other linear transformations of the feed-forward network.

[0086] Additionally, aspects of the linear transformation 892 can be similar to the linear transformations discussed above with respect to the multi-head attention layers 862, 878, and 882 and the feed-forward layers 866 and 886. The softmax 894 can also convert the output of the linear transformation 892 into a predicted next token probability, as indicated by the output probability 896. It should be understood that the illustrated architecture is provided as an example, and in other examples, any of a variety of other model architectures can be used in accordance with the disclosed aspects. In some instances, multiple iterations of processing are performed in accordance with the above aspects (e.g., using Figure 8A Generative model package 804 or Figure 8BThe encoder 852 and decoder 854 in the example are executed to generate a series of output tokens (e.g., words), which are then combined to produce a complete sentence (and / or any of a variety of other contents). It should be understood that other generative models can generate multiple output tokens in a single iteration and, therefore, can use a reduced number of iterations or a single iteration.

[0087] Accordingly, the output probabilities 896 can thus form the embedding output 806 according to aspects described herein, such that the output of the generative ML model (e.g., which can include structured outputs) is used as a proxy for determining an execution plan according to aspects described herein (e.g., similar to Figure 1 In other examples, output 806 is provided as output for the generation of an execution plan.

[0088] Figures 9 to 11 and the associated descriptions provide a discussion of various operating environments in which aspects of the present disclosure may be practiced. Figures 9 to 11 The devices and systems shown and discussed are for purposes of example and explanation and are not limiting of the vast number of computing device configurations that may be used to practice the aspects of the present disclosure described herein.

[0089] Figure 9 is a block diagram illustrating the physical components (eg, hardware) of a computing device 900 that may practice aspects of the present disclosure. The computing device components described below may be applicable to the computing devices described above, including Figure 1 1. In a basic configuration, computing device 900 may include at least one processing unit 902 and system memory 904. Depending on the configuration and type of computing device, system memory 904 may include, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of these memories.

[0090] System memory 904 may include an operating system 905 and one or more program modules 906 suitable for running software applications 920, such as one or more components supported by the system described herein. As an example, system memory 904 may store a goal generation engine 924 and / or an execution plan generation engine 926. Operating system 905 may, for example, be suitable for controlling the operation of computing device 900.

[0091] Furthermore, aspects of the present disclosure may be practiced in conjunction with graphics libraries, other operating systems, or any other application programs and are not limited to any particular application or system. Figure 9908. The computing device 900 may have additional features or functionality. For example, the computing device 900 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage may be Figure 9 909 and a non-removable storage device 910.

[0092] As described above, a number of program modules and data files may be stored in system memory 904. When executed on processing unit 902, program modules 906 (e.g., applications 920) may perform processes including, but not limited to, the aspects described herein. Other program modules that may be used in accordance with aspects of the present disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided applications, and the like.

[0093] Furthermore, aspects of the present disclosure may be practiced on circuits comprising discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. For example, aspects of the present disclosure may be practiced via a system on a chip (SOC) wherein Figure 9 Each or many components shown in can be integrated into a single integrated circuit. Such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit and various application functions, all of which are integrated (or "burned") onto a chip substrate as a single integrated circuit. When operated via the SOC, the functions described herein regarding the ability to switch protocols for the client can be operated via dedicated logic integrated on a single integrated circuit (chip) together with other components of the computing device 900. Some aspects of the present disclosure can also be practiced using other technologies (e.g., AND, OR, and NOT) capable of performing logical operations, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, some aspects of the present invention may be practiced within a general-purpose computer or in any other circuit or system.

[0094] The computing device 900 may also have one or more input devices 912, such as a keyboard, a mouse, a pen, an audio or voice input device, a touch or slide input device, etc. Output devices 914, such as a display, a speaker, a printer, etc. may also be included. The aforementioned devices are examples, and other devices may be used. The computing device 900 may include one or more communication connections 916 that allow communication with other computing devices 950. Examples of suitable communication connections 916 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuitry; universal serial bus (USB), parallel and / or serial ports.

[0095] The term computer-readable media as used herein may include computer storage media. Computer storage media may include volatile and non-volatile media, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures or program modules). System memory 904, removable storage device 909 and non-removable storage device 910 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other product that can be used to store information and can be accessed by computing device 900. Any such computer storage media may be part of computing device 900. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0096] Communication media may be embodied by computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0097] Figure 10 1002 is a block diagram illustrating an architecture of one aspect of a computing device. That is, a computing device can incorporate system (e.g., architecture) 1002 to implement some aspects. In some examples, system 1002 is implemented as a "smartphone" capable of running one or more applications (e.g., a browser, email, calendar, contact manager, messaging client, games, and media client / player). In some aspects, system 1002 is integrated into a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.

[0098] One or more application programs 1066 can be loaded into memory 1062 and run on or in association with operating system 1064. Examples of application programs include a phone dialer, an email program, a personal information management (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. System 1002 also includes a non-volatile storage area 1068 within memory 1062. Non-volatile storage area 1068 can be used to store persistent information that should not be lost if system 1002 loses power. Application programs 1066 can use and store information in non-volatile storage area 1068, such as emails or other messages used by an email application. A synchronization application (not shown) also resides on system 1002 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep information stored in non-volatile storage area 1068 synchronized with corresponding information stored on the host computer. It should be understood that other applications may be loaded into the memory 1062 and executed on the mobile computing device 1000 described herein (eg, an embedded object memory insertion engine, an embedded object memory retrieval engine, etc.).

[0099] The system 1002 has a power source 1070, which can be implemented as one or more batteries. The power source 1070 can also include an external power source, such as an AC adapter or a powered docking station to replenish or recharge the batteries.

[0100] System 1002 may also include a radio interface layer 1072 that performs the functions of transmitting and receiving radio frequency communications. Radio interface layer 1072 facilitates wireless connectivity between system 1002 and the "outside world" via a communications carrier or service provider. Transmissions to and from radio interface layer 1072 are controlled by operating system 1064. In other words, communications received by radio interface layer 1072 can be passed to application programs 1066 via operating system 1064, and vice versa.

[0101] The visual indicator 1020 can be used to provide a visual notification, and / or the audio interface 1074 can be used to generate an audible notification via the audio transducer 1025. In the example shown, the visual indicator 1020 is a light emitting diode (LED), and the audio transducer 1025 is a speaker. These devices can be directly coupled to the power supply 1070 so that when activated, they remain on for the duration indicated by the notification mechanism, even if the processor 1060 and / or the dedicated processor 1061 and other components may be turned off to save battery power. The LED can be programmed to remain on indefinitely until the user takes action to indicate the power-on status of the device. The audio interface 1074 is used to provide audible signals to the user and receive audible signals from the user. For example, in addition to being coupled to the audio transducer 1025, the audio interface 1074 can also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. According to aspects of the present disclosure, the microphone can also be used as an audio sensor to facilitate control of notifications, as described below. The system 1002 may also include a video interface 1076 that enables operation of the onboard camera 1030 to record still images, video streams, and the like.

[0102] The computing device implementing system 1002 may have additional features or functionality. For example, the computing device may also include additional data storage devices (removable and / or non-transportable), such as magnetic disks, optical disks, or tapes. Such additional storage may be Figure 10 denoted by non-volatile storage area 1068.

[0103] As described above, data / information generated or collected by a computing device and stored via system 1002 can be stored locally on the computing device, or the data can be stored on any number of storage media that can be accessed by the device via the radio interface layer 1072 or via a wired connection between the computing device and a separate computing device associated with the computing device (e.g., a server computer in a distributed computing network such as the Internet). It should be understood that such data / information can be accessed via the computing device via the radio interface layer 1072 or via a distributed computing network. Similarly, such data / information can be easily transferred between computing devices for storage and use according to well-known data / information transmission and storage components (including email and collaborative data / information sharing systems).

[0104] Figure 11One aspect of the architecture of a system for processing data received at a computing system from a remote source (e.g., a personal computer 1104, a tablet computing device 1106, or a mobile computing device 1108) is shown, as described above. The content displayed at the server device 1102 can be stored in different communication channels or other storage types. For example, a directory service 1124, a portal 1125, a mailbox service 1126, an instant messaging storage 1128, or a social networking site 1130 can be used to store various documents.

[0105] An application 1120 (e.g., similar to application 920) can be employed by a client communicating with server device 1102. Additionally or alternatively, a goal generation engine 1121 and / or an execution plan generation engine 1122 can be employed by server device 1102. Server device 1102 can provide data to and from client computing devices such as personal computers 1104, tablet computing devices 1106, and / or mobile computing devices 1108 (e.g., smartphones) via network 1115. As examples, the computer systems described above can be embodied in personal computers 1104, tablet computing devices 1106, and / or mobile computing devices 1108 (e.g., smartphones). In addition to receiving graphics data that can be pre-processed at a graphics originating system or post-processed at a receiving computing system, any of these examples of computing devices can obtain content from storage 1116.

[0106] For example, various aspects of the present disclosure are described above with reference to block diagrams and / or operational diagrams of methods, systems, and computer program products according to various aspects of the present disclosure. The functions / actions indicated in the blocks may not occur in the order shown in any flowchart. For example, depending on the functions / actions involved, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order.

[0107] The description and explanation of one or more aspects provided in this application are not intended to limit or restrict the scope of this disclosure in any way. The aspects, examples and details provided in this application are considered to be sufficient to convey the claimed aspects of the disclosure and enable others to make and use the claimed aspects of this disclosure. The disclosure claimed should not be interpreted as being limited to any aspect, example or details provided in this application. No matter whether shown and described in combination or individually, various features (both structure and method) are intended to be selectively included or omitted to produce an embodiment with a specific feature set. The description and explanation of this application have been provided, and those skilled in the art can envision variations, modifications and alternative aspects within the spirit of the broader aspects of the overall inventive concept embodied in this application, which variations, modifications and alternative aspects do not depart from the broader scope of the disclosure claimed.

Claims

1. A method for generating a safety report, the method comprising: Receive user queries and security data; providing the user query and the security data to a semantic model, wherein the semantic model generates one or more first embeddings; receiving one or more second embeddings from a data model, wherein the data model is generated based on historical threat intelligence data; generating an execution plan based on the one or more first embeddings and the one or more second embeddings; as well as Returns a report corresponding to the described execution plan.

2. The method of claim 1, wherein the semantic model comprises a generative large language model (LLM) using one of zero-shot cues, one-shot cues, or few-shot cues.

3. The method according to claim 1 , wherein generating the execution plan comprises: determining a corresponding similarity between the second embedding and the first embedding; determining an instruction based on the similarity between the second embedding and the first embedding; as well as The execution plan is generated based on the instructions. The method of claim 1 , wherein the report comprises one or more natural languages corresponding to instructions for resolving the security incident.

5. The method of claim 1 , wherein the report comprises computer-readable instructions, and wherein the method further comprises: The instructions are executed to perform a set of operations based on the execution plan.

6. The method according to claim 1, further comprising: receiving user feedback based on the report; generating updated one or more first embeddings based on the user feedback; as well as The execution plan is updated based on the updated one or more first embeddings and the one or more second embeddings.

7. The method of claim 1, wherein the security data corresponds to one or more incidents, the one or more incidents being associated with a computing environment. The method of claim 7 , wherein the safety data comprises raw logs associated with the one or more incidents.

9. A system for generating a safety report, the system comprising: processor; as well as a memory comprising instructions that, when executed by the processor, cause the system to perform a set of operations comprising: Receive user queries and security data; providing the user query and the security data to a semantic model, wherein the semantic model generates one or more first embeddings; receiving one or more second embeddings from a data model, wherein the data model is generated based on historical threat intelligence data; generating an execution plan based on the one or more first embeddings and the one or more second embeddings; and Returns a report corresponding to the described execution plan.

10. The system of claim 9, wherein the semantic model comprises a generative large language model (LLM) using one of zero-shot prompts, one-shot prompts, or few-shot prompts.

11. The system according to claim 9, wherein generating the execution plan comprises: determining a corresponding similarity between the second embedding and the first embedding; determining an instruction based on the similarity between the second embedding and the first embedding; as well as The execution plan is generated based on the instructions.

12. The system of claim 9, wherein the report includes one or more natural languages corresponding to instructions for resolving the security incident.

13. The system of claim 9, wherein the report comprises computer-readable instructions, and wherein the set of operations further comprises: The instructions are executed to perform a set of operations based on the execution plan.

14. A method for generating a safety report, the method comprising: Receive user queries and security data; providing the user query and the security data to a semantic model, wherein the semantic model generates one or more first embeddings; receiving one or more second embeddings from the data model; generating an execution plan based on the one or more first embeddings and the one or more second embeddings; as well as The execution plan is automatically executed via one or more processors.

15. The method of claim 14, wherein the security data comprises raw logs corresponding to one or more incidents associated with a computing environment.

Images