Playbook Custom Code Execution via On-Premises Automation Broker
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing and searching massive quantities of machine data generated by modern computing environments is challenging due to the diverse types and formats of data, which can be time-consuming and inefficient with conventional pre-processing methods.
Innovation Solution
A data intake and query system utilizing a late-binding schema and flexible extraction rules to process and search machine data at search time, enabling flexible schema development and refinement based on user insights, and utilizing a metadata catalog for field extraction and indexing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional pre-processing methods are used to extract and store only pre-specified data items, then storage requirements are reduced and retrieval efficiency is improved, but data flexibility and analysis capability are worsened
Solution Approach 1:
The system performs preliminary indexing of all machine data at ingestion time, creating a searchable metadata catalog without discarding raw data. This preliminary action enables both efficient retrieval (by having data ready-indexed) and flexible analysis (by preserving all original data for later exploration)
Solution Approach 2:
The system extracts and stores metadata and indexes from machine data while retaining the full raw data. This extraction approach provides efficient access to key information through metadata while preserving the complete data set for comprehensive analysis when needed
2Adaptability or versatility
If all machine data is stored minimally processed for later retrieval, then data flexibility and analysis capability are improved, but processing time and computational resources are worsened
Solution Approach 1:
The system performs preliminary indexing and metadata extraction at data ingestion time, organizing all machine data into a searchable structure before retrieval is needed. This upfront preparation eliminates the need to process raw data during searches, reducing analysis time while maintaining full data flexibility
Solution Approach 2:
The system implements a dynamic schema that can adapt to different analysis needs. The metadata catalog and extraction rules can be refined and developed based on user insights, allowing the system to optimize for specific query patterns while maintaining the ability to handle diverse analysis requirements
3Speed
If custom code blocks are executed within the playbook runtime environment, then execution speed is improved, but system security and stability are worsened due to potential malicious or erroneous code
Solution Approach 1:
The system introduces a sandboxed execution environment as an intermediary between the custom code and the host system. This sandbox acts as a mediator that allows code to execute with necessary permissions while preventing it from accessing or compromising critical system resources, thus maintaining both speed and security
Solution Approach 2:
The execution environment is segmented into isolated containers or sandboxes for each custom code block. This segmentation ensures that even if one code block is malicious or erroneous, its impact is contained within its own isolated environment and cannot affect other code blocks or the host system
Data Source
AI summary
Techniques are described for enabling a cloud-based IT and security operations application to execute playbooks containing custom code in a manner that mitigates types of risk related to the misuse of cloud-based resources and security of user data. Users use a client application to create and modify playbooks and, upon receiving input to save a playbook, the client application determines whether the playbook includes custom code. If the client application determines that the playbook includes custom code, the client application establishes a connection with a proxy application (also referred to as an “automation broker”) running in the user's own on-premises network and sends a representation of the playbook to the proxy application. The client application further sends to the IT and security operations application an identifier of the playbook and an indication that the playbook (or the custom code portions of the playbook) is stored within the user's on-premises network.