Containerized SIEM Deployment Architecture for Scalable Data Ingestion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current security solutions struggle to effectively leverage exponential data growth in security analytics due to limitations in data ingestion, processing, and storage, leading to significant blind spots in IT security and operations.
Innovation Solution
A containerized application software deployment architecture is implemented, allowing multiple instances of SIEM application components to run on each host, optimized for resource utilization, with dedicated hosts for ingress gateways and flexible storage configurations, enhancing data ingestion rates and system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional security solutions are used to process security data, then data processing capability is limited, but infrastructure cost and complexity increase to handle exponential data growth
Solution Approach 1:
The patent segments the SIEM application into multiple independent containerized components (ingress gateways, indexers, search heads, storage units) that can be deployed and scaled independently across multiple hosts. This modular architecture enables parallel processing of security data, significantly increasing throughput and processing capability while distributing infrastructure complexity across manageable units rather than requiring a monolithic complex system
Solution Approach 2:
The containerized architecture creates universal building blocks that can perform multiple functions. For example, indexer containers can both ingest and index data, search head containers can query and analyze data, and the same container types can be replicated across different hosts to handle various data streams. This multi-functionality increases processing capability through resource sharing and eliminates the need for specialized hardware for each function, reducing overall infrastructure complexity
2Productivity
If more infrastructure is deployed to handle increased data rates, then data ingestion capability improves, but resource utilization decreases
Solution Approach 1:
The patent merges multiple SIEM application instances into a coordinated federated cluster where containers are distributed across multiple hosts. Instead of deploying separate monolithic SIEM systems on each host (which would waste resources), the system combines multiple lightweight container instances that share common infrastructure resources such as storage backends, network connections, and processing power. This merging approach increases total data ingestion rate across the cluster while maintaining high resource utilization through shared resources
Solution Approach 2:
The patent transitions from a single-dimension scaling approach (adding more powerful individual servers) to a multi-dimensional distributed architecture where scalability is achieved by adding container instances across multiple hosts in a federated cluster. This dimensional shift allows the system to increase data ingestion rates by distributing workloads across the cluster while efficiently utilizing the aggregate resources of all hosts, rather than requiring each individual host to be over-provisioned
3Use of energy by moving object
If containerized architecture is implemented to improve resource utilization, then infrastructure efficiency increases, but system complexity increases
Solution Approach 1:
The containerized SIEM components are designed to be self-service in nature, with each container instance automatically managing its own configuration, resource allocation, and operational parameters. The federated cluster enables automatic service discovery and coordination between components, reducing the need for manual configuration and management. This self-service capability allows the system to achieve high resource utilization through automated resource management while offsetting the increased deployment complexity through reduced operational overhead
Data Source
AI summary
Embodiments described herein are generally directed to a containerized application software deployment architecture. According to an example, a system includes multiple hosts that are part of aa stretch cluster spanning multiple data centers. Each of the hosts include a processing resource, a memory, and a storage device. Each host of a first subset of the multiple hosts runs multiple containerized instances of a component of a Security Information and Event Management (SIEM) application within respective containers. At least one host of the multiple hosts is separate from the first subset and is dedicated to running at least one containerized ingress gateway application operable to load balance requests directed to the SIEM application among the multiple containerized instances running on the first subset of hosts.