Implicit Push Data Transfer in Containerized Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In containerized computer systems, existing methods for communicating data between processes within different containers are inefficient, often requiring local collector processes that consume network bandwidth and complicate deployments, while also losing context information about which container or host generated metrics.
Innovation Solution
A method that allows data communication between a first process within a first container and a second process within a second container without the need for a local collector process, using input source instructions to monitor system calls, generate enriched messages, and transmit them to a metric collector, which aggregates and sends them to a monitoring application, thereby reducing bandwidth consumption and maintaining context information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If local collector processes are used for data communication between containers, then data collection capability is improved, but network bandwidth consumption increases and deployment complexity increases
Solution Approach 1:
The patent merges the metric collection functionality into a centralized collector process that receives data directly from application processes through shared memory or inter-process communication, eliminating the need for separate local collector processes in each container. This consolidation reduces network bandwidth consumption by avoiding redundant data collection layers while maintaining reliable data collection capability.
Solution Approach 2:
The centralized collector process performs multiple functions: it collects metrics from multiple containers, aggregates data, and provides context information about source containers and hosts. This universal approach replaces the need for specialized local collectors in each container, reducing overall system complexity and bandwidth usage.
2Reliability
If local collector processes are deployed in each container, then data collection is improved, but device complexity increases
Solution Approach 1:
The patent extracts the metric collection functionality from individual container processes and consolidates it into a separate centralized collector process. This extraction eliminates the need to embed collector agents in each container, significantly reducing deployment complexity while maintaining data collection reliability through the centralized service.
Solution Approach 2:
The centralized collector process acts as an intermediary between application processes and the monitoring system. It receives metrics directly from applications through system call monitoring or inter-process communication, eliminating the need for local collectors in each container and simplifying the overall system architecture.
3Productivity
If metrics are collected without context information, then data processing is simplified, but information completeness deteriorates
Solution Approach 1:
The centralized collector process enriches metrics with context information (container ID, host information, process details) at the time of collection, before data processing occurs. This preliminary enrichment ensures that context information is preserved without complicating subsequent data processing, as the contextual data is already attached to each metric record.
Solution Approach 2:
The collector process serves as an intermediary that adds context information to metrics as they are collected from application processes. By embedding container and host identification data at the collection stage, the system maintains information completeness while keeping data processing efficient, as the contextual information is already structured and attached.
Data Source
AI summary
A computer-implemented method comprises executing, in a first container of a first computer system, input source instructions; executing, using the same first computer system, a plurality of containerized application programs in different corresponding containers; monitoring, by the input source instructions, the one or more different containerized application programs by identifying one or more system calls that resulted from the different container applications generating statistical messages relating to operation of the containerized application programs; generating, by the input source instructions, one or more enriched messages based on the system calls that were identified and based on the statistical messages; transmitting the one or more enriched messages to a first metric collector, and aggregating a plurality of the enriched messages into a set of aggregated metrics values; sending, from the first metric collector to a monitoring application that is hosted on a second computer system, the aggregated metrics values.