Docker Container High Availability Big Data Framework

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing big data framework systems face challenges in maximizing server availability and efficiently collecting structured and unstructured data from renewable energy sources, particularly due to high costs and resource requirements associated with hardware dualization and operating system virtualization.

Innovation Solution

A container-based high availability big data framework system utilizing Docker container technology to construct database servers, with dualized container modules for improved availability, and a method involving a data collection server that uses Docker containers to collect and store data, including structured data from external servers and unstructured data via FTP, with error recovery mechanisms and conversion using cx_Freeze library.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If hardware dualization is used to maximize server availability, then availability is improved, but cost and resource requirements increase

Engineering Contradiction:
Improveserver availabilityVSAvoidhardware resources
Core Design Contradiction:
ReliabilityVSQuantity of substance

Solution Approach 1:

The patent uses Docker container technology to create virtual copies of database servers. Instead of duplicating physical hardware, the system creates containerized virtual instances that can be rapidly deployed and managed. The containerization approach allows multiple database server instances to run on shared physical infrastructure, reducing hardware requirements while maintaining availability through virtual duplication.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent introduces a container orchestration layer as an intermediary between physical hardware and database services. This intermediary manages resource allocation, container lifecycle, and failover mechanisms, enabling efficient utilization of underlying hardware resources while maintaining high availability through automated container management and recovery.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Reliability

If operating system level virtualization is used, then server availability is improved, but resource overhead and complexity increase

Engineering Contradiction:
Improveserver availabilityVSAvoidvirtualization complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the database server into independent containerized units using Docker technology. Each database instance runs in its own container with isolated file systems, processes, and dependencies. This segmentation allows individual containers to be managed, updated, and recovered independently without affecting the entire system, reducing operational complexity while maintaining availability.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent employs ephemeral container instances that can be rapidly created and destroyed. When failures occur or updates are needed, new container instances are spawned and activated quickly, while failed containers are discarded and recreated. This approach reduces complexity by avoiding long-term management of virtualized operating systems and enables simple restart-based recovery mechanisms.

Inventive Principle:
Principle #27Cheap short-living objects (Disposable)

3Reliability

If container dualization is implemented, then availability is maximized and costs are reduced, but system complexity increases

Engineering Contradiction:
Improveserver availabilityVSAvoidcontainer management complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent creates universal container templates that can serve multiple database instances and configurations. A single container image can be instantiated multiple times with different parameters to create dualized database servers. This multi-functionality approach reduces management complexity by using standardized, reusable container definitions rather than custom configurations for each instance.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent manages container dualization by changing parameters within standardized container templates rather than creating fundamentally different system configurations. Availability is achieved by modifying instance parameters (such as container IDs, network assignments, and data paths) of otherwise identical container templates, simplifying management through parameter-based differentiation rather than structural complexity.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS20240211296A1Container-based high availability big data framework system and operating method thereof
Publication Date: 2024.06.27 KOREA ELECTRONICS TECH INST
  • US20240211296A1 patent drawing
  • US20240211296A1 patent drawing
  • US20240211296A1 patent drawing

AI summary

Proposed is a container-based high availability big data framework system. The system may include an external server configured to provide structured data. The system may also include a data collection server constructed based on a docker container environment and configured to collect structured data by requesting the structured data from the external server at a predetermined time interval. The system may further include a monitoring server configured to execute a new docker image by generating a trigger when detecting the occurrence of an error while the structured data are collected.