Dynamic Scaling System for Cloud Big Data Resource Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current cloud computing systems lack an efficient mechanism for automatically allocating computing resources and adjusting virtual hosts and system parameters, leading to suboptimal use of limited resources and inefficiencies in big data processing.

Innovation Solution

A dynamic scaling system that performs profiling, classification, and prediction to determine the optimal computing node number and system parameters for tasks, ensuring fair resource allocation and maximizing efficiency by automatically adjusting virtual hosts in big data cloud computing architectures.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If users adjust or set the number of virtual hosts and system parameters by themselves based on experience, then users can satisfy their own needs, but the allocation of limited resources becomes unfair and the overall efficiency of the computing system degrades

Engineering Contradiction:

Improveuser ability to adjust virtual hostsVSAvoidoverall efficiency of computing system

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system enables self-service by allowing users to adjust virtual hosts and system parameters through an automated resource optimization mechanism that analyzes task characteristics and automatically determines optimal configurations, eliminating the need for manual experience-based adjustments while maintaining user needs satisfaction

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The resource optimization mechanism implements feedback by continuously monitoring task execution characteristics, resource usage patterns, and system performance, then using this information to dynamically adjust virtual host allocation and system parameters, ensuring fair and efficient resource distribution across multiple users

Inventive Principle:

Principle #23Feedback

2Ease of operation

If users adjust or set the number of virtual hosts and system parameters by themselves, then users can meet their requirements, but it becomes impossible to optimize use of the limited resources

Engineering Contradiction:

Improveuser control over virtual hostsVSAvoidwaste of limited computing resources

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The system replaces manual user control with an automated resource optimization mechanism that independently analyzes task requirements and optimizes virtual host allocation, maintaining ease of operation through automated decision-making while eliminating resource waste associated with suboptimal manual configurations

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The resource optimization mechanism dynamically changes system parameters including virtual host count, memory allocation, and processing configurations based on real-time analysis of task characteristics and resource availability, optimizing resource utilization without requiring direct user intervention

Inventive Principle:

Principle #35Parameter changes

3Loss of time

If the number of virtual hosts and system parameters are adjusted by users based on experience rather than effective analysis, then users can make quick decisions, but the computing resources are not effectively allocated

Engineering Contradiction:

Improvetime for resource allocation decisionVSAvoidaccuracy of resource allocation

Core Design Contradiction:

Loss of timeVSMeasurement precision

Solution Approach 1:

The system replaces the mechanical process of manual experience-based decision-making with an automated computational mechanism that performs effective analysis of task characteristics and resource requirements, achieving both rapid decision-making and high allocation accuracy through algorithmic optimization

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The resource optimization mechanism acts as an intermediary between user requests and computing resource allocation, performing automated analysis and optimization to bridge the gap between quick user decisions and accurate resource distribution, eliminating the need for users to possess deep technical knowledge

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10460241B2Server and cloud computing resource optimization method thereof for cloud big data computing architecture

Publication Date: 2019.10.29 INSTITUTE FOR INFORMATION INDUSTRY

AI summary

A server and a cloud computing resource optimization method thereof for big data cloud computing architecture are provided. The server runs a dynamic scaling system to perform the following operations: receiving a task message; executing a profiling procedure to generate a profile based on an to-be-executed task recorded in the task message; executing a classifying procedure to determine a task classification of the to-be-executed task; executing a prediction procedure to obtain a plurality of predicted execution times corresponding to a plurality of computing node numbers, a computing node type and a system parameter of the to-be-executed task; executing an optimization procedure to determine a practical computing node number of the to-be-executed task; and transmitting an optimization output message to a management server to make the management server allocate at least one data computing system to execute a program file of the to-be-executed task.