DSN Performance Modeling With Neural Network Training

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing data storage systems face challenges in dynamically adjusting resource provisioning to match changing needs, leading to inefficiencies in resource allocation and data storage management.

Innovation Solution

A dispersed storage network (DSN) with a managing unit and integrity processing unit that uses error encoding techniques like Cauchy Reed-Solomon encoding to distribute data across multiple storage units, ensuring data integrity and availability even with storage unit failures, and a neural network model for predictive performance optimization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If data is stored using traditional RAID or dispersed storage systems with fixed resource provisioning, then data storage capacity is provided, but resource allocation efficiency deteriorates when needs change over time

Engineering Contradiction:
Improvedata storage reliabilityVSAvoidresource allocation adaptability
Core Design Contradiction:
ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic resource provisioning by using a neural network model that continuously learns from actual system performance data and adjusts resource allocation parameters in real-time. The system transitions from static, predetermined resource provisioning to dynamic adaptation based on changing workload conditions, enabling the storage system to optimize its performance characteristics adaptively.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent employs feedback mechanisms where the neural network model receives actual performance measurements from the dispersed storage system and uses this information to refine its predictions and adjust resource allocation. This closed-loop feedback enables the system to learn from past performance and continuously improve resource provisioning decisions, resolving the contradiction between reliability and adaptability.

Inventive Principle:
Principle #23Feedback

2Reliability

If redundant copies of data are stored to ensure availability, then data availability is improved, but storage efficiency and resource utilization deteriorate

Engineering Contradiction:
Improvedata availabilityVSAvoidstorage resource utilization
Core Design Contradiction:
ReliabilityVSQuantity of substance

Solution Approach 1:

The patent changes the fundamental parameter of data representation by using error correction codes (such as Reed-Solomon or Cauchy Reed-Solomon encoding) instead of simple redundant copying. This encoding transforms data into a form where any k out of n encoded slices can reconstruct the original data, providing both availability and efficient resource utilization through mathematical optimization rather than brute-force redundancy.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent creates a composite storage structure by combining multiple encoded data slices with error correction capabilities. Instead of storing identical copies, the system stores complementary encoded portions that work together to ensure data availability, similar to how composite materials combine different substances to achieve superior properties while using materials efficiently.

Inventive Principle:
Principle #40Composite materials

3Ease of manufacture

If fixed resource provisioning is used initially, then system setup is simple, but performance optimization deteriorates when needs change over time

Engineering Contradiction:
Improvesystem setup simplicityVSAvoidsystem performance
Core Design Contradiction:
Ease of manufactureVSProductivity

Solution Approach 1:

The patent implements self-service optimization where the neural network model autonomously learns from system performance data and automatically adjusts resource allocation parameters without requiring manual reconfiguration. The system serves itself by continuously monitoring performance metrics and making intelligent decisions about resource provisioning, maintaining both simplicity and high performance simultaneously.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS10089178B2Developing an accurate dispersed storage network memory performance model through training
Publication Date: 2018.10.02 PURE STORAGE INC
  • US10089178B2 patent drawing
  • US10089178B2 patent drawing
  • US10089178B2 patent drawing

AI summary

A computing device includes an interface configured to interface and communicate with a dispersed or distributed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device receives first samples corresponding to inputs that characterize configuration of the DSN and receives second samples corresponding to outputs that characterize system behavior of the DSN. The computing device then processes the first and samples to generate a DSN model to generate predictive performance of the outputs based on various values of the inputs. In some instances, the DSN model is based on a neural network model that employs the inputs that characterize the configuration of the DSN and generates the outputs that characterize system behavior of the DSN.