Training Data Stream Segmentation for Edge Intelligence Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current solutions for edge intelligence struggle to efficiently manage complex data sources, leading to inefficiencies and high costs in data processing and storage.

Innovation Solution

The method involves defining clear data streams within a storage pool, where new training data is stored in a collected data stream, refined with historical data to create refined training data stored in a refined data stream, and ultimately integrated into a full data stream for comprehensive data management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If new training data is continuously stored in a single storage pool without structured organization, then data collection is simple, but data management complexity and storage costs increase significantly

Engineering Contradiction:

Improvedata collection simplicityVSAvoiddata management complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The storage pool is segmented into three distinct data streams: collected data stream for raw data ingestion, refined data stream for processed training data, and full data stream for complete datasets. This segmentation organizes data by processing stage, simplifying management while maintaining collection efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The collected data stream acts as an intermediary buffer between data collection and refinement processes. Raw data flows into this intermediate stream first, allowing decoupling of collection and processing operations, thereby reducing management complexity without affecting collection simplicity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If all training data is stored in full detail without refinement, then data completeness is maintained, but storage costs and processing time increase

Engineering Contradiction:

Improvedata completenessVSAvoidstorage and processing cost

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

Data is segmented into different refinement levels across three streams. The refined data stream contains processed, de-noised training data for efficient model training, while the full data stream preserves complete raw data for verification or reprocessing, balancing completeness with efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different data streams have different quality characteristics tailored to their purposes. The refined data stream has high quality (de-noised, processed) data optimized for training, while the collected data stream maintains raw completeness. This local quality differentiation reduces overall storage and processing costs while maintaining necessary data integrity.

Inventive Principle:

Principle #3Local quality

3Manufacturing precision

If data refinement processes are applied to all collected data, then training data quality improves, but processing time and computational resources increase

Engineering Contradiction:

Improvetraining data qualityVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The refinement process is segmented and applied selectively. Only data in the collected data stream that requires processing is refined and moved to the refined data stream, while other data can be directly utilized or stored in the full data stream without intensive processing, reducing overall processing time while maintaining quality where needed.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Refinement is applied partially rather than universally. The system refines data to the necessary degree for training purposes without excessive processing, balancing quality improvement with processing efficiency. Not all data requires the same level of refinement, and the system adapts the processing intensity accordingly.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12271829B2Method, electronic device, and computer program product for managing training data

Publication Date: 2025.04.08 DELL PROD LP

AI summary

In a method for managing training data in an illustrative embodiment, in response to a determination that new training data is collected by a sensor, the new training data is stored into a collected data stream of a storage pool; in response to a determination that the new training data and historical data stored in a full data stream of the storage pool are refined into refined training data, the refined training data is stored into a refined data stream of the storage pool; and the new training data is stored into the full data stream. In this way, data streams become clear and storage costs are reduced. This greatly reduces the difficulty of developing complex scenarios such as autonomous driving.