Streaming Data Replication Prioritization via Value Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack efficient methods for replicating streaming data from IoT devices to data storage clusters based on the importance of the data for various target applications, leading to delayed decision-making and increased costs due to inefficient data processing and replication techniques.
Innovation Solution
A method that characterizes streaming data records by determining attribute values, assigning category levels, ranking these levels based on their importance to specific applications, and computing a replication value to decide on immediate, lazy, or no replication, ensuring that highly valued data is prioritized for immediate replication while less valuable data is processed later.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all streaming data records are replicated immediately to data storage clusters, then data availability for applications is improved, but system resource consumption and processing costs increase
Solution Approach 1:
The patent applies local quality by assigning different replication priorities to different data records based on their specific characteristics and application requirements. Instead of uniform immediate replication, the system evaluates each record's attribute values against application-specific criteria to determine individual replication urgency, ensuring resources are allocated efficiently to only those records that applications truly need immediately.
Solution Approach 2:
The system changes the parameter of replication timing from a fixed immediate replication policy to a dynamic policy based on computed replication values. The replication value is derived from comparing data record attribute values with application-specific expected ranges, allowing the system to adjust replication timing parameters based on actual data characteristics and application needs.
2Loss of energy
If data replication is delayed to reduce processing costs, then system resource consumption decreases, but decision-making time for applications increases
Solution Approach 1:
The system performs preliminary evaluation of data records by computing replication values before actual replication occurs. This preliminary action involves comparing attribute values with application-specific criteria to predict which records will be high-value for replication, allowing the system to prepare and prioritize replication tasks in advance rather than reacting to application requests in real-time.
Solution Approach 2:
The patent implements feedback mechanisms where application performance metrics and data usage patterns are continuously monitored. This feedback informs the replication value computation, allowing the system to learn from past decisions and adjust replication priorities dynamically. The system uses feedback to refine which attribute comparisons are most valuable for predicting application needs.
3Productivity
If selective replication based on attribute analysis is implemented, then replication efficiency is improved, but system complexity increases
Solution Approach 1:
The patent segments the data replication process into distinct modular components: attribute value extraction, comparison with expected ranges, replication value computation, and replication execution. Each component handles a specific aspect of the decision-making process independently, making the overall complex system manageable through clear separation of concerns and enabling parallel processing of different data records.
Solution Approach 2:
The system introduces an intermediary replication value computation layer between data record identification and actual replication execution. This intermediary computes a numeric replication value based on attribute comparisons, serving as a mediator that translates complex multi-factor analysis into a simple prioritization metric that can be easily acted upon by the replication mechanism.
4Measurement precision
If comprehensive data characterization is performed for each data record, then replication accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies partial action by selectively analyzing only the most relevant attribute values for each data record based on application-specific criteria. Rather than performing exhaustive analysis of all possible attributes, the system identifies and compares only those attributes that are critical for the target applications, achieving sufficient replication accuracy with reduced processing effort.
Data Source
AI summary
Application-specific prioritization of streaming data replication. Data streamed from connected devices is selectively replicated to data storage clusters based on needs of the applications being served by the data. Data characterization supports prioritized replication processing. Statistical metrics compare streaming data with estimated values to characterize the data for prioritization.