Distributed Data Processing Reusing Past Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed processing systems, reusing past processing results is challenging when there are differences between current and past input data sets, as the summed results from past processes cannot be easily divided to match the current data set, leading to difficulties in utilizing past processing outcomes effectively.
Innovation Solution
A method is implemented where the system searches for overlapping results from past data sets, acquires results specific to differential data, and removes the effect of differential data from past results to generate relevant outcomes for the current data set, thereby improving the reusability of past processing results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If past processing results are reused for current data sets, then processing efficiency is improved, but the system cannot handle differences between current and past input data sets
Solution Approach 1:
The patent segments the past input data set and current input data set into overlapping and non-overlapping portions. By dividing the data sets and identifying the overlapping portion, the system can selectively reuse processing results only for the overlapping data, thereby handling differences between data sets while maintaining processing efficiency through partial result reuse.
2Speed
If complete past processing results are used, then processing speed is improved, but the results cannot be accurately matched to current data set requirements
Solution Approach 1:
The patent extracts and identifies the overlapping portion between the past input data set and the current input data set. By taking out only the relevant overlapping data portion, the system retrieves and uses only the corresponding past processing results that accurately match the current data set requirements, ensuring both speed and accuracy.
3Use of energy by moving object
If past results are stored and reused, then computational load is reduced, but the complexity of matching and adjusting results increases
Solution Approach 1:
The patent performs preliminary identification of the overlapping portion between data sets before retrieving past processing results. By预先 identifying which portions overlap, the system simplifies the subsequent result retrieval and adjustment process, reducing both computational load and the complexity of matching results.
Data Source
AI summary
In a data processing method, a storage apparatus stores one or more results of a second process, corresponding to input data sets processed in the past. When a first input data set is designated, a search unit searches the storage apparatus for a result of the second process, corresponding to a second input data set having an overlapping part with the first input data set. A data processing unit acquires a result of the first process, corresponding to differential input data that is included in the second input data set but not included in the first input data set, and generates a result of the second process, corresponding to the first input data set, by removing the effect of the differential input data from the result of the second process found as a result of the search, using the acquired result of the first process.