Shared Parse Scope for Cross-Computer Data De-Duplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data de-duplication environments involving multiple computers, the lack of a standardized approach to chunking and hashing leads to difficulties in collaboration, as each computer may implement different methods, making it challenging for them to work together effectively.
Innovation Solution
The establishment of a shared parse scope through the communication and negotiation of de-duplication rules between computers, using parse scope logic to determine and persist a common set of rules for data processing, chunking, hashing, and indexing, ensuring a deterministic approach across all participating computers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each computer implements its own de-duplication approach independently, then each computer can optimize its own performance, but collaboration between computers becomes difficult or impossible
Solution Approach 1:
The patent changes the parameters of de-duplication processing by introducing parse scope information that defines specific parameters for chunking, hashing, and indexing. This allows different computers to adjust their processing parameters to match a common standard, enabling collaboration while maintaining optimized performance through parameter negotiation and adaptation.
Solution Approach 2:
The patent introduces parse scope information as an intermediary element that mediates between different de-duplication implementations. This intermediary carries the necessary control information about how data should be parsed, chunked, and hashed, allowing computers with different native approaches to collaborate by translating their operations through this common intermediary representation.
2Reliability
If a central control dictates conditions and parameters for communication, then coordination between computers is achieved, but the system becomes complex and difficult to scale
Solution Approach 1:
The patent extracts the control information about de-duplication parameters from a central control authority and embeds it directly into the parse scope information that travels with the data. This eliminates the need for complex centralized control mechanisms while maintaining reliable coordination, as each computer can independently interpret and follow the parse scope instructions.
Solution Approach 2:
The patent performs preliminary action by establishing the parse scope information before the actual de-duplication processing begins. This preliminary definition of chunking boundaries, hashing algorithms, and indexing strategies ensures reliable coordination without requiring complex runtime control mechanisms, as all decisions are predetermined in the parse scope data.
3Adaptability or versatility
If de-duplication systems become more varied and sophisticated with multi-layered approaches, then processing capability improves, but consistency in deterministic processing across different computers deteriorates
Solution Approach 1:
The patent uses parameter changes to reconcile sophisticated multi-layered de-duplication approaches with processing consistency. By explicitly defining parameters in the parse scope information (chunk size, hash algorithm, indexing method), the system allows computers to use their sophisticated processing capabilities while ensuring they all follow the same deterministic rules for a given data set.
Data Source
AI summary
Example apparatus, methods, and computers control establishing a shared parse scope between two computers that intend to be involved in a shared de-duplication action. One example method includes, upon determining that a first de-duplication logic and a second de-duplication logic are to participate in a shared de-duplication action for an object, controlling the first de-duplication logic to establish a shared parse scope with the second de-duplication logic. Establishing the shared parse scope may include negotiations between the computers, where the negotiations transfer dialect information. The dialect information may take the form of rules. The method may also include persisting the shared parse scope.