Distributed File System Directory Attribute Acquisition Via Indexed Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Concurrent operations in distributed file systems lead to transaction conflicts and low throughput due to the need for serial execution of directory updates, affecting the efficiency and accuracy of attribute information acquisition.
Innovation Solution
Implement a memory queue to store attribute modification information, using log and transaction indexes to determine the source of attribute information, allowing for asynchronous updates and avoiding transaction conflicts by reading from both the memory queue and attribute file based on index comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If serial execution of directory updates is implemented to ensure data consistency, then transaction conflicts are avoided, but throughput decreases
Solution Approach 1:
The patent segments the attribute information acquisition process into two independent paths: one for committed transactions (using attribute files) and one for uncommitted transactions (using memory queue). This segmentation allows parallel processing of different transaction states, resolving the contradiction between maintaining data consistency and achieving high throughput.
Solution Approach 2:
The patent introduces a memory queue as an intermediary component between the attribute file and the directory update process. The memory queue buffers attribute modification information for uncommitted transactions, allowing the system to accept and queue multiple concurrent updates without blocking, thereby maintaining both consistency and high throughput.
2Measurement precision
If attribute information is always read from the attribute file to ensure accuracy, then data accuracy is maintained, but acquisition speed decreases
Solution Approach 1:
The patent implements a dynamic attribute information acquisition strategy that adapts based on transaction commit status. The system dynamically selects the data source (memory queue or attribute file) according to the transaction index comparison, switching between fast but potentially uncommitted data and slow but committed data, thereby optimizing both speed and accuracy.
Solution Approach 2:
The patent uses feedback from the transaction index and log index comparison to determine the appropriate data source. By continuously monitoring the transaction index and comparing it with the log index, the system provides feedback control that directs attribute information acquisition to the correct source, ensuring accuracy while maximizing speed.
3Productivity
If concurrent operations are allowed to improve throughput, then processing capacity increases, but transaction conflicts increase
Solution Approach 1:
The patent resolves transaction conflicts by adding a temporal dimension to the attribute information storage architecture. It creates a time-based separation between committed and uncommitted transactions using the transaction index, allowing multiple concurrent operations to proceed in parallel without conflicts by directing them to appropriate time-stamped data sources.
Solution Approach 2:
The patent creates a copy mechanism where attribute modification information is first written to the memory queue (a temporary copy) before being committed to the attribute file (the permanent copy). This copying approach allows concurrent operations to read from the appropriate copy based on commit status, eliminating conflicts while maintaining high processing capacity.
Data Source
AI summary
The present disclosure provides an acquisition method and apparatus for attribute information of a directory of a distributed file system, and a device, and relates to the field of computer technologies and, in particular, to the field of big data and a distributed file system. The specific implementation is as follows: when an acquisition instruction is acquired, judging whether a log index of a directory indicated by the acquisition instruction is less than a transaction index of the directory, and whether a queue index of the directory is greater than or equal to the transaction index; if both of the above conditions are met, reading attribute modification information of a directory from a memory queue; and determining the attribute information of the current directory according to the attribute modification information and an attribute file of the directory.