Decentralized file data retrieval cache acceleration technology

Abstract

The invention discloses a decentralized file data retrieval cache acceleration technology, which effectively solves the problems of single-point failure, low data retrieval efficiency and inconsistent cache data in a traditional centralized file retrieval system by constructing a distributed hash table (DHT) network and recording a cache state by utilizing a block chain technology. The file indexing module is used for generating a unique hash value for file data in the DHT network and storing the unique hash value in a plurality of nodes in the network to realize decentralized file indexing; the non-tampering and decentration characteristics of the block chain are utilized, the cache state of each node is recorded and synchronized, and the consistency and security of cached data are ensured; the efficient retrieval algorithm is designed in combination with the file hash value and the cache state information, the network request frequency is reduced, and the data retrieval speed is increased.

Description

Technical Field

[0001] The present invention relates to the field of data storage and retrieval technology, and in particular to a decentralized file data retrieval cache acceleration technology. Background Art

[0002] With the advent of the big data era, the amount of file data has exploded. Traditional centralized file storage and retrieval systems face problems such as single point failure, slow data retrieval, and low cache utilization. Centralized systems rely on some high-performance servers for data processing and storage. Once these servers fail, the entire system will be paralyzed. In addition, as the amount of data increases, the processing power and storage space of centralized servers gradually reach bottlenecks, making it difficult to meet the needs of efficient retrieval. Therefore, decentralized technology has gradually become a key way to solve these problems. Summary of the invention

[0003] The technical problem of the present invention is to solve how to construct a decentralized file data retrieval cache acceleration technology to improve the retrieval speed and cache utilization of file data, while reducing the system's dependence on a single node and improving the overall reliability and scalability of the system.

[0004] The technical solution adopted by the present invention to solve the above problems is: a decentralized file data retrieval cache acceleration technology, which effectively solves the problems of single point failure, low data retrieval efficiency and inconsistent cache data in traditional centralized file retrieval systems by constructing a distributed hash table DHT network and using blockchain technology to record the cache status; the technology is used to generate a unique hash value for file data in the DHT network and store it in multiple nodes in the network to achieve decentralized file indexing; the technology utilizes the immutability and decentralization characteristics of blockchain to record and synchronize the cache status of each node to ensure the consistency and security of cache data; the technology combines file hash values ​​and cache status information to design an efficient retrieval algorithm to reduce the number of network requests and improve data retrieval speed.

[0005] The present invention provides a decentralized file data retrieval cache acceleration technology, which includes the following steps:

[0006] The first step is to generate a unique hash value using a hash algorithm for the file data to be retrieved;

[0007] The second step is to store the file hash value and its metadata in multiple nodes of the DHT network to build a distributed index;

[0008] The third step is to use blockchain technology to record the cache status of each node, including the cached file hash value and cache timestamp information;

[0009] Step 4: executing an efficient retrieval algorithm based on user requests, combining distributed indexes and cache status information, and dynamically adjusting cache content during the retrieval process;

[0010] Step 5: Using digital signature and encryption technology to ensure the integrity of file data and security during transmission;

[0011] The sixth step is to implement role-based access control to ensure that different users can only access file data within their authority scope.

[0012] Preferably, the files with high real-time requirements are preferentially retained in the cache, and the retrieval speed is increased through a fast retrieval path.

[0013] Preferably, a visual monitoring interface is provided to monitor key indicators such as network status, cache utilization, retrieval efficiency, etc. in real time, so as to facilitate administrators to perform network optimization and troubleshooting.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] (1) The retrieval speed of file data has been improved, and the number of network requests and data transmission delays have been significantly reduced through distributed indexing and cache optimization technology.

[0016] (2) The reliability and scalability of the system are enhanced. The decentralized architecture reduces the dependence on a single node and improves the system's fault tolerance and scalability.

[0017] (III) Optimized cache utilization. The intelligent cache replacement strategy ensures that the most frequently used or important file data is stored in the cache, thus improving the utilization efficiency of cache resources.

[0018] (IV) Improved security: through digital signature and encryption technology, the integrity of file data and the security of transmission are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 This is an accompanying diagram of a decentralized file data retrieval cache acceleration technology of the present invention.

[0021] Figure 1In: 101, generate unique hash value; 102, build distributed index; 103, blockchain records cache status; 104, execute retrieval algorithm and dynamically adjust cache content; 105, digital signature and encryption; 106, role-based access control. DETAILED DESCRIPTION

[0022] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Example 1

[0024] according to Figure 1 As shown, the entire workflow of a decentralized file data retrieval cache acceleration technology of the present invention is shown in detail. The figure includes six main steps, which correspond to the key technical points of the present invention.

[0025] In the first step 101, the system receives the file data to be retrieved as input, and generates a unique hash value of fixed length by using a strong hash algorithm. This hash value is used as a unique identifier of the file for subsequent file retrieval and index construction.

[0026] In the second step 102, the generated hash value and its related metadata are encapsulated into a data packet and broadcasted in the network through the distributed hash table protocol. After receiving the data packet, multiple nodes in the DHT network store it in an appropriate location according to the hash value, and jointly build a distributed index system to achieve decentralized storage and fast retrieval of file data.

[0027] In the third step 103, the system uses the immutability and distributed characteristics of blockchain technology to record the cache status of each node. Whenever a node caches or updates a file, the cached file hash value, cache timestamp and other information are recorded on the blockchain. This step ensures the consistency and security of the cache status, facilitating subsequent efficient retrieval.

[0028] In the fourth step 104, the user initiates a file retrieval request through the system. The system quickly locates the file storage location in the distributed index based on the hash value in the request. At the same time, combined with the cache status information on the blockchain, the file is retrieved from the cache first. If the cache misses, it is obtained from other nodes through the DHT network. During the retrieval process, the system dynamically adjusts the cache content according to the file access frequency and cache strategy to improve the cache hit rate and retrieval efficiency.

[0029] Step 5 105, in order to ensure the integrity of the file data and the security of transmission, the system uses digital signature and encryption technology to protect the file. The file will be encrypted before transmission and attached with the sender's digital signature. The receiver can confirm the authenticity and integrity of the file only after verifying the validity of the signature and decrypting the file.

[0030] In step 106, the system implements role-based access control to ensure that users can only access file data within their authority. When a user logs in, the system verifies their identity and assigns corresponding roles and permissions. When a user initiates a file access request, the system checks their permissions and only allows access to files within the authorized scope. This step effectively prevents the risk of data leakage and unauthorized access.

[0031] Through Example 1, a decentralized file data retrieval cache acceleration technology can be implemented. This technology effectively solves the single point failure, low data retrieval efficiency and inconsistent cache data problems existing in traditional centralized file retrieval systems by constructing a distributed hash table DHT network and using blockchain technology to record the cache status; the technology is used to generate a unique hash value for file data in the DHT network and store it in multiple nodes in the network to achieve decentralized file indexing; the technology utilizes the immutability and decentralization characteristics of the blockchain to record and synchronize the cache status of each node to ensure the consistency and security of the cache data; the technology combines the file hash value and cache status information to design an efficient retrieval algorithm to reduce the number of network requests and improve the data retrieval speed. Obviously, the above embodiments are only examples for clearly illustrating the technical solution of the present invention, and are not limitations on the implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The present invention discloses a decentralized file data retrieval cache acceleration technology, which effectively solves the problems of single point failure, low data retrieval efficiency and inconsistent cache data in traditional centralized file retrieval systems by constructing a distributed hash table DHT network and using blockchain technology to record cache status; the technology is used to generate a unique hash value for file data in the DHT network and store it in multiple nodes in the network to achieve decentralized file indexing; the technology utilizes the immutability and decentralization characteristics of blockchain to record and synchronize the cache status of each node to ensure the consistency and security of cache data; the technology combines file hash values ​​and cache status information to design an efficient retrieval algorithm, reduce the number of network requests, and improve data retrieval speed.

2. A decentralized file data retrieval cache acceleration technology according to claim 1, characterized in that: The intelligent cache management module also includes a cache replacement strategy, which dynamically adjusts cache content and optimizes cache resource utilization based on file access frequency, cache space occupancy rate, and file importance factors.

3. A decentralized file data retrieval cache acceleration technology according to claim 1 or claim 2, characterized in that: The technology also includes a security verification mechanism that ensures the integrity of file data and security during transmission through digital signature and encryption technology.

4. A decentralized file data retrieval cache acceleration technology according to claim 1, characterized in that: The technical implementation includes the following steps: The first step is to generate a unique hash value using a hash algorithm for the file data to be retrieved; The second step is to store the file hash value and its metadata in multiple nodes of the DHT network to build a distributed index; The third step is to use blockchain technology to record the cache status of each node, including the cached file hash value and cache timestamp information; Step 4: executing an efficient retrieval algorithm based on user requests, combining distributed indexes and cache status information, and dynamically adjusting cache content during the retrieval process; Step 5: Using digital signature and encryption technology to ensure the integrity of file data and security during transmission; The sixth step is to implement role-based access control to ensure that different users can only access file data within their authority scope.

5. The decentralized file data retrieval cache acceleration technology according to claim 1 is characterized in that: The technology also includes a user authority management module for implementing role-based access control to ensure that different users can only access file data within their authority scope.

6. A decentralized file data retrieval cache acceleration technology according to claim 2, characterized in that: The cache replacement strategy also takes into account the real-time requirements of files. Files with high real-time requirements are preferentially retained in the cache.

7. A decentralized file data retrieval cache acceleration technology according to claim 3, characterized in that: The security verification mechanism also includes encrypting data during file transmission to ensure the security of the data transmission process.

8. A decentralized file data retrieval cache acceleration technology according to any one of claims, characterized in that: The technology also provides a visual monitoring interface for real-time monitoring of key indicators of network status, cache utilization, and retrieval efficiency, making it easier for administrators to perform network optimization and troubleshooting.

Images