Distributed file system for consumer-grade devices and data security management method thereof

Abstract

The application discloses a kind of distributed file systems for consumer-grade equipment and its data security management method, belong to computer system technical field.System includes: virtual namespace management module, for maintaining by local main namespace tree and remote snapshot subtree merged into virtual namespace tree;Access control and encryption module, for executing access control based on the attribute set of virtual namespace tree node;Data processing pipeline module, for the pipeline operation of transmission and decryption by independent data stream and control stream channel to realize file block division.This application tightly couples access control and encryption strategy based on unified attribute set, without file re-encryption when permission changes, improves system flexibility;And hide security operation delay by data block division and pipeline processing, improve file access efficiency, effectively solve the balance problem between security, flexibility and efficiency of consumer equipment distributed file system.

Description

Technical Field

[0001] This invention relates to the field of computer system technology, and in particular to a distributed file system and its data security management method applicable to consumer devices such as smartphones, tablets, and laptops. Background Technology

[0002] With the widespread adoption of consumer electronics such as smartphones, tablets, and laptops, and the continuous evolution of wireless connectivity technologies like 5G, Wi-Fi 6, and near-field communication, users commonly own and use multiple smart terminal devices simultaneously. Against this backdrop, seamless file sharing and data collaboration across devices has become a rigid requirement. Distributed File System (DFS), as a technical architecture providing a unified file access interface across multiple computing nodes, has become a crucial technical path to achieving this goal due to its transparency to upper-layer applications. Traditional Network File System (NFS) and Hadoop Distributed File System (HDFS) have been thoroughly validated in enterprise-level data center environments, effectively managing relatively stable and homogeneous server clusters.

[0003] However, the dynamic, self-organizing networks of consumer devices differ fundamentally from data center environments, posing significant challenges to traditional DFS architectures. First, consumer device groups are highly dynamic and unstable. Devices can join or leave the network at any time, network topology changes frequently, and file owners have far more frequent and arbitrary needs to adjust access permissions than in data center environments. For example, a user might need to share a photo directory with a newly acquainted friend or immediately revoke document access rights for a group member. Existing security solutions fall short in addressing these needs: architectures relying on centralized authentication servers (such as Kerberos) are prone to single points of failure and suffer from significant delays in permission updates; while solutions based on complex proxy re-encryption or full file re-encryption are difficult to implement efficiently on resource-constrained mobile devices due to their enormous computational and communication overhead.

[0004] Secondly, the performance overhead introduced by security mechanisms is amplified dramatically in mobile scenarios. Traditional secure DFS typically employs a serial processing mode of "complete transmission first, then overall decryption" when handling remote file access. This leads to the computational latency of secure operations (especially asymmetric cryptographic operations) being compounded by network transmission and storage I / O latency, resulting in a significant increase in overall response time. Furthermore, traditional overall file encryption and transmission modes cannot fully utilize the multi-core parallel processing capabilities of modern mobile devices, nor do they effectively explore the pipelined parallel potential between network transmission and data decryption. This results in low utilization of CPU, I / O, and network resources, creating an inherent contradiction that "security inevitably sacrifices performance."

[0005] Therefore, there is an urgent need in this field for an innovative distributed file system solution that can fundamentally adapt to the dynamic characteristics and resource constraints of consumer devices, and achieve deep collaboration between security policies, namespace management and data path optimization from the initial design stage, so as to achieve the design goals of high-level security, dynamic flexibility and low operation latency in complex and ever-changing mobile environments. Summary of the Invention

[0006] The purpose of this invention is to provide a distributed file system and its data security management method to address the following contradictory problems existing in consumer device scenarios: First, the dynamic networking and frequent permission changes of consumer devices require the file system to have highly flexible access control capabilities; second, limited computing power and unstable network conditions require the file system to have a highly efficient security processing mechanism. However, existing distributed file systems struggle to balance security, flexibility, and performance, making it difficult to meet the secure file sharing needs of consumer device environments.

[0007] To achieve the above objectives, this invention proposes a distributed file system scheme that supports attribute-driven access control, has fine-grained data security protection capabilities, and can run efficiently on resource-constrained devices.

[0008] To achieve the above objectives, the present invention proposes the following technical solution: A distributed file system for consumer devices, the system being deployed in a peer-to-peer network of multiple consumer devices, wherein the device storing and sharing files is called the data owner device, and the device accessing remotely shared files is called the visitor device, and each device includes: The virtual namespace management module is used to maintain a virtual namespace tree for local users. The virtual namespace tree is formed by merging the local main namespace tree with snapshot namespace subtrees from one or more remote devices. The main namespace tree represents the metadata of files and directories owned by the local user. The snapshot namespace subtree is a copy of a portion of the directory hierarchy and file metadata filtered based on access permissions from the main namespace tree of the remote device. The access control and encryption module, coupled with the virtual namespace management module, is used to perform access control based on the attribute set set on the virtual namespace tree node, and to encrypt and decrypt file content using a hybrid encryption mechanism based on the same attribute set. The data processing pipeline module works in conjunction with the virtual namespace management module and the access control and encryption module to divide files into fixed-size data blocks for management and to establish independent control flow channels and data flow channels, so that the transmission and decryption operations of file data blocks can be performed in parallel. The communication daemon module includes a first daemon for transmitting control messages and a second daemon for transmitting file data blocks. The second daemon is configured to bind to a computing core dedicated to data processing and maintain a long-term TCP connection with the remote device to reduce latency caused by frequent connection establishment.

[0009] Furthermore, the attribute set, wherein the attributes are used to identify the characteristics of the device or user; and the attribute permissions on the node are combined through a logic gate structure, wherein an "AND" logic gate is set between the parent node and the child node, so that the access permissions of the child node are inherited from the parent node.

[0010] Furthermore, the hybrid encryption mechanism includes: encrypting the file content using a symmetric encryption algorithm, and encrypting the symmetric key used by the symmetric encryption algorithm using an attribute-based encryption algorithm.

[0011] Furthermore, the attribute-based encryption algorithm is an attribute-based encryption algorithm based on the ciphertext policy.

[0012] Furthermore, the symmetric encryption algorithm employs a no-padding mode, ensuring that the data block size remains unchanged before and after encryption.

[0013] Furthermore, the data processing pipeline module sets the data block size to be aligned with the maximum transmission unit of the underlying network protocol.

[0014] A data security management method based on the above-mentioned distributed file system includes: On the data owner's device, set a set of attributes for the file or directory to be shared, which defines the user or device authorized to access it; Based on the aforementioned attribute set, a hybrid encryption mechanism is used to encrypt the file, generating the file ciphertext and the corresponding encryption symmetric key. Based on the attribute set, a snapshot namespace subtree is generated from the main namespace tree of the data owner device and distributed to the authorized visitor devices; On the visitor's device, the received snapshot namespace subtree is merged with the local main namespace tree to form a local virtual namespace tree; When a visitor device requests access to a file located on the data owner device through its virtual namespace tree, the request is initiated through the control flow channel. The data owner device sends encrypted file data blocks to the visitor device via a data stream channel; After receiving the data block, the visitor device initiates the decryption process, realizing a pipelined operation of data transmission and decryption.

[0015] Furthermore, when the access permissions of the file or directory need to be changed, the data owner device updates the attribute settings of the target node and notifies the authorized visitor device to update its snapshot namespace subtree to achieve the permission change, without the need to re-encrypt the file content.

[0016] A consumer-grade device includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the functions of modules in the distributed file system or executes the data security management method.

[0017] Compared with the prior art, the present invention has the following significant advantages: 1) High flexibility: Access control and encryption policies are tightly coupled through an attribute-based virtual namespace tree. When permissions change, only attribute settings and snapshot subtrees need to be adjusted. There is no need for complex re-encryption operations. It can quickly adapt to the characteristics of dynamic networking of consumer devices and frequent changes in permissions.

[0018] 2) High efficiency: By segmenting files, separating data flow from control flow, and using pipelined processing, the time for I / O, network transmission, and decryption operations is effectively overlapped, hiding most of the delays caused by security mechanisms and significantly improving file access speed.

[0019] 3) Strong security: It adopts a hybrid encryption method that combines attribute-based encryption and symmetric encryption. While ensuring data confidentiality, it ensures that only users whose attributes meet the policy can decrypt files and effectively resist collusion attacks.

[0020] 4) Transparent and easy to use: The system is implemented as a kernel module, which is transparent to upper-layer applications. Users can enjoy secure and efficient cross-device file sharing services without modifying the application. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall architecture of the distributed file system of the present invention; Figure 2 This is a schematic diagram illustrating the composition and merging process of the virtual namespace tree in this invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention will be described in detail below with reference to specific examples. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0023] In a preferred embodiment of the invention, the distributed file system is deployed in a peer-to-peer network environment consisting of multiple consumer devices. In actual testing, the system employed a test platform consisting of five devices: two Google Pixel 6 Pro smartphones running Android 13.0, two Google Pixel 3 smartphones running Android 10.0, and a ThinkPad laptop running Ubuntu 16.04. These devices were interconnected via a dual-band Wi-Fi network with a bandwidth of 1200 Mbps.

[0024] See Figure 1 The system's core architecture is deployed in the device's kernel space, providing a unified file access interface to user space through designated mount points. The system comprises four main functional modules: a virtual namespace management module responsible for maintaining and merging the namespace tree structure; an access control and encryption module implementing attribute-based security mechanisms; a data processing pipeline module responsible for data segmentation and pipelined processing; and a communication daemon module containing dedicated control and data daemons, handling different types of communication tasks respectively.

[0025] In terms of virtual namespace management, each device maintains a complete main namespace tree, such as Figure 2 As shown in (a), all metadata information for storing local files is stored. This metadata includes file paths, standard file attributes, and extended attribute sets. When a user needs to share a file, the system will prune and generate a corresponding snapshot subtree from the main namespace tree according to the set attribute strategy, such as... Figure 2 As shown in (b). This snapshot subtree contains only the directory structure and file metadata that are authorized to be seen, and is pushed to the authorized user's device via the control daemon. After obtaining the snapshot subtree, the receiving device merges it with its local main namespace tree, constructing a unified virtual namespace tree through path matching and reference links, as shown in (b). Figure 2 As shown in (c). During the merging process, if files with the same name are encountered, the system will automatically add the owner's identifier as a suffix to resolve the naming conflict.

[0026] The access control and encryption mechanism employs an attribute-based hybrid encryption scheme. The system extends the standard file attribute mechanism by introducing multiple attributes, including user identity attributes, device characteristic attributes, and security policy attributes. During encryption, the system first generates a random symmetric key to encrypt the file data using AES-128, and then encrypts the symmetric key using an attribute-based encryption algorithm based on the ciphertext policy. The access policy set during encryption is defined through logical expressions; only users meeting these attribute conditions can obtain decryption privileges. The encrypted symmetric key is stored as an extended attribute in the file's metadata, forming a complete encrypted file along with the file content.

[0027] Data processing employs a block-based and pipelined approach. Files are divided into fixed-size data blocks, preferably 64KB, a size aligned with the network transmission mechanism to ensure the block size remains unchanged before and after encryption. The system establishes an independent dual-channel communication mechanism: the control channel handles metadata requests and authorization verification, while the data channel focuses on high-speed transmission of encrypted data blocks. This separation design allows data decryption to proceed in parallel with network transmission. Decryption can begin immediately upon the arrival of the first data block at the receiving end, while subsequent data blocks are still in transmission, thus achieving time overlap between network transmission and decryption operations.

[0028] The permission change handling employs a lightweight design. When file access permissions need to be modified, the data owner only needs to adjust the attribute settings of the corresponding node. The system then generates a corresponding incremental snapshot update and pushes it to the relevant devices via the control channel. Upon receiving the update, the receiving device adjusts its local virtual namespace tree, making the permission change take effect immediately. The advantage of this mechanism is that it eliminates the need to re-encrypt file content, significantly reducing the overhead of permission management.

[0029] In its implementation, the system also incorporates several performance optimization measures. For memory management, a compact data structure is used to store attribute information, keeping the average additional storage overhead per file below 1KB. For communication resource management, a dynamic resource allocation and release mechanism is implemented, automatically putting the data channel to sleep after file transfer is complete to reduce energy consumption. For network connection maintenance, a periodic heartbeat detection mechanism is used to ensure timely detection of device offline status and cleanup of related resources.

[0030] Through practical testing, the technical solution of this invention, with full security mechanisms enabled, reduces file access latency by an average of 69.5% and improves the efficiency of permission change operations by 82.3% compared to existing secure distributed file systems, while maintaining system security and flexibility. These improvements make this invention particularly suitable for deployment in resource-constrained consumer device environments with high user experience requirements.

[0031] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A distributed file system for consumer devices, characterized in that, The system is deployed in a peer-to-peer network consisting of multiple consumer-grade devices. The device that stores and shares files is called the data owner device, and the device that accesses remotely shared files is called the visitor device. Each device includes: The virtual namespace management module is used to maintain a virtual namespace tree for local users. The virtual namespace tree is formed by merging the local main namespace tree with snapshot namespace subtrees from one or more remote devices. The main namespace tree represents the metadata of files and directories owned by the local user. The snapshot namespace subtree is a copy of a portion of the directory hierarchy and file metadata filtered based on access permissions from the main namespace tree of the remote device. The access control and encryption module, coupled with the virtual namespace management module, is used to perform access control based on the attribute set set on the virtual namespace tree node, and to encrypt and decrypt file content using a hybrid encryption mechanism based on the same attribute set. The data processing pipeline module works in conjunction with the virtual namespace management module and the access control and encryption module to divide files into fixed-size data blocks for management and to establish independent control flow channels and data flow channels, so that the transmission and decryption operations of file data blocks can be performed in parallel. The communication daemon module includes a first daemon for transmitting control messages and a second daemon for transmitting file data blocks. The second daemon is configured to bind to a computing core dedicated to data processing and maintain a long-term transmission control protocol (TCP) connection with the remote device to reduce latency caused by frequent connection establishment.

2. The distributed file system according to claim 1, characterized in that, The attribute set, wherein the attributes are used to identify the characteristics of the device or user; and the attribute permissions on the node are combined through a logic gate structure, wherein an "AND" logic gate is set between the parent node and the child node, so that the access permissions of the child node are inherited from the parent node.

3. The distributed file system according to claim 1, characterized in that, The hybrid encryption mechanism includes: encrypting the file content using a symmetric encryption algorithm, and encrypting the symmetric key used by the symmetric encryption algorithm using an attribute-based encryption algorithm.

4. The distributed file system according to claim 3, characterized in that, The attribute-based encryption algorithm is an attribute-based encryption algorithm based on the ciphertext policy.

5. The distributed file system according to claim 3, characterized in that, The symmetric encryption algorithm uses a no-padding mode, which keeps the data block size unchanged before and after encryption.

6. The distributed file system according to claim 1, characterized in that, The data processing pipeline module sets the data block size to be aligned with the maximum transmission unit of the underlying network protocol.

7. A data security management method based on the distributed file system according to any one of claims 1 to 6, characterized in that, include: On the data owner's device, set a set of attributes for the file or directory to be shared, which defines the user or device authorized to access it; Based on the aforementioned attribute set, a hybrid encryption mechanism is used to encrypt the file, generating the file ciphertext and the corresponding encryption symmetric key. Based on the attribute set, a snapshot namespace subtree is generated from the main namespace tree of the data owner device and distributed to the authorized visitor devices; On the visitor's device, the received snapshot namespace subtree is merged with the local main namespace tree to form a local virtual namespace tree; When a visitor device requests access to a file located on the data owner device through its virtual namespace tree, the request is initiated through the control flow channel. The data owner device sends encrypted file data blocks to the visitor device via a data stream channel; After receiving the data block, the visitor device initiates the decryption process, realizing a pipelined operation of data transmission and decryption.

8. The data security management method according to claim 7, characterized in that, When the access permissions of a file or directory need to be changed, the data owner device updates the attribute settings of the target node and notifies the authorized visitor device to update its snapshot namespace subtree to achieve the permission change, without the need to re-encrypt the file content.

9. A consumer-grade device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the functions of a module in a distributed file system as described in any one of claims 1 to 6, or performs a data security management method as described in claim 7 or 8.

Images