Metadata encrypting-decrypting method and system based on distributed file system

Abstract

The invention discloses a metadata encrypting-decrypting method and system based on a distributed file system, and relates to the field of Hadoop security of big data. The method comprises the following steps that: an HDFS (Hadoop Distributed File System) client transmits a data writing request to a metadata node; the metadata node encrypts metadata according to the data writing request; the HDFSclient acquires encrypted metadata from the metadata node while reading the metadata, and transmits the encrypted metadata to a data node; and the data node decrypts a ciphertext of the metadata by using a public key issued by a key management server, acquires a plaintext of the metadata, and transmits file streams of the metadata to the HDFS client. Compared with big data encryption, the method and the system have the advantage that the metadata magnitude is relatively small. The metadata are encrypted and decrypted, so that the data transmission efficiency is increased, and the privacy protection of the data is further enhanced.

Description

technical field [0001] The invention relates to the Hadoop security field of big data, in particular to a metadata encryption and decryption method and system based on a distributed file system. Background technique [0002] Big data is playing an increasingly important role in the current social and economic development, but at the same time of centralized processing and mass storage of big data, security issues are also facing more and more serious problems. [0003] HDFS (Hadoop Distributed File System) is designed as a distributed file system suitable for running on commodity hardware. It has a lot in common with the existing distributed file system, but at the same time, its difference from other distributed file systems is also obvious. HDFS is a highly fault-tolerant system suitable for deployment on cheap machines; HDFS can provide high-throughput data access, which is very suitable for applications on large-scale data sets; HDFS relaxes the requirements of (relax) ...

AI Technical Summary

Problems solved by technology

Such as application-level encryption, which is the safest and most flexible method. Application-level encryption can accurately reflect the needs of users. However, this encryption method is difficult to write applications.

Database-level encryption has properties similar to application-level encryption, most database vendors provide some form of encryption, however this can have performance issues, e.g. indexes cannot be encrypted

Encryption at the file system level provides high-performance, application-transparent encryption that is generally easy to deploy, but it cannot simulate some policies at the application level, such as multi-tenant applications based on end-users may want to encrypt, a database may require different The encryption settings for each column are stored in a single file

[0004] In terms of big data protection, the conventional method is to ensure system security through traditional components and products such as firewalls, VPN intrusion detection, and antivirus. Sensitive information and sensitive data protection

Images