Hadoop cluster big data access method and system based on blockchain

A Hadoop cluster and access method technology, applied in the field of blockchain data access, can solve the problems of data loss privacy, leakage, lack of reliable authority management mechanism, etc., to achieve the effect of collection and quick response

Active Publication Date: 2020-06-30
SHANDONG INST OF BUSINESS & TECH
View PDF5 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] As more and more users and applications access the Hadoop cluster, the security of the cluster becomes extremely important, but at present Hadoop needs to be improved in terms of security
Common security problems of Hadoop include: complex user and authority management, lack of reliable authority management mechanism, and server single point of failure, etc.
The inventor found that although Hadoop generally uses the Kerberos identity authentication mechanism to authenticate users accessing Hadoop, after the user passes the identity authentication, it is no longer supervised, which leads to the inability to control illegal operations of legitimate users , once the malicious user passes the verification, he can use identity disguise to read the data in the cluster arbitrarily, which will lead to the loss of data and the leakage of privacy

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hadoop cluster big data access method and system based on blockchain
  • Hadoop cluster big data access method and system based on blockchain
  • Hadoop cluster big data access method and system based on blockchain

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0038] Such as figure 1 As shown, the present embodiment provides a blockchain-based Hadoop cluster big data access method, which includes:

[0039] (1) Receive the authentication request sent by the client, and generate a user label after the authentication is successful.

[0040] In a specific implementation, after receiving the authentication request sent by the client, it also includes:

[0041] Obtain the current user's secret key from the database, verify the identity of the current user, and if the verification is passed, feed back the user ticket and session key, and then generate a user label.

[0042] As an implementation, use the Kerberos authentication mechanism to solve the authentication from the client to the server, and generate the key Keytab for mutual communication between the client and the server on its key distribution center. Through these keys, nodes can House type certification and provide corresponding services. Using Kerberos authentication as the...

Embodiment 2

[0115] The present embodiment provides a blockchain-based Hadoop cluster big data access system, which includes:

[0116] An authentication module, which is used to receive an authentication request sent by the client, and generate a user label after the authentication is successful;

[0117] Role assignment module, which is used to assign roles to users through smart contracts in the blockchain according to the access purpose in the user label;

[0118] The access behavior recording module is used to use the distributed Namenode server as the accounting node of the blockchain, and receive the status information of the Datanode server itself and the user's access behavior periodically fed back by the Datanode server through the heartbeat mechanism;

[0119] The access control module is used to calculate the user's access risk value based on the user's access behavior and based on information entropy, and then compare it with the set access risk threshold to determine whether t...

Embodiment 3

[0124] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in the block chain-based Hadoop cluster big data access method described in Embodiment 1 are implemented.

[0125] This embodiment uses blockchain technology to deploy distributed Namenode server clusters to achieve joint maintenance of metadata and allocation of user access tasks, breaking the data "island"; receiving the status information and status information of the Datanode server itself that the Datanode server periodically feeds back through the heartbeat mechanism The user's access behavior realizes the collection of user behavior, and can respond more quickly to the failure of the Datanode node; write the user role assignment task into the smart contract, and the smart contract assigns different roles according to the different access purposes of the user. Roles have different permissions, and the roles are...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of big data access, and provides a Hadoop cluster big data access method and system based on blockchain. The method comprises the steps: receiving an authenticationrequest sent by a client, and generating a user label after authentication succeeds; according to the access purpose in the user label, allocating a role to the user through an intelligent contract inthe blockchain; taking the distributed Namenode server as an accounting node of the blockchain, and receiving the state information of the Datanode server and the access behavior of the user, which are periodically fed back by the Datanode server through a heartbeat mechanism, wherein the access behaviors of the users in the accounting nodes are kept consistent through a consensus mechanism; calculating an access risk value of the user according to the access behavior of the user and based on the information entropy, comparing the access risk value with a set access risk threshold, judging whether the user role is corrected or not, and accessing the data in the Datanode server according to the corrected user role, thereby realizing management of the user access control authority.

Description

technical field [0001] The invention belongs to the field of blockchain data access, and in particular relates to a blockchain-based Hadoop cluster big data access method and system. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] With the rapid development of computer technology and the Internet, data information technology is also becoming more and more popular. A large amount of data is generated in the fields of network, mobile smart terminal, Internet of Things terminal and medical scientific research every moment. The concept of big data is born from this. . Big data has five characteristics: large volume, high speed, variety, low value density, and authenticity, and its scale far exceeds the storage and processing capabilities of a single computer. In order to solve the storage and analysis problems of massive data, Apache develop...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F21/62G06F21/64G06F21/31G06F16/27
CPCG06F21/6218G06F21/64G06F21/31G06F16/27Y02D10/00
Inventor 管绍朋张聪辉李奕孙文文
Owner SHANDONG INST OF BUSINESS & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap