Supercharge Your Innovation With Domain-Expert AI Agents!

A detection method based on security configuration of hadoop

A technology of security configuration and detection method, applied in the field of big data, can solve the problem that the update configuration cannot be used directly, and achieve the effect of avoiding loopholes

Active Publication Date: 2019-02-15
STATE GRID FUJIAN ELECTRIC POWER CO LTD +1
View PDF12 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, this version is applicable in the production environment and brings security vulnerabilities, because the user password is stored in the property file as plain text, and each update configuration of the system cannot be used directly, and the security configuration needs to be checked

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] In the following, the technical solution of the present invention will be described in detail.

[0024] The invention provides a detection method based on hadoop security configuration, including the detection of Hadoop output terminal, the detection of distributed NoSQL database and the detection of network management server; wherein,

[0025] The detection of the Hadoop output terminal is carried out in sequence according to the following steps:

[0026] Step 1.1: Set up a secure Flume proxy channel for the HDFS and HBase output terminals, set the Flume logo in the KDC and generate a keytab file;

[0027] Step 1.2: To create a Flume user, you need to be a Hadoop super user, and create a user group for Flume simulation users, allowing different sources to be simulated, injecting data into Hadoop, and ensuring that user permissions in Hadoop are always followed during data ingestion;

[0028] Step 1.3: Use the security configuration program to set up, and perform a sec...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention relates to a detection method based on security configuration of hadoop. The method Includes detecting the Hadoop output, the detection of distributed NoSQL database and networkmanagement server, where the detection of the Hadoop output, Setting up a secure Flume agent for the HDFS and HBase outputs, setting the Flume ID in the KDC and generate the keytab file, the Flume user needs to be a Hadoop superuser, Creating user groups for Flume to emulate users, allowing different sources to be emulated, inject data into Hadoop, ensuring that user privileges in Hadoop are observed throughout the data ingestion process, setting up by the security configuration program, and performing security checks on Flume emulation users. The invention detects the safety configuration ofthe Hadoop output terminal, the distributed NoSQL database and the network management server by simulating the user terminal and the data transmission process, thereby avoiding the leak caused by theupgrading and updating of the system in advance.

Description

technical field [0001] The invention belongs to the field of big data, and in particular relates to a detection method based on hadoop security configuration. Background technique [0002] Hadoop consists of multiple components, and Sqoop is used to implement structured data transmission and conversion between RDBMS, NoSQL and Hadoop. The original version of Sqoop, Sqoop1, was used as a command-line client tool to generate MapReduce code based on metadata of structured data sources. Connection parameters are set through the command line, including password information required to submit MapReduce jobs to Hadoop, etc. In this version, there is no secure way to include user passwords, all are transmitted in clear text. Therefore, the application of this version in the production environment brings security vulnerabilities, because the user password is stored in the property file as plain text, and each update configuration of the system cannot be used directly, and the secur...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F21/57G06F21/60
CPCG06F21/577G06F21/602
Inventor 何金栋唐志军林承华赵志超吴丹吴丽进谢新志罗富财
Owner STATE GRID FUJIAN ELECTRIC POWER CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com