Big data processing method

A technology of big data processing and processing method, applied in the field of big data processing, can solve the problems of high cost, data cannot be processed, long time, etc., and achieve the effect of lowering the entry threshold and improving resource utilization.

Inactive Publication Date: 2015-01-28
XIAN FUTURE INT INFORMATION CO LTD
View PDF3 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There are valuable patterns and information hidden in the data. Using traditional data processing methods, it will take a long time and a huge cost to mine the information in big data, and even some data cannot be processed.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data processing method
  • Big data processing method
  • Big data processing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0026] A kind of big data processing method of the present invention, such as figure 1 shown, including the following steps:

[0027] Step 1, create a Hadoop cluster on the OpenStack cloud platform to provide the basic environment for big data processing;

[0028] Among them, such as figure 2 As shown, step 1 is specifically implemented according to the following steps:

[0029] Step 1.1, the user applies for an OpenStack account, and uses the OpenStack account to log in to the OpenStack cloud platform;

[0030] Step 1.2, the user uploads the image to the OpenStack cloud platform and registers the image;

[0031] Step 1.3, users create networks and routes, node group modules and cluster templates;

[0032] Step 1.4, the user creates a Hadoop cluster by selecting the Plugin and Hadoop version, filling in the cluster name, selecting the c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a big data processing method which includes the following steps: building a Hadoop cluster on an Open Stack cloud platform to provide basic environment for big data processing; importing data into HDFS and Swift to build a data source; processing the data built in the step 2 by a user, displaying a processing result through the Web page or assigning the processing result to the output file under the specific route. By means of the big data processing method based on Open Stack and Hadoop, the server resource utilization rate is improved, and the big data access requirement is reduced.

Description

technical field [0001] The invention belongs to the technical field of big data processing, and relates to a big data processing method. Background technique [0002] With the increasing popularity of the network information age, mobile Internet, social network, and e-commerce have greatly expanded the boundaries and application fields of the Internet. We are now in a "big data" era of explosive data growth. , culture, people's lives and other aspects have a profound impact, the era of big data poses new challenges and opportunities for human data control ability. Big data has the characteristics of mass, diversity, high speed, variability, various data types, low relative density of data value, and high timeliness requirements, which have exceeded the processing capacity of traditional database processing systems. There are valuable patterns and information hidden in the data. Using traditional data processing methods, it takes a long time and a huge cost to mine the infor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08
CPCH04L67/02G06F16/25H04L67/1001
Inventor 王茜李安颖史晨昱葛新梁小江
Owner XIAN FUTURE INT INFORMATION CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap