A batch processing method for large computer data

Abstract

The invention belongs to the technical field of big data batch processing and discloses a batch processing method for large computer data, which inputs customer data through a data input module by using a data input device. The main control module utilizes the scheduling algorithm to schedule the data resources to be processed through the resource scheduling module, and the resource scheduling module adopts the Min-Min scheduling algorithm, which utilizes batch processing program to schedule processor to batch process the job to be processed by batch processing execution module; Encryption module is used to encrypt the large data by the encryption program. The analysis module analyzes the big data by using the analysis program, and the data storage module stores the big data resources by using the memory. large data information content is displayed by a display module using that display. The invention does not need to obtain large data from a large amount of large data in a distributeddatabase, so the invention is short in time and easy to realize.

Description

technical field [0001] The invention belongs to the technical field of big data batch processing, in particular to a batch processing method of computer big data. Background technique [0002] Big data includes structured, semi-structured and unstructured data, and unstructured data is increasingly becoming the main part of data. According to the survey report of IDC, 80% of the data in enterprises are unstructured data, and these data are growing exponentially by 60% every year. Big data is just a manifestation or characteristic of the Internet at the present stage. There is no need to myth it or maintain awe of it. Against the background of technological innovation represented by cloud computing, these things seem difficult to collect The data used and used are beginning to be easily utilized. Through continuous innovation in all walks of life, big data will gradually create more value for human beings. The generation of big data analysis is aimed at IT management. Enter...

AI Technical Summary

Problems solved by technology

[0004] (1) Traditional big data security protection technologies cannot protect sensitive information and sensitive data within the big data platform, which may easily cause data leakage and cause losses to users

[0005] (2) When analyzing big data, the analysis time is long, the work efficiency is low, and it is not easy to implement, and batch analysis errors are prone to occur

[0006] (3) The traditional resource scheduling module has a low scheduling rate for massive data resources, resulting in a slow batch processing rate and wasting more time

[0007] When analyzing big data, the existing technology does not use the granular computing method to analyze the inaccurate solution of big data problems, and convert the input of the problem from the most fine-grained raw data to information granular representation, under the premise of retaining the information and value contained in the data , greatly reducing the amount of data

Images