Data search method in big data environment

Abstract

The invention provides a data search method in a big data environment. The method comprises the steps that a client sends a search request to a big data center; the big data center performs security authentication on the received search request; if the authentication is passed, the search request is subjected to type judgment and then is further input to a search recording apparatus; if the authentication is not passed, the client sends a search response containing refusal information; and the big data center performs log recording on the current search and updates the search recording apparatus. According to the method, a quick search can be carried out by utilizing records, so that the search time is shortened, the situation that the working load and the energy consumption of the big data center are increased because same search requests unnecessarily occupy resources of the big data center is reduced, the data scheduling efficiency and speed of the big data center are improved and increased, the security of the whole big data center is prevented from being threatened, and the accuracy and comprehensiveness are enhanced.

Description

technical field [0001] The invention relates to the field of electrical digital data processing, and more specifically, to a data search method in a big data environment. Background technique [0002] With the continuous improvement of social industrialization and informatization, data has replaced computing as the center of information computing, and cloud computing and big data are becoming a trend and trend. Including storage capacity, availability, I / O performance, data security, scalability and many other aspects. Big data is very large and complex data sets. Big data has 4V: Volume (a large amount), the amount of data continues to increase rapidly; Velocity (high speed), data I / O speed is faster; Variety (variety), data types and sources are diversified; Value (value), it exists in all aspects available value. In addition, in addition to representing massive amounts of information, big data is also a source for people to gain new cognition and create new value. It h...

AI Technical Summary

Problems solved by technology

However, it can be said that there are pros and cons

While big data brings convenience, more and more human activities will generate PB-level data every month. At the same time, more and more demands need to process tens of PB, hundreds of PB or even EB-level data every day. The amount of data is extremely huge, and the amount of information that can be searched through search engines is increasing, resulting in lower search efficiency, greatly affecting search efficiency, and increasing the workload and energy consumption of the big data center.

[0004] Although there are a variety of data search methods, most of them are aimed at a single database or a limited number of data sources, and seldom involve the demand for big data search

Images