Big data storage and search method and apparatus

A technology of a storage device and a data storage system, applied in the field of search method and device, and the storage of big data, can solve the problems of multi-time and system resources, waste, long time consumption, etc., to reduce redundancy, improve utilization, reduce wasteful effect

Inactive Publication Date: 2016-09-28
BEIJING QIHOO TECH CO LTD +1
View PDF7 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Considering that the search operation at the big data level has a high order of magnitude, it will inevitably waste more time and system resources
Moreover, it takes too long, which is als

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data storage and search method and apparatus
  • Big data storage and search method and apparatus
  • Big data storage and search method and apparatus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0060] In order to solve the above technical problems, an embodiment of the present invention provides a method for storing big data. figure 1 A processing flowchart of a method for storing big data according to an embodiment of the present invention is shown. see figure 1 , the storage methods of big data include at least:

[0061] Step S102, obtaining the original log of the big data and analyzing its specific log content;

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a big data storage and search method and apparatus. The storage method comprises the steps of obtaining original logs of big data and analyzing specific log content of the original logs; classifying the original logs according to the specific log content, generating a document by a specified number of original log sets, and establishing a mapping relationship between the document and a segmented word, wherein the segmented word is matched with the specific log content of the document; combining documents to generate a combined file, wherein a plurality of segmented words corresponding to the number of combined documents can be provided when the file is searched; and storing the file, instead of the original log, in a distributed storage system framework. With the adoption of the method and apparatus, the data redundancy can be greatly reduced, so that the waste of server resources is reduced and the utilization rate of storage resources is increased.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to methods and devices for storing and searching big data. Background technique [0002] With the development of computers and networks, the search function has become the most commonly used function, and users can obtain target information conveniently and quickly through search operations. However, with the development of business, the amount of data that can be searched is also increasing. At present, the increasing amount of data is called big data, and the search operation on it is called searching for big data. [0003] The data volume of big data is usually more than one million, or even more than one trillion or one trillion. For such a huge data system, first of all, higher requirements are put on data storage. For example, in the currently commonly used ES system (ElectedSearch), in this system, index data and original data required for search and matching operations mu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/13G06F16/148G06F16/1744G06F16/1815G06F16/182
Inventor 魏自立李浩穆玉伟赵晶晶蒋东冯鑫
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap