Log processing method and device

A processing method and log technology, applied in the Internet field, can solve the problems of low data processing efficiency, inconsistent with the development trend of big data, time-consuming and labor-intensive, etc.

Active Publication Date: 2017-01-04
BEIJING QIHOO TECH CO LTD +1
View PDF6 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the prior art, when a user wants to process the data output by a certain data source, the data processing program needs to be manually written according to the corresponding processing requirements. Different data processing requirements nee

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log processing method and device
  • Log processing method and device
  • Log processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0080] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0081] figure 1 A flowchart of a log processing method according to an embodiment of the present invention is shown. Such as figure 1 As shown, the method includes:

[0082] Step S110, receiving a piece of data stream from the input source every preset time period, each piece of data stream including multiple logs generated in the previous preset time period.

[0083] Step S120, after receiving each segment of the data stream, p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a log processing method and device; the method comprises: receiving from an input source, a data stream in certain time intervals, wherein each data stream includes multiple logs generated in a previous preset period of time; after receiving the data streams, initially analyzing each log in each data stream to obtain specified structural data corresponding to each log; initially aggregating the multiple specified structural data corresponding to the multiple logs in the data stream, and caching the initially-aggregated specified structural data; when computing time for a preset statistical period is up, computing the cached specified structural data to obtain log processing results corresponding to the statistical period. There is no need for using storage media in this process, there is no delay occurring due to enabling a process to read and write the storage media, all initial analysis, initial aggregating and computing for the logs are finished at a time, the efficiency is high, running is stable, intermediate data is rarely lost, and timeliness of log processing can be guaranteed as far as possible.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a log processing method and device. Background technique [0002] With the continuous development of Internet technology, the trend of Internet big data is becoming more and more significant. Every Internet business line is constantly generating new data. It is very important to further process the generated data to give feedback on the operation of Internet business one of the jobs. In the prior art, when a user wants to process the data output by a certain data source, he needs to manually write a data processing program according to the corresponding processing requirements. Different data processing requirements require rewriting different programs, and different users need to write their own The required procedures are very time-consuming and labor-intensive, which makes data processing inefficient and does not conform to the development trend of big data. Contents o...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/1815
Inventor 徐胜国王义辉王素梅沈迪李铮
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap