Preprocessing method, device and system for website access logs

A preprocessing device and website access technology, which is applied in the field of data processing, can solve the problems of long time-consuming log file preprocessing, low efficiency, and slow log processing speed, and achieve the effects of reducing processing time, improving processing efficiency, and improving efficiency

Active Publication Date: 2014-02-19
BEIJING GRIDSUM TECH CO LTD
View PDF6 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Aiming at the problem in the related art that the preprocessing of the website access log files takes a long time due to multiple read and write operations, resulting in slow log processing and low efficien

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Preprocessing method, device and system for website access logs
  • Preprocessing method, device and system for website access logs
  • Preprocessing method, device and system for website access logs

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and examples.

[0029] figure 1 is a schematic structural diagram of a system for preprocessing website access logs according to an embodiment of the present invention. Such as figure 1 As shown, the system may include: multiple cluster servers 2 and a log preprocessing server 1 .

[0030] Wherein, a plurality of cluster servers 2 are used to record original logs.

[0031] The log preprocessing server 1 is connected to multiple cluster servers 2, and is used to read original logs from the cluster servers, and after merging and sorting the original logs to obtain intermediate log streams, split the intermediate log streams to obtain preprocessed logs.

[0032] Using the above system, after the log preproces...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a preprocessing method, device and system for website access logs. The method comprises the steps that original logs are read from a cluster server; the original logs are combined and ordered to obtain a middle log stream; the middle log stream is segmented to obtain a preprocessing log. By the adoption of the preprocessing method, device and system, the problems that due to repeated read-write operation in the prior art, preprocessing for the website access logs consumes a long time, so the speed and efficiency of log processing are low are resolved, preprocessing for log data is completed by means of single-time read-write, processing time and the number of processed intermediate files are reduced, and therefore log processing efficiency is improved.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a method, device and system for preprocessing website access logs. Background technique [0002] With the development of the Internet, the number of Internet users continues to increase, and the number of website visits continues to rise. A single server can no longer satisfy a large number of website visits. A common method is to use a load balancing cluster, through one or more front-end load balancers , distribute the workload to a group of back-end servers, and the back-end servers receive requests and record logs. As the number of visits continues to increase, the size of the log file continues to expand, but the processing time requirement for the corresponding log file does not decrease. Therefore, how to improve the processing efficiency of log files has become a problem that must be faced in this field. [0003] The earliest log processing method is to directly r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/24H04L29/08
Inventor 何恺铎饶峰云
Owner BEIJING GRIDSUM TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap