Web access log processing method based on storm

A processing method and log technology, applied in the field of data processing, can solve the problems of time-consuming and laborious, inconvenient retrieval, etc., and achieve the effect of reducing the difficulty of retrieval and facilitating viewing

Inactive Publication Date: 2019-01-25
SICHUAN CHANGHONG ELECTRIC CO LTD
View PDF8 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Once a security incident or business failure occurs, troubleshooting is like finding a needle in a haystack, which is time-consuming and laborious. Troubleshooting and retrieval of server problems are very inconvenient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Web access log processing method based on storm
  • Web access log processing method based on storm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other implementations obtained by persons of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

[0048] Such as figure 1 As shown, a storm-based Web access log processing method includes the following steps:

[0049] Step 1. Log collection:

[0050] Use the Logstash log collection and processing framework to collect logs;

[0051] Log collection sources mainly include web server middleware logs, such as access logs and error logs generated by Nginx, Tomcat, and Apache. Supports the adaptation of web access logs.

[0052] The log collection method adopt...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a Web access log processing method based on storm, comprising four steps of log collection, log reception, log processing and log storage. The purpose is to realize the collection and standardization and enrichment of access logs, and store the enriched logs and statistical data into a database for convenient management personnel to view. The difficulty of searching for server problems is greatly reduced, which is convenient for operation and maintenance personnel to analyze, troubleshoot and trace the corresponding faults, incidents and accidents.

Description

technical field [0001] The invention belongs to the technical field of data processing, and in particular relates to a storm-based Web access log processing method. Background technique [0002] With the gradual expansion of the number of web servers and business volume, the amount of access log data of nginx, tomcat and other servers is increasing day by day, and the log management modules are relatively independent, and there are heterogeneous log formats. Once a security incident or business failure occurs, troubleshooting is like finding a needle in a haystack, which is time-consuming and laborious. Troubleshooting and retrieval of server problems are very inconvenient. In order to solve the problem of massive log data analysis and troubleshooting, and solve specific business and system failures, it is necessary to classify and store the web access log data of related businesses, so that operation and maintenance personnel and security operation and maintenance personnel...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/24H04L12/26H04L29/08
CPCH04L41/069H04L43/067H04L67/02H04L67/565
Inventor 张侃
Owner SICHUAN CHANGHONG ELECTRIC CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap