Log data storing method and system

A log and data technology, applied in the computer field, can solve the problem of not being able to ensure that the log data is not lost, and achieve the effect of ensuring no loss and fast storage

Active Publication Date: 2017-01-25
RENMIN UNIVERSITY OF CHINA
View PDF7 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The traditional log processing technology cannot quickly realize the temporary storage of detailed log data in the log data, and cannot ensure that the log data enters the data warehouse quickly without loss

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log data storing method and system
  • Log data storing method and system
  • Log data storing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0051] Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the description of the present invention refers to the presence of said features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of computers, and discloses a log data storing method and system. The method comprises the steps that log data is divided into multiple log record fragments according to different entity clusters to which the log data belongs; all the log record fragments are written in different themes of a distributed message queue respectively; by adopting a multi-thread mode, the log record fragments stored in the different themes of the distributed message queue are parallelly loaded in a distributed file system. According to the log data storing method and system, not only are lossless temporary storage and rapid loading of the log data achieved, but also the condition that the log data is loaded in a data warehouse in a convenience query format can be guaranteed.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and system for storing log data. Background technique [0002] Log data contains valuable information. Timely and effective storage and analysis of log data can bring objective business value. For example, by analyzing server operation log data, we can analyze the cause of the failure. By analyzing the log data of e-commerce websites, we can understand the changes in users' recent browsing / purchasing behavior, and then make personalized recommendations for them. It can be seen that personalized analysis requires us to retain detailed log data, while real-time analysis requires us to load the data into the data warehouse as soon as possible. These are the two challenges of personalized real-time analysis, that is, the detailed data cannot be lost, and the data must be loaded as soon as possible. [0003] Traditional log processing technologies only focus on macro in...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F17/30
CPCG06F3/0604G06F3/064G06F3/067G06F16/1815G06F16/182
Inventor 陈跃国覃雄派杜小勇金国栋丛一鸣刘阳
Owner RENMIN UNIVERSITY OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap