Unlock instant, AI-driven research and patent intelligence for your innovation.

Log processing method and device, storage medium and processor

A processing method and log technology, applied in the field of data processing, can solve the problems of difficult log processing, no solution proposed, complex log processing, etc.

Active Publication Date: 2020-09-18
SUZHOU DEEPLEAPER INFORMATION & TECH CO LTD
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, the processing of logs is very complicated. The standard log processing methods provided by streaming systems such as the existing memory distributed computing framework (Spark) and streaming computing framework (Storm) can only filter and convert logs. The processing of logs is relatively limited, and it cannot adapt to the coexistence of multiple downstream businesses with different businesses, making it difficult to effectively process logs
[0004] Aiming at the problem that it is difficult to effectively process logs in the above-mentioned prior art, no effective solution has been proposed yet

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log processing method and device, storage medium and processor
  • Log processing method and device, storage medium and processor
  • Log processing method and device, storage medium and processor

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0032] According to an embodiment of the present invention, an embodiment of a log processing method is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and, although A logical order is shown in the flowcharts, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0033] figure 1 is a flowchart of a log processing method according to an embodiment of the present invention. Such as figure 1 As shown, the method may include the following steps:

[0034] Step S102, acquiring logs to be processed.

[0035] In the technical solution provided in step S102 of the present invention, the log to be processed is an input log entered at the log entry. Optionally, the log to be processed in this embodiment is first input to the verifier, and the log to be processed is verified by th...

Embodiment 2

[0083] The technical solutions of the embodiments of the present invention are illustrated below in combination with preferred implementation modes.

[0084] Short video app logs record various user behaviors, such as logging in and out of the app, browsing and watching short videos, searching for short videos, etc. Properly defining, collecting, and processing these logs is of great significance for continuously optimizing the App.

[0085] First, the definition of a log should be comprehensive and concise. Among them, comprehensive means that user behavior needs to be recorded as much as possible, and concise means that in order to reduce the network overhead of transmission, the log should only record core information and discard redundant information. The log should contain the following elements: App version number, timestamp, user action, user ID, object type, object ID, and page where the action occurred. Redundant information, such as user and object details, should ...

Embodiment 3

[0153] The embodiment of the present invention also provides a log processing device. It should be noted that the log processing apparatus in this embodiment can be used to execute the log processing method in the embodiment of the present invention.

[0154] Figure 4 is a schematic diagram of a log processing device according to an embodiment of the present invention. Such as Figure 4 As shown, the log processing apparatus 40 may include: an acquisition unit 41 , a replication unit 42 , a processing unit 43 and a sending unit 44 .

[0155] The acquiring unit 41 is configured to acquire logs to be processed.

[0156] The copying unit 42 is configured to copy the log to be processed to obtain multiple copies of the log to be processed.

[0157] The processing unit 43 is configured to process multiple logs to be processed in parallel in the data flow and at least one business flow respectively.

[0158] The sending unit 44 is configured to send the first processing result...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a log processing method and device, a storage medium and a processor. The method comprises the steps of obtaining a to-be-processed log; copying the to-be-processed log to obtain a plurality of to-be-processed logs; respectively carrying out parallel processing on the plurality of logs to be processed in the data flow and the at least one service flow; and sending the firstprocessing result obtained in the data flow to a first data processing system, and sending the second processing result obtained in each service flow to a second data processing system. The log processing method and device solve the technical problem that logs are difficult to process effectively.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a log processing method, device, storage medium and processor. Background technique [0002] At present, the definition, collection, and processing of logs are of great significance for continuous optimization of a client (Application, App for short). [0003] However, the processing of logs is very complicated. The standard log processing methods provided by streaming systems such as the existing memory distributed computing framework (Spark) and streaming computing framework (Storm) can only filter and convert logs. This makes the processing of logs relatively limited, and cannot adapt to the coexistence of multiple downstream businesses with different services, making it difficult to effectively process logs. [0004] Aiming at the problem that it is difficult to effectively process logs in the above-mentioned prior art, no effective solution has been proposed yet. Co...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/17G06F16/18G06F16/182
CPCG06F16/1734G06F16/1815G06F16/182
Inventor 黄宇王风雷李东军
Owner SUZHOU DEEPLEAPER INFORMATION & TECH CO LTD