Combined type data cutting method based on regular expression and separators

Abstract

The invention discloses a combined type data cutting method based on regular expression and separators. The method comprises the following steps: a) reading a log file; b) recognizing and extracting an event in the log file according to a timestamp or a start symbol; c) extracting data of the extracted event content according to the fixed separator or regular expression way; selecting a plurality of preset character strings by the step b) as the start symbol, and converting the plurality of preset character strings into a plurality of regular expressions, traversing each line of log data, orderly matching with each regular expression. Through the adoption of the combined type data cutting method based on regular expression and separators provided by the invention, the different cutting ways are used according to different event types, each cutting way is corresponding to a log within the range, and the character string operation is provided through each way, thereby satisfying the fast recognition cutting of various complex logs, the method is simple and easy to use, and good in expansibility.

Description

technical field [0001] The invention relates to a data cutting and extraction method, in particular to a combined data cutting method based on regular expressions and delimiters. Background technique [0002] The existing data cutting and extraction methods mainly include the following three methods: [0003] 1. Use the method of writing your own program to cut and extract data, write the corresponding data cutting program according to different data formats, and control the data cutting rules, extraction and output in the program. Programs can be written in different programming languages ​​for different platforms. The flexibility of writing programs for data cutting is very high, which can basically meet all data cutting needs. [0004] 2. Use fixed separators to cut and extract data. This solution is very suitable for data comparison and formatting. Usually, data is formatted according to a certain fixed separator. You only need to follow this separator. Segmentation, ...

AI Technical Summary

Problems solved by technology

And the efficiency is high, currently the data formatted according to the fixed delimiter can be extracted

[0005] 3. Use regular expressions to extract data. At present, the types and formats of data sources are very diverse. Often the output of data does not use a fixed format. The cutting scheme of fixed separators often cannot meet this, so it is necessary to have Only solutions with very high cutting flexibility can carry out data cutting

[0006] The programming code matching of method 1 has the following disadvantages: for each type of data, it is necessary to write and test the program once, which takes a lot of time

The commonly used separator matching in method 2 has the following disadvantages: it can only cut formatted data; it cannot be accurately extracted; the recognition content is easily misplaced and the flexibility is low

The regular expression matching of method 2 has the following disadvantages: it needs to have a higher technical level; the regular expression is very complicated to write, and it is easy to misuse; it is difficult to maintain and modify; the scalability is poor; Not accurate enough; cannot be accurately extracted; identification content is easily misplaced

Images