Method and device for reading and writing log files
A technology of log files and reading and writing methods, which is applied in the field of data processing, can solve the problems of long time and low efficiency of importing into the database, and achieve the effects of shortening processing time, improving performance and long solving time
Active Publication Date: 2017-10-27
BEIJING GRIDSUM TECH CO LTD
View PDF6 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] In view of the long time and low efficiency of importing log files into the database in the related art, no effective solution has been proposed at present. Therefore, the main purpose of the present invention is to provide a method and device for reading and writing log files to solve the above problems. question
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0020] First of all, some nouns or terms appearing in the process of describing the embodiments of the present invention are applicable to the following explanations:
[0021] Rollback (Rollback): Rollback generally refers to the behavior of returning to the last correct state after the program update fails.
[0022] Serialize: The process of converting the state of an object into a format that can be persisted or transmitted.
[0023] Deserialize (Deserialize): The process of converting data streams into objects.
[0024] Regular Expression (Regular Expression): A regular expression uses a single string to describe and match a series of strings that meet a certain syntax rule. In many text editors, regular expressions are usually used to retrieve and replace those that meet The text of a pattern.
[0025] JSON (JavaScript Object Notation): A lightweight data exchange format.
[0026] Primary Key (Primary Key): It is one or more fields in the table, and its value is used to...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a method and device for reading and writing a log file. The method comprises the steps of reading, wherein read log data in the log file are stored a corresponding internal storage set; detection, wherein whether the number of lines of the internal storage set is larger than a preset threshold value or not is detected; insertion, wherein the log data in the internal storage set are inserted into a first temporary table in a database under the condition that the number of the lines of the internal storage set is larger than the preset threshold value; return, wherein the step of reading is executed again until the log data are all stored in the first temporary table; updating, wherein data in the first temporary table are used for updating a formal table of the database. The method and device solve the problems that in the prior art, the time of importing the log file into the database is long and efficiency is low, the effect of carrying out reading and writing simultaneously is achieved, the parallel processing mechanism is used, the overall processing time of a program is shortened, and performance is improved.
Description
technical field [0001] The invention relates to the field of data processing, in particular to a method and device for reading and writing log files. Background technique [0002] The usual way of importing log files into the database in the prior art is to read the log files in the specified format line by line through the program and write them into the memory line by line. After the log files are processed, the memory is written into the database at one time, that is, After reading the file, it will be written into the database. The execution time of the program using this scheme = the time of reading the file + the time of writing to the database. In the case of a large amount of data in the log file, the time to read the file and the execution time to write to the database are greatly increased, resulting in a significant increase in the overall execution time of the program. Using the above method, because the execution process is serial and linear, during the process...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
CPCG06F16/13G06F16/1815G06F16/1858
Inventor 吴充郭涛
Owner BEIJING GRIDSUM TECH CO LTD