System log processing method and device, computer equipment and storage medium
A technology of system logs and processing methods, which is applied in the field of big data processing, can solve problems affecting information query and display, and achieve the effect of improving efficiency and reducing one-time serialization and deserialization operations
Pending Publication Date: 2021-11-26
太平金融科技服务(上海)有限公司
View PDF0 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] However, the log collection process of LogStash is to collect log information line by line. When Logstash parses the log from the text log file, if it is an error log, it cannot be parsed and summarized into a complete log information, which will affect the search process. This part of the information query display
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0053] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.
[0054] The system log processing method provided by this application can be applied to such as figure 1shown in the application environment. Among them, the log platform 102 can communicate with the server 104 corresponding to each microservice and the user terminal 106 through the network, and the log platform can be integrated with a log processing framework of the improved ELK model, and the log processing framework of the improved ELK model It can include the log collection part logback for collecting logs, wherein the log collection part logback can obtain the logs o...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a system log processing method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a log output by a system, and creating a corresponding event object; judging whether an abnormal error report field exists in the event object; when the abnormal error reporting field exists in the event object, obtaining log information corresponding to the event object, and performing typesetting processing on the log information; performing structured processing on the log information obtained after typesetting to obtain a structured object, the log information obtained after typesetting comprises at least one line, and the structured object comprises the log information; storing the structured object after being subjected to serialization operation, obtaining the stored structured object subjected to serialization operation, and deserializing the serialized structured object into a structured object; receiving a log query request; querying the stored structured object according to the log query request, and outputting the queried structured object. By adopting the method, the log information can be accurately queried, and content omission is prevented.
Description
technical field [0001] The present application relates to the technical field of big data processing, in particular to a system log processing method, device, computer equipment and storage medium. Background technique [0002] Logs are an important source of information for software systems to find software abnormalities and errors. In the microservice architecture, a business request will pass through multiple services, and the link is long. Collecting logs on the end-to-end link can help developers determine the specific location of the exception and error. In the daily operation and maintenance of the system, statistical analysis of the system operation can be performed to provide effective data analysis for system optimization. [0003] At present, the mainstream of microservice application log processing uses ELK mode (ElasticSearch+LogStash+Kibana) to collect, analyze, store and display logs. Among them, LogStash, as the aggregation of logs, can collect logs through ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/18G06F16/172G06F16/25G06F16/2455
CPCG06F16/1815G06F16/172G06F16/254G06F16/2455
Inventor 郑烈彬赖欣联
Owner 太平金融科技服务(上海)有限公司