Log analysis method, system, apparatus, device, and storage medium

Abstract

The application discloses a log analysis method, system, device, equipment and storage medium, relates to the technical field of data processing, and the log analysis method comprises the following steps: receiving a file path sent by a front end, directly reading a corresponding target log file based on the file path, performing abnormality detection on the target log file based on a preset abnormality detection rule, obtaining an abnormality detection result, performing chart drawing based on a preset chart configuration file and the target log file, obtaining a target chart, and sending the abnormality detection result and the target chart to the front end, so that the front end visually displays the abnormality detection result and the target chart; the application directly reads the corresponding target file, automatically performs abnormality detection through the preset abnormality detection rule, and performs chart drawing through a preconfigured template, so that time-consuming network transmission, manual abnormality detection and secondary development of a drawing tool are not needed, and the efficiency of log analysis is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a log analysis method, system, device, equipment and storage medium. BACKGROUND

[0002] Log analysis is a process of collecting, processing and interpreting log data generated by a system or an application. The current log analysis method needs to transmit log files through a network, and when performing log analysis, a developer needs to use preset chart drawing parameters of an existing tool to draw charts and manually observe charts and locate problems. Transmitting log files through a network and the developer observing and interpreting charts will consume a large amount of time, and the existing chart drawing tool needs to be redeveloped if it is applied to different business scenarios, which leads to low efficiency of the current log analysis method. SUMMARY

[0003] The main purpose of the present application is to provide a log analysis method, system, device, equipment and storage medium, which aims to solve the technical problem of low efficiency of the current log analysis method.

[0004] To achieve the above purpose, the present application provides a log analysis method applied to a local server of a log analysis system, the log analysis system further comprising a front end, the front end and the local server being directly connected, and the method comprising:

[0005] receiving a file path sent by the front end, and directly reading a corresponding target log file based on the file path, wherein the file path is sent to the local server by the front end in response to a target file selection operation of a user;

[0006] performing abnormality detection on the target log file based on preset abnormality detection rules to obtain an abnormality detection result;

[0007] performing chart drawing based on a preset chart configuration file and the target log file to obtain a target chart, wherein the chart configuration file is preconfigured by a user for a specific business scenario;

[0008] sending the abnormality detection result and the target chart to the front end for visual display of the abnormality detection result and the target chart by the front end.

[0009] In an embodiment, the abnormality detection rules are obtained by a user extending original abnormality detection rules based on a specific business scenario, and the step of performing abnormality detection on the target log file based on preset abnormality detection rules to obtain an abnormality detection result comprises:

[0010] Call a log file parser to parse the target log file to obtain a plurality of target log entries;

[0011] Based on the anomaly detection rule, the target log entries are traversed and detected to obtain the anomaly detection result.

[0012] In an embodiment, the step of performing chart drawing based on the preset chart configuration file and the target log file to obtain a target chart comprises:

[0013] Call a configuration file parser to parse the chart configuration file to obtain a chart configuration template, wherein the chart configuration template comprises chart style configuration and chart content configuration;

[0014] Based on the chart style configuration, the chart content configuration, and the target log file, the target chart with corresponding style and response content is drawn.

[0015] In an embodiment, the chart configuration template further comprises chart source data configuration and data mapping configuration, and the step of drawing the target chart with corresponding style and response content based on the chart style configuration, the chart content configuration, and the target log file further comprises:

[0016] Based on the chart source data configuration, chart source data for chart drawing is determined from the target log file;

[0017] Based on the data mapping configuration, the chart source data is data mapped to obtain target mapping data;

[0018] Based on the chart style configuration, the chart content configuration, and the target mapping data, the target chart with corresponding style and response content is drawn.

[0019] To achieve the above-mentioned purpose, the present application further proposes a log analysis method applied to a front end of a log analysis system, wherein the log analysis system further comprises a local server, the front end and the local server are directly connected, and the method comprises:

[0020] In response to a target file selection operation of a user, a file path of the target file is sent to the local server, so that the local server directly reads a corresponding target log file based on the file path, performs anomaly detection on the target log file based on a preset anomaly detection rule to obtain an anomaly detection result, performs chart drawing based on a preset chart configuration file and the target log file to obtain a target chart, and sends the anomaly detection result and the target chart to the local server; wherein the chart configuration file is pre-configured by the user for a specific business scenario.

[0021] The local server sends the abnormality detection result and the target chart to the front end, and the front end visually displays the abnormality detection result and the target chart.

[0022] In an embodiment, before the step of sending the file path of the target file to the local server in response to the target file selection operation of the user, the method further comprises:

[0023] In response to a file directory selection operation of the user, determining a target file directory;

[0024] Obtaining log file information of each log file in the target file directory;

[0025] Visually displaying the log file information locally, so that the user selects a target log file from each log file.

[0026] In addition, to achieve the above-mentioned purpose, the application further provides a log analysis system, which comprises a local server and a front end, and the local server and the front end are directly connected.

[0027] The local server receives the file path sent by the front end, directly reads the corresponding target log file based on the file path, performs abnormality detection on the target log file based on a preset abnormality detection rule, obtains an abnormality detection result, performs chart drawing based on a preset chart configuration file and the target log file, obtains a target chart, and sends the abnormality detection result and the target chart to the front end, so that the front end visually displays the abnormality detection result and the target chart, wherein the chart configuration file is pre-configured by the user for a specific business scenario.

[0028] The front end sends the file path of the target file to the local server in response to a target file selection operation of the user, receives the abnormality detection result and the target chart sent by the local server, and visually displays the received abnormality detection result and the target chart.

[0029] In addition, to achieve the above-mentioned purpose, the application further provides a log analysis device, which comprises:

[0030] The file reading module is configured to receive a file path sent by the front end and directly read a corresponding target log file based on the file path, wherein the file path is sent by the front end to the local server in response to a target file selection operation of the user.

[0031] The abnormality detection module is configured to perform abnormality detection on the target log file based on a preset abnormality detection rule, and obtain an abnormality detection result.

[0032] a chart drawing module, configured to draw a chart based on a preset chart configuration file and the target log file, to obtain a target chart, wherein the chart configuration file is configured by a user in advance for a specific business scenario;

[0033] a sending module, configured to send the abnormality detection result and the target chart to the front end, so that the front end visually displays the abnormality detection result and the target chart.

[0034] In addition, to achieve the above object, the present application further provides a log analysis device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the log analysis method as described above.

[0035] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the log analysis method as described above.

[0036] The one or more technical solutions provided by the present application have at least the following technical effects:

[0037] The present application receives the file path sent by the front end, directly reads the corresponding target log file based on the file path, performs abnormality detection on the target log file based on a preset abnormality detection rule, obtains an abnormality detection result, draws a chart based on a preset chart configuration file and the target log file, obtains a target chart, sends the abnormality detection result and the target chart to the front end, so that the front end visually displays the abnormality detection result and the target chart, wherein the file path is sent by the front end to the local in response to a target file selection operation of a user, and the chart configuration file is configured by a user in advance for a specific business scenario.

[0038] Compared with the current log analysis method, the present application directly reads the corresponding target file based on the file path, without the need for network transmission of the file. In the process of log analysis, the abnormality is automatically detected by the preset abnormality detection rule, without the need for manual observation of the chart for problem positioning by the developer. Then, the target drawing template configured by the user in advance obtained by the analysis can enable the user to use the template configured by himself according to the specific business scenario for chart drawing, instead of using the preset parameters of the existing tool. Therefore, the present application does not need to perform time-consuming network transmission, manual abnormality detection and secondary development of the drawing tool, so as to improve the efficiency of log analysis. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, based on these drawings, other drawings can also be obtained without creative labor.

[0041] Figure 1 A flowchart provided for the log analysis method embodiment one of the present application;

[0042] Figure 2 A flowchart provided for the log analysis method embodiment two of the present application;

[0043] Figure 3 A module structure diagram of the log analysis device of the present application embodiment;

[0044] Figure 4 A device structure diagram of the hardware running environment involved in the log analysis method in the present application embodiment.

[0045] The purpose implementation, functional features and advantages of the present application will be further explained with reference to the accompanying drawings combined with the embodiments. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0047] In order to better understand the technical solutions of the present application, the following will be described in detail combined with the drawings in the specification and specific embodiments.

[0048] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as tablet computer, personal computer, mobile phone, etc., or an electronic device, log analysis device, etc. capable of realizing the above functions. The present embodiment and each of the following embodiments will be described below taking the log analysis system as an example.

[0049] The current log analysis methods mainly include a client-server architecture scheme and a single machine file access scheme. The client-server architecture scheme reads log files through a client software, transmits data to a server for analysis and visualization processing. The typical process includes: a user selects a log file on the client, data is uploaded to the server through the network, the server analyzes the log and generates a chart, and chart data is returned to the client for display. The single machine file access scheme mainly directly opens a log file through a local software, manually filters key fields for analysis.

[0050] However, the current method needs to transmit log files through the network, and when performing log analysis, the developer needs to use the preset chart drawing parameters of the existing tool to draw a chart, and manually observe the chart and locate the problem. Transmitting log files through the network and the developer observing and interpreting the chart will consume a lot of time, and the existing chart drawing tool needs to be redeveloped if it is applied to different business scenarios, resulting in low efficiency of the current log analysis method.

[0051] Based on this, the log analysis method provided in the embodiments of the present application is applied to a local server of a log analysis system, and the log analysis system further includes a front end. The front end and the local server are directly connected, and the log analysis method is described with reference to Figure 1 , Figure 1 The flowchart of the first embodiment of the log analysis method of the present application is shown in the figure.

[0052] In this embodiment, the log analysis method includes steps S10-S40:

[0053] Step S10, receiving a file path sent by the front end, directly reading a corresponding target log file based on the file path, wherein the file path is sent by the front end to the local in response to a target file selection operation of the user by the front end;

[0054] It should be noted that in this embodiment, the server is an HTTP (Hypertext Transfer Protocol) server, the HTTP server is built using bokeh (a visualization database), the front end is a user operation interface, the user operation page includes a browser page, is responsible for receiving user input and operation and displaying data, the file path is a string used to identify a file storage location in the system, the target log file is a log file specified by the user, contains log content that needs to be analyzed by the user, and reading of the log is performed through an HTTP service module.

[0055] It can be understood that in the traditional method, the user needs to select a log file through the front end, the front end uploads the file to the remote server, and the server parses and processes the file after receiving the file. However, if the file is a large log file, uploading the log file requires bandwidth and takes a long time, and the file upload requires the server to store the file, which occupies the server memory, thereby reducing the efficiency of log analysis.

[0056] Therefore, the embodiment directly connects the local server and the front end, and in the case that the local server receives the file path sent by the front end, the local server can directly read the corresponding file through the corresponding browser and API (Application Programming Interface, application programming interface), thereby avoiding redundant network transmission links, so that the log analysis does not need to wait for the completion of network transmission, and the efficiency of log analysis is improved. Since the above method of the embodiment does not transmit the log file through the network, the embodiment can analyze the log file without network, improving the flexibility of log analysis. And since the above method of the embodiment directly reads the corresponding log file, there is no transmission and copying operation of the log file, so that additional storage of the log file is not required, reducing the use of resources.

[0057] In an embodiment, the reading operation of the log file further includes: dividing the log file into multiple parts, and reading the log file in blocks through memory mapping, improving the reading efficiency of the log file and preventing the blocking of reading threads caused by too much file reading at a time.

[0058] In an embodiment, the step of dividing the log file into multiple parts and reading the log file in blocks through memory mapping further includes: when dividing the log file in blocks, performing importance calculation on the log file based on log file content of the log file, wherein the log file content includes timestamp, log description, log level, source identifier and user information. The log file with an importance greater than a preset threshold is divided into a block to obtain a target log file block, and the target log file block is read based on a preset file reading strategy, wherein the file reading strategy includes: preferentially reading the target log file block and only reading the log file block.

[0059] In an embodiment, based on the log file content, the step of performing the importance calculation on the log file comprises: determining a business scenario matching degree of each log file content in the log file based on a current business scenario, determining a weight of the each log file content based on the business scenario matching degree, and performing the importance calculation on the log file based on the weight. For example, in the risk analysis scenario of flight control, flight data and timestamp data in the log file can be assigned a higher weight.

[0060] In step S20, the target log file is subjected to abnormality detection based on a preset abnormality detection rule, to obtain an abnormality detection result.

[0061] It should be noted that the abnormality detection is performed by a local error analysis module, the abnormality detection rule is a conditional expression or a pattern matching rule for identifying potential errors, abnormal behaviors or key events in the log, and the abnormality detection rule includes keyword detection, regular expression detection and state sequence detection. The abnormality detection result is a set of abnormal information output after scanning the log file, and the abnormality detection result includes an abnormal type, an abnormal result text and an abnormal occurrence time.

[0062] In a feasible implementation, the abnormality detection rule is obtained by extending an original abnormality detection rule based on a specific business scenario by a user, and the specific implementation of performing the abnormality detection on the target log file based on the preset abnormality detection rule to obtain the abnormality detection result can also be: calling a log file parser to parse the target log file to obtain a plurality of target log entries, and performing traversal detection on the target log entries based on the abnormality detection rule to obtain the abnormality detection result.

[0063] It should be noted that the log parsing is performed by a log file parser of a local log parsing module, the log file parser is a component for parsing a raw log file structured format into a plurality of independent log entries, and the target log entry is a log record after parsing, mainly including a timestamp, a log level, log content and the like.

[0064] It can be understood that the current log analysis method usually needs manual viewing of the content of the log, and the developer needs to manually judge whether there is an abnormality in combination with the business logic, so that the use of the current log analysis method needs to wait for the developer to make a judgment, resulting in a low efficiency of log analysis. Although a few systems introduce a rule engine, the rules of the rule engine are mostly hard coded, and lack of scalability and flexibility, so that the abnormality detection rule used for the current business scenario is not applicable to other business scenarios. If the abnormality detection is to be performed by the rule engine in other business scenarios, the development needs to be re-performed, resulting in poor flexibility of log analysis.

[0065] It can be understood that the embodiment performs log file anomaly detection through the extensible anomaly detection rule, without manual judgment of the anomaly by the developer, thereby improving the efficiency of log analysis. Moreover, the anomaly detection rule is extensible, and thus, the anomaly detection rule can be extended based on different business scenarios, without re-development of the anomaly detection rule, thereby improving the flexibility of log analysis.

[0066] In an embodiment, the anomaly detection rule includes a plurality of anomaly detection sub-rules applicable to different scenarios, and the anomaly detection sub-rules are stored in a database. During anomaly detection, the anomaly detection rule in the database can be matched based on a specific business scenario to obtain a target anomaly detection rule, thereby further improving the efficiency of log analysis without manual configuration of the anomaly detection rule.

[0067] In an embodiment, the anomaly detection rule further includes a context combination rule. During anomaly detection, the context of the log file is associated based on the timestamp and log category of the log file to obtain log file association information. The log anomaly detection is performed based on the log file association information and the context combination rule to obtain an anomaly detection result.

[0068] In step S30, a target chart is obtained by chart drawing based on a preset chart configuration file and the target log file. The chart configuration file is configured by a user in advance for a specific business scenario.

[0069] It should be noted that the chart configuration file is a structured configuration file for describing information such as chart type, data field mapping relationship, and style setting. In the embodiment, the chart configuration file is a JSON (JavaScript Object Notation) file.

[0070] It can be understood that the current chart drawing of the log file usually uses preset parameters of an existing tool. However, the chart drawing requirements are different for different users and different business scenarios. Therefore, if the current method is used for chart drawing, the development of the chart drawing tool needs to be re-performed with the change of the business scenario and the user, thereby reducing the flexibility and efficiency of log analysis.

[0071] Therefore, the embodiment performs chart drawing through the chart configuration file. The chart configuration file is stored in the form of JSON in the embodiment, thereby making the chart configuration file independent of the code and reusable across projects. When a user performs chart drawing of the log file, the user only needs to simply modify the configuration file and load the corresponding configuration file to achieve the corresponding chart drawing, thereby improving the efficiency of log file analysis without re-development.

[0072] In an implementation, the chart is drawn based on the preset chart configuration file and the target log file, and a specific embodiment of the target chart is obtained, which can also be:

[0073] A configuration file parser is called to parse the chart configuration file to obtain a chart configuration template, wherein the chart configuration template includes chart style configuration and chart content configuration, and the target chart with corresponding style and response content is drawn based on the chart style configuration, the chart content configuration and the target log file.

[0074] It should be noted that the parsing of the configuration file is performed by a configuration file parser of a local template engine module, and the configuration file parser is used to read and parse components of the chart configuration file and convert them into executable data structures. The chart style configuration is used to describe the settings of the appearance attributes of the chart, and the appearance attributes include color, font, legend position and background color. The chart content configuration describes which data fields should be displayed by the chart and the mapping relationship thereof, and specifically includes 6-axis, Y-axis bound log fields and their contents.

[0075] It can be understood that, by pre-configuring the chart style configuration and the chart content configuration, the target chart with corresponding style and content is drawn when the chart is drawn, and when the chart style and content need to be changed, only the chart configuration file needs to be changed, so that the drawn chart can become the required chart, thereby improving the flexibility of log chart drawing.

[0076] In an implementation, the chart configuration template further includes chart source data configuration and data mapping configuration, and the step of drawing the target chart with corresponding style and response content based on the chart style configuration, the chart content configuration and the target log file further includes:

[0077] Based on the chart source data configuration, chart source data used for chart drawing is determined from the target log file, data mapping is performed on the chart source data based on the data mapping configuration to obtain target mapping data, and the target chart with corresponding style and response content is drawn based on the chart style configuration, the chart content configuration and the target mapping data.

[0078] It should be noted that the chart data source configuration is the data source of the chart, including fields, data and keywords extracted from the log for chart drawing. The data mapping configuration is a calculation method of the original data for chart drawing.

[0079] It can be understood that when the chart is drawn, not all data of the log file is usually used, but data that needs to be drawn, and the data required by different businesses is different, and the data that needs to be drawn when log analysis is performed is also different. In order to improve the effect of chart drawing, the data needs to be processed when drawing the chart, so that the data chart drawn can more intuitively represent the state of the corresponding object of the log.

[0080] Therefore, the embodiment determines the required source data from the log file through chart source data configuration and data mapping configuration, and then converts the source data into mapping data that can better reflect the characteristics of the corresponding object through the mapping rule, thereby improving the effect of the drawn chart. The above process is automatically performed during chart drawing, so that the embodiment does not need to manually perform data screening and calculation through the above method, improves the effect of chart drawing, and improves the efficiency of chart drawing.

[0081] In an embodiment, the determination manner of the chart configuration file further includes that the chart configuration file includes a plurality of chart configuration subfiles, the chart configuration subfiles are stored in a database, and a target chart configuration subfile is determined from the chart configuration subfiles in the database based on the obtained business type and a preset chart drawing type. By automatically matching the chart drawing file based on the business type and the chart drawing type, manual chart configuration can be avoided, and the efficiency of chart drawing is improved.

[0082] In an embodiment, each part in the drawn target chart is further associated with source data when the chart is drawn. The embodiment can also respond to a chart clicking operation of a user, determine source data corresponding to the clicking operation, and display the source data.

[0083] In an embodiment, the step of determining the source data corresponding to the clicking operation and displaying the source data further includes: if the source data is mapping data calculated through a mapping rule, the mapping data is displayed, and the corresponding source data is determined based on the mapping rule. By associating the target chart with the source data of the log, the user can directly view the corresponding target data through the chart, so that the user can quickly obtain the required data for analysis, and the analysis effect of the target chart is improved.

[0084] Step S40, the abnormality detection result and the target chart are sent to the front end, so that the front end visually displays the abnormality detection result and the target chart.

[0085] It can be understood that the anomaly detection result is usually displayed in the form of text in the console or the log panel, and since the anomaly detection result and the target chart are obtained by different parts and the anomaly detection result is usually text data while the target chart is image data. The anomaly detection result and the target chart are descriptions of different aspects of a problem, and the two parts need to be combined when solving the corresponding problem. In the current log analysis method, only the anomaly detection result or the target chart is usually displayed, resulting in a decline in the effect of log analysis.

[0086] Therefore, the anomaly detection result and the target chart are sent to the front end in the embodiment, so that the front end can combine and display the anomaly detection result and the target chart, thereby helping the user to quickly solve the problem and improving the effect of log analysis.

[0087] In an embodiment, the anomaly detection result and the target chart are bidirectionally associated. The embodiment can correspond to a target chart click operation of the user, determine whether the clicked chart part is an abnormal area based on the clicked chart part, and if so, jump to the corresponding abnormal information text part.

[0088] The embodiment also responds to an anomaly detection result click operation of the user, jumps to the corresponding target chart part based on the clicked target anomaly detection result.

[0089] Through the bidirectional association and jumping of the target chart and the anomaly detection result, the effect of log analysis can be improved, so that the user can quickly determine the correlation of the problem and quickly solve the problem.

[0090] In summary, the embodiment receives a file path sent by the front end, directly reads a corresponding target log file based on the file path, performs anomaly detection on the target log file based on a preset anomaly detection rule to obtain an anomaly detection result, performs chart drawing based on a preset chart configuration file and the target log file to obtain a target chart, sends the anomaly detection result and the target chart to the front end for visual display of the anomaly detection result and the target chart by the front end, wherein the file path is sent to the local by the front end in response to a target file selection operation of the user, and the chart configuration file is preconfigured by the user for a specific business scenario.

[0091] Compared with the current log analysis method, the embodiment directly reads the corresponding target file based on the file path without network transmission of the file. In the log analysis process, the abnormality is automatically detected through the preset abnormality detection rule without manual observation of the chart by the developer for problem positioning. Then, the user can use the target drawing template configured in advance by the user instead of using the preset parameters of the existing tool to draw the chart according to the specific business scenario. Therefore, the embodiment does not need to perform time-consuming network transmission, manual abnormality detection and secondary development of the drawing tool, thereby improving the efficiency of log analysis.

[0092] Based on this, the embodiment of the application further provides a log analysis method applied to a front end of a log analysis system, wherein the log analysis system further comprises a local server, the front end and the local server are directly connected, and the log analysis method is as described in the first embodiment. Figure 2 , Figure 2 The flowchart of the log analysis method of the first embodiment of the application is shown in FIG. 1.

[0093] In the embodiment, the log analysis method comprises steps A10-A20.

[0094] In step A10, in response to a target file selection operation of a user, the file path of the target file is sent to the local server, so that the local server directly reads the corresponding target log file based on the file path, performs abnormality detection on the target log file based on a preset abnormality detection rule to obtain an abnormality detection result, performs chart drawing based on a preset chart configuration file and the target log file to obtain a target chart, and sends the abnormality detection result and the target chart to the local server; wherein the chart configuration file is configured in advance by the user for a specific business scenario.

[0095] It should be noted that the selection of the target log file is based on the file interaction module of the front end, wherein the front end refers to a user operation interface, and the operation page comprises a file list window, a browser window and a drawing alarm window.

[0096] It can be understood that, unlike the general log analysis method, the log analysis method of the embodiment does not need to perform transmission and copying of the log file, but directly reads the log file through the local server. The target log file to be read needs to be selected by the user, and therefore the embodiment provides the front end interface for the user to select the log file.

[0097] After the user completes the selection of the target log, the local server needs to read the corresponding log file, therefore, the embodiment sends the file path of the target file to the local server in response to the target file selection operation of the user. Since the front end is directly connected with the server, the embodiment can directly transmit the file path without the need of transmission through the network. The transmission of the file path can enable the local server to directly read the target file and perform the subsequent parsing and drawing operation based on the file path, thereby improving the efficiency of log analysis.

[0098] In step A20, the abnormality detection result and the target chart sent by the local server are received, and the received abnormality detection result and target chart are visually displayed.

[0099] It should be noted that the visual display of the abnormality detection result and the target chart is performed by the front-end display module of the front end, and the front end receives an HTML (HyperText Markup Language) file of the target chart, and the abnormality detection result and the target chart are specifically displayed in the browser page of the front-end display module.

[0100] It can be understood that, since the front end and the local server are directly connected, the abnormality detection result and the target chart obtained by the local server by analyzing the log file can be directly sent to the front end without the need of transmission through the network, thereby improving the efficiency of visual display. Meanwhile, the embodiment displays the abnormality detection result and the target chart in the same interface, which facilitates the user to quickly find and solve problems, and improves the effect of log analysis.

[0101] In an embodiment, before the target chart is visually displayed, the browser further renders the target chart to convert it into a visual image that can be displayed, and reduces the computing resources consumed thereby.

[0102] In an embodiment, if the local server generates an error in the process of drawing the chart, thereby failing to completely draw the target chart, the embodiment displays the drawing error information through the drawing alarm page of the front end, so that the user can quickly solve the problem and improve the reliability of log analysis.

[0103] In a feasible embodiment, before the file path of the target file is sent to the local server in response to the target file selection operation of the user, the specific implementation can also be:

[0104] In response to a file directory selection operation of the user, a target file directory is determined, log file information of each log file in the target file directory is acquired, the log file information is visually displayed locally, and a target log file is selected from each log file by the user.

[0105] It should be noted that the file target selection operation is an operation of selecting a directory containing multiple log files by the user through an operation on an interface, and the log file information includes size, type and generation time of the log file.

[0106] It can be understood that in the traditional log analysis method, the user cannot view the overview information of all log files in the current directory, and if multiple log files need to be analyzed, the selection and uploading operations must be repeatedly performed multiple times, which makes the log selection interaction inefficient and reduces the log analysis efficiency.

[0107] Therefore, the embodiment responds to the log directory selection operation of the user, automatically lists all log files therein, so that the user can directly view the log file information on the front-end interface and preview and filter the log files, thereby directly selecting the log files that need to be analyzed without multiple selection of the log files, improving the selection efficiency of the log files and further improving the analysis efficiency of the logs.

[0108] In an embodiment, the front-end can also acquire a business type, determine relevant log files based on the business type and the log file information, and sort the log files, so that the user can select a target log file directly from the relevant log files.

[0109] In summary, the embodiment responds to the target file selection operation of the user, sends a file path of the target file to the local server, so that the local server directly reads a corresponding target log file based on the file path, performs abnormality detection on the target log file based on a preset abnormality detection rule to obtain an abnormality detection result, performs chart drawing based on a preset chart configuration file and the target log file to obtain a target chart, sends the abnormality detection result and the target chart to the local server, receives the abnormality detection result and the target chart sent by the local server, and visually displays the received abnormality detection result and the target chart, wherein the chart configuration file is preconfigured by the user for a specific business scenario.

[0110] The embodiment shows the corresponding log file under the catalog in response to the catalog selection operation of the user, so that the user does not need to select the log file multiple times, and the selection efficiency of the log file is improved. In response to the log file selection operation of the user, the corresponding file path is sent, so that the user can directly read the corresponding log file. Since the front end and the local server are directly connected, the file path can be directly sent without network, and since the reading is directly performed through the local server, network transmission is not needed, so that the efficiency of log analysis is improved.

[0111] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the log analysis method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0112] The present application also provides a log analysis system, which comprises a local server and a front end, and the local server and the front end are directly connected.

[0113] The local server receives the file path sent by the front end, directly reads the corresponding target log file based on the file path, performs abnormality detection on the target log file based on a preset abnormality detection rule, obtains an abnormality detection result, performs chart drawing based on a preset chart configuration file and the target log file, obtains a target chart, and sends the abnormality detection result and the target chart to the front end, so that the front end visually displays the abnormality detection result and the target chart. The chart configuration file is preconfigured by the user for a specific business scenario.

[0114] The front end sends the file path of the target file to the local server in response to the target file selection operation of the user, receives the abnormality detection result and the target chart sent by the local server, and visually displays the received abnormality detection result and target chart.

[0115] The present application also provides a log analysis device, please refer to Figure 3 , the log analysis device comprises:

[0116] The file reading module 10 is used for receiving the file path sent by the front end, and directly reading the corresponding target log file based on the file path. The file path is sent by the front end to the local server in response to the target file selection operation of the user.

[0117] The abnormality detection module 20 is used for performing abnormality detection on the target log file based on a preset abnormality detection rule, and obtaining an abnormality detection result.

[0118] The chart drawing module 30 is configured to draw a target chart based on a preset chart configuration file and the target log file, wherein the chart configuration file is configured by a user in advance for a specific business scenario.

[0119] The sending module 40 is configured to send the abnormality detection result and the target chart to the front end, so that the front end visually displays the abnormality detection result and the target chart.

[0120] In an embodiment, the abnormality detection module further comprises:

[0121] The log analysis sub-module is configured to call a log file parser to parse the target log file to obtain a plurality of target log entries.

[0122] The abnormality detection sub-module is configured to traverse and detect the target log entries based on the abnormality detection rule to obtain the abnormality detection result.

[0123] In an embodiment, the chart drawing module further comprises:

[0124] The configuration analysis sub-module is configured to call a configuration file parser to parse the chart configuration file to obtain a chart configuration template, wherein the chart configuration template comprises a chart style configuration and a chart content configuration.

[0125] The chart drawing sub-module is configured to draw the target chart with corresponding styles and responsive contents based on the chart style configuration, the chart content configuration and the target log file.

[0126] In an embodiment, the chart drawing sub-module further comprises:

[0127] The data determination unit is configured to determine chart source data for chart drawing from the target log file based on the chart source data configuration.

[0128] The data mapping unit is configured to perform data mapping on the chart source data based on the data mapping configuration to obtain target mapping data.

[0129] The chart drawing unit is configured to draw the target chart with corresponding styles and responsive contents based on the chart style configuration, the chart content configuration and the target mapping data.

[0130] In an embodiment, the log analysis apparatus further comprises:

[0131] The path sending module is configured to, in response to a target file selection operation of a user, send a file path of the target file to the local server, so that the local server reads a corresponding target log file based on the file path, performs abnormality detection on the target log file based on a preset abnormality detection rule, obtains an abnormality detection result, performs chart drawing based on a preset chart configuration file and the target log file, obtains a target chart, and sends the abnormality detection result and the target chart to the local server; wherein the chart configuration file is a chart configuration file preconfigured by the user for a specific business scenario.

[0132] The visualization module is configured to receive the abnormality detection result and the target chart sent by the local server, and visually display the received abnormality detection result and target chart.

[0133] In an embodiment, the log analysis apparatus further comprises:

[0134] The directory determination module is configured to, in response to a file directory selection operation of a user, determine a target file directory.

[0135] The information acquisition module is configured to acquire log file information of each log file in the target file directory.

[0136] The information display module is configured to visually display the log file information on the local server, so that the user selects a target log file from each log file.

[0137] The log analysis apparatus provided by the present application adopts the log analysis method in the above embodiments, and can solve the technical problem of low efficiency of the current log analysis method. Compared with the prior art, the log analysis apparatus provided by the present application has the same beneficial effects as the log analysis method provided by the above embodiments, and other technical features in the log analysis apparatus are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0138] The present application provides a log analysis device, which comprises at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the log analysis method in the above embodiment one.

[0139] The following refers to Figure 4The diagram illustrates a structural schematic of a log analysis device suitable for implementing embodiments of this application. The log analysis device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The log analysis device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 4 As shown, the log analysis device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the log analysis device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the log analysis device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows log analysis devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0141] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0142] The log analysis device provided by the present application adopts the log analysis method in the above-mentioned embodiments, and can solve the technical problem of low efficiency of the current log analysis method. Compared with the prior art, the log analysis device provided by the present application has the same beneficial effects as the log analysis method provided by the above-mentioned embodiments, and other technical features in the log analysis device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0143] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0144] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0145] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer program) for executing the log analysis method in the above-mentioned embodiments.

[0146] The computer readable storage medium provided in the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any suitable medium, including but not limited to electrical wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.

[0147] The above computer readable storage medium can be contained in the log analysis device, or can exist separately without being assembled into the log analysis device.

[0148] The above computer readable storage medium carries one or more programs, which, when executed by the log analysis device, cause the log analysis device to perform the above log analysis method.

[0149] Computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0150] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0151] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the names of the modules do not limit the modules themselves.

[0152] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the above-mentioned log analysis method, and can solve the technical problem of low efficiency of the current log analysis method. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the log analysis method provided by the above-mentioned embodiments, which will not be repeated here.

[0153] The user-related data (for example, user attribute data, user behavior data, and user geographic location, etc., the data types herein should be modified adaptively according to the scheme content) involved in the present application are obtained after obtaining the user's permission or consent; that is, when the present application is applied to a specific product or technology, the user's permission is required to realize the acquisition and processing of related data, and the processing of related data needs to comply with relevant laws, regulations and regulatory standards of relevant countries and regions.

[0154] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structural transformation made by using the contents of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A log analysis method, characterized in that, A local server is used in a log analysis system, the log analysis system also includes a front-end, the front-end and the local server are directly connected, and the method includes: Receive the file path sent by the front end, and directly read the corresponding target log file based on the file path, wherein the file path is sent by the front end to the local machine in response to the user's target file selection operation; Based on preset anomaly detection rules, anomaly detection is performed on the target log file to obtain anomaly detection results; Based on the preset chart configuration file and the target log file, a chart is drawn to obtain the target chart. The chart configuration file is pre-configured by the user for a specific business scenario. The steps for drawing a chart based on a preset chart configuration file and the target log file to obtain the target chart include: The configuration file parser is invoked to parse the chart configuration file and obtain the chart configuration template, wherein the chart configuration template includes chart style configuration and chart content configuration; Based on the chart style configuration, the chart content configuration, and the target log file, draw the target chart with the corresponding style and content; The chart configuration template also includes chart source data configuration and data mapping configuration. The step of drawing the target chart with corresponding style and content based on the chart style configuration, the chart content configuration, and the target log file further includes: Based on the chart source data configuration, determine the chart source data used for chart drawing from the target log file; Based on the data mapping configuration, the chart source data is mapped to obtain the target mapped data; Based on the chart style configuration, the chart content configuration, and the target mapping data, draw the target chart with the corresponding style and content; The anomaly detection results and the target chart are sent to the front end so that the front end can visualize the anomaly detection results and the target chart.

2. The method as described in claim 1, characterized in that, The anomaly detection rules are derived by the user by extending the original anomaly detection rules based on specific business scenarios. The steps of performing anomaly detection on the target log file based on the preset anomaly detection rules to obtain the anomaly detection results include: The log file parser is invoked to parse the target log file, resulting in multiple target log entries; Based on the aforementioned anomaly detection rules, the target log entries are traversed and detected to obtain the anomaly detection results.

3. A log analysis method, characterized in that, A front-end application is used in a log analysis system, the log analysis system also including a local server, the front-end and the local server being directly connected, the method comprising: In response to the user's target file selection operation, the file path of the target file is sent to the local server, so that the local server can directly read the corresponding target log file based on the file path. Based on preset anomaly detection rules, anomaly detection is performed on the target log file to obtain the anomaly detection result. A configuration file parser is invoked to parse the preset chart configuration file to obtain a chart configuration template. The chart configuration template includes chart style configuration, chart content configuration, chart source data configuration, and data mapping configuration. Based on the chart source data configuration, chart source data for chart drawing is determined from the target log file. Based on the data mapping configuration, data mapping is performed on the chart source data to obtain target mapping data. Based on the chart style configuration, the chart content configuration, and the target mapping data, a target chart with the corresponding style and content is drawn. The anomaly detection result and the target chart are then sent to the local server. The chart configuration file is pre-configured by the user for a specific business scenario. Receive the anomaly detection results and target chart sent by the local server, and visualize the received anomaly detection results and target chart.

4. The method as described in claim 3, characterized in that, Before the step of sending the file path of the target file to the local server in response to the user's target file selection operation, the method further includes: In response to the user's file directory selection action, determine the target file directory; Obtain log file information for each log file in the target file directory; The log file information is visualized locally so that users can select the target log file from the various log files.

5. A log analysis system, characterized in that, The log analysis system includes a local server and a front-end, and the local server and the front-end are directly connected: The local server receives the file path sent by the front end, directly reads the corresponding target log file based on the file path, performs anomaly detection on the target log file based on preset anomaly detection rules, obtains the anomaly detection result, calls the configuration file parser to parse the preset chart configuration file, obtains the chart configuration template, the chart configuration template includes chart style configuration, chart content configuration, chart source data configuration, and data mapping configuration, determines the chart source data for chart drawing from the target log file based on the chart source data configuration, performs data mapping on the chart source data based on the data mapping configuration, obtains target mapping data, draws a target chart with corresponding style and content based on the chart style configuration, the chart content configuration, and the target mapping data, and sends the anomaly detection result and the target chart to the front end for the front end to visualize the anomaly detection result and the target chart. The chart configuration file is pre-configured by the user for a specific business scenario. The front end, in response to the user's target file selection operation, sends the file path of the target file to the local server, receives the anomaly detection results and target chart sent by the local server, and visualizes the received anomaly detection results and target chart.

6. A log analysis device, characterized in that, The device includes: The file reading module is used to receive a file path sent by the front end and directly read the corresponding target log file based on the file path. The file path is sent by the front end to the local machine in response to the user's target file selection operation. An anomaly detection module is used to perform anomaly detection on the target log file based on preset anomaly detection rules and obtain anomaly detection results; The chart drawing module is used to draw charts based on a preset chart configuration file and the target log file to obtain the target chart. The chart configuration file is pre-configured by the user for a specific business scenario. The sending module is used to send the anomaly detection result and the target chart to the front end, so that the front end can visualize the anomaly detection result and the target chart; The configuration parsing submodule is used to call the configuration file parser to parse the chart configuration file and obtain the chart configuration template. The chart configuration template includes chart style configuration and chart content configuration, as well as chart source data configuration and data mapping configuration. The chart drawing submodule is used to draw the target chart with corresponding style and content based on the chart style configuration, the chart content configuration and the target log file; A data determination unit is used to determine the chart source data for chart drawing from the target log file based on the chart source data configuration. The data mapping unit is used to perform data mapping on the chart source data based on the data mapping configuration to obtain target mapping data; The chart drawing unit is used to draw the target chart with corresponding style and corresponding content based on the chart style configuration, the chart content configuration and the target mapping data.

7. A log analysis device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the log analysis method as described in any one of claims 1 to 4.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the log analysis method as described in any one of claims 1 to 4.

Images