A log rapid analysis method, device, and storage medium

Abstract

The present application provides a log rapid analysis method, device, and storage medium. A log rapid analysis method is applied to industrial computers. The method includes the following steps: step S1, obtaining the operation log generated during the operation process, and preprocessing the operation log; step S2, determining the type of the operation log, and parsing the operation log according to the preset rules corresponding to the type of the operation log; step S3, displaying the parsing result of the parsed operation log. The present application can quickly discover problems in the operation log through the parsing results, which is convenient for staff to verify the correctness of industrial computer data, and to discover problems in the operation process of the industrial computer and daily operation and maintenance in advance.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a log rapid analysis method, device, and storage medium. Background Art

[0002] An industrial computer generates nearly 100M plain text logs, which takes several days to read manually. Except for holidays, the industrial computers in the factory basically run 24 hours a day. All customers add up to thousands of computers, generating a huge amount of data every day.

[0003] Different software has different log formats, including but not limited to: the time taken to communicate with the PLC, the time taken for the entire process, the received messages, etc. However, most of the log data becomes electronic waste and enters the recycling bin. The processing of industrial computer logs has the following problems:

[0004] Log data is collected and summarized manually, and it is impossible to confirm whether the data is lost or damaged. During the collection process, the log data is not preliminarily processed. 100M of data often has no more than 1K of valid data, which affects the collection and transmission efficiency. Due to the large amount of log data to read, problems can only be solved when they occur, and then related logs can be found from many logs. Early warning and prevention cannot be carried out, which causes inconvenience to the handling of industrial computer problems and daily operation and maintenance. Summary of the invention

[0005] In order to solve the above problems, the embodiments of the present application provide a log rapid analysis method, device, and storage medium to quickly discover problems in the operation log, thereby facilitating the processing of industrial computer problems and daily operation and maintenance.

[0006] The present application provides a log rapid analysis method, which comprises the following steps:

[0007] A log rapid analysis method is applied to an industrial computer, and the method comprises the following steps:

[0008] Step S1, obtaining the operation log generated during the operation process, and preprocessing the operation log;

[0009] Step S2: determine the type of the operation log, and parse the operation log according to a preset rule corresponding to the type of the operation log;

[0010] Step S3: display the parsed result of the running log.

[0011] Optionally, in combination with any of the above aspects, in another implementation of this aspect, the method further includes:

[0012] Step S4: Acquire characteristic information of the required operation log, and match the operation log that meets the characteristic information in the analysis result.

[0013] Optionally, in combination with any of the above aspects, in another implementation of this aspect, the operation log is preprocessed in step S1, specifically,

[0014] Step S11, uniformly process all operation logs to determine whether the operation logs are legal logs; if the operation logs are legal logs, execute step S2; if the operation logs are illegal logs, record them as 0.

[0015] Optionally, in combination with any of the above aspects, in another implementation of the present aspect, in step S2, the type of the operation log is determined, and the operation log is parsed according to a preset rule, specifically,

[0016] The type of the operation log is obtained according to the content in the operation log; the types of the operation log include complex types and simple types, and the complex type of the operation log is parsed by regular expressions; the simple type of the operation log is parsed by program code.

[0017] Optionally, in combination with any of the above aspects, in another implementation method of the present aspect, parsing the operation log according to preset rules also includes: obtaining the data row of the operation log, determining whether the data row is the first record, and if so, determining whether the first record includes an end feature string, and if it includes the end feature string, adding it to an illegal log list; when the data row is not the first record, determining whether the data row includes a start feature string, and if it includes the start feature string, counting the time and adding it to the time list.

[0018] Optionally, in combination with any of the above aspects, in another implementation of the present aspect, the time list includes average time, median time, and continuous maximum time exceeding a threshold, and the time list is accurate to milliseconds or microseconds.

[0019] Optionally, in combination with any of the above aspects, in another implementation of this aspect, the following steps are further included:

[0020] Step S5: uploading the analysis result to the log server, and analyzing the analysis result through the log server.

[0021] Optionally, in combination with any of the above aspects, in another implementation of the present aspect, in the step S11, it is determined whether the operation log is a legal log based on the thread ID and the operation time in the operation log.

[0022] In another aspect of the present application, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a log rapid analysis method as described above is implemented.

[0023] In another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed, a method for rapid log analysis as described above is implemented.

[0024] As described above, the present application provides a method, device, and storage medium for rapid log analysis, which obtains the operation log generated during the operation process, pre-processes the operation log, determines the type of the operation log, parses the operation log according to the preset rules corresponding to the type of the operation log, and displays the analysis results of the operation log after analysis. The analysis results can quickly discover problems in the operation log, which is convenient for staff to verify the correctness of industrial computer data and discover problems in the operation process of the industrial computer and daily operation and maintenance in advance.

[0025] The above summary is provided to introduce some concepts in a simplified form, which will be further described in detail in the following detailed description. The above summary is neither intended to identify the key features or essential features of the claimed subject matter nor to be used to help determine the scope of the claimed subject matter. The claimed subject matter of the application is not limited to implementations that solve any or all of the disadvantages pointed out in the background technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can also be obtained based on these drawings without paying creative labor. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application for those skilled in the art by referring to specific embodiments.

[0027] Figure 1 A flow chart of a log rapid analysis method provided by this application;

[0028] Figure 2 A schematic diagram of a parsing and matching interface in a log rapid analysis method provided by this application;

[0029] Figure 3 A schematic diagram of the process of parsing and running logs in a log rapid analysis method provided by this application;

[0030] Figure 4 A schematic diagram of the display interface of a log rapid analysis method provided in this application. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0032] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0033] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. An exception to this definition will occur only when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0034] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0035] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0036] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] See also Figure 1 The present application provides a log rapid analysis method, device, and storage medium to quickly discover problems in operation logs, which is convenient for the processing of industrial computer problems and daily operation and maintenance.

[0038] Specifically, the method comprises the following steps:

[0039] Step S1: Obtain the operation logs generated during the operation process and pre-process the operation logs. After obtaining the operation logs, first pre-process all the operation logs, specifically,

[0040] Step S11, uniformly process all the operation logs, and determine whether the operation logs are legal logs; if the operation logs are legal logs, execute step S2; if the operation logs are illegal logs, record it as 0. In this application, whether the operation logs are legal logs is determined based on the thread id and the running time in the operation logs. When the thread id and the running time in the operation log exist, the operation log is considered to be a legal log. When the thread id does not exist, it is considered to be an illegal log and recorded as 0.

[0041] Step S2: determine the type of the operation log, and parse the operation log according to the preset rules corresponding to the type of the operation log. Read the content of the operation log line by line, and determine the type of the operation log according to the content, so as to parse different operation logs and improve the efficiency of parsing.

[0042] After obtaining the running log, the corresponding preset rules are matched through the IMatch matching structure, and the factory interface is matched through IMatchFactory, and the matching factory creates matching objects as needed. The matching entry class is matched through CMathAllThread, and the processing threads are matched to process multiple threads at the same time. According to the thread id in the running log, the pre-processed running log is assigned to different matching objects. All classes are indirectly called through the matching entry class.

[0043] In step S2, the type of the operation log is determined, and the operation log is parsed according to a preset rule corresponding to the type of the operation log, specifically,

[0044] The type of the operation log is obtained according to the content in the operation log; the types of the operation log include complex types and simple types, and the operation log of the complex type is matched and parsed by regular expression; regular expression (regular expression) describes a string matching pattern (pattern), which can be used to check whether a string contains a certain substring, replace the matched substring, or extract a substring that meets a certain condition from a string, etc. The operation log of the simple type is matched and parsed by program code. The type of the operation log is mainly determined by the features in the operation log. When there is a single feature, it is a simple type, and when there are start features and end features, etc., it is a complex type. In addition, the operation log of the complex type can be matched and parsed through the mathematics class. The mathematics class encapsulates some mathematical operations that are not provided by the system, such as: the maximum substring sum, to meet the parsing requirements of the operation log.

[0045] In the run log, CMatchPreEndNotSame: The match object with different start and end features. For example, in a match, the start feature is "CHiKCamera::DataCallback:begin" and the end feature is "CHiKCamera::DataCallback:end".

[0046] CMatchPreEndSame: Single feature matching. The feature is "ZF11108". The time difference between the second record including the log of ZF11108 and the first record including the log of ZF11108. The third record including the log of ZF11108 and the time difference between the second record including the log of ZF11108.

[0047] The parsing process is as follows Figure 2 , 3 As shown, the parsing of the operation log according to the preset rules also includes: obtaining the data row of the operation log, determining whether the data row is the first record, and if so, determining whether the first record includes an end feature string, and if it includes an end feature string, adding it to the illegal log list; when the data row is not the first record, determining whether the data row includes a start feature string, and if it includes a start feature string, counting the time and adding it to the time list. The time list includes the average time, the median time, and the continuous maximum time exceeding the threshold, and the time list is accurate to milliseconds or microseconds.

[0048] Step S3: Display the parsed results of the running log. Figure 4 As shown, after parsing, the content of the operation log can be directly displayed, so that the staff can quickly view the valid content of the operation log without reading the format file in the operation log, thereby improving the efficiency of operation and maintenance.

[0049] Step S4, obtain the characteristic information of the required operation log, and match the operation log that meets the characteristic information in the analysis result. When the staff needs to query the operation log of a certain time period or a certain industrial computer, they can enter the characteristic information required to query the operation log, such as a specific time period, the number of the industrial computer, the start characteristic or the end characteristic, etc. After obtaining the above characteristic information, directly match the operation log that meets the characteristic information in the analysis result. Matching in the analysis result means matching the qualified operation log from the operation log that has been quickly analyzed. There is no need to analyze all the logs, and the required logs can be accurately and quickly determined, which is convenient for the staff to find.

[0050] Step S5, uploading the analysis result to the log server, and analyzing the analysis result through the log server. When the industrial computer or the network meets the transmission requirements, the analysis result of the operation log is uploaded to the log server. The staff can upload the analysis results of multiple operation logs in the log server to perform operation analysis of the industrial computer or joint analysis of big data of multiple industrial computers, so as to timely and quickly understand the operation status of the industrial computer and perform early warning and prevention.

[0051] The present application provides a method, device, and storage medium for rapid log analysis, which obtains operation logs generated during operation, pre-processes the operation logs, determines the type of the operation logs, parses the operation logs according to preset rules corresponding to the types of the operation logs, and displays the parsed results of the parsed operation logs. The parsed results can be used to quickly discover problems in the operation logs, facilitate staff to verify the correctness of industrial computer data, and discover problems in the operation process of the industrial computer and daily operation and maintenance in advance.

[0052] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0053] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0054] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0055] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0056] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as above, and includes a number of instructions for a terminal device (which can be an electrical device or a network device, etc.) to execute the method of each embodiment of the present application.

[0057] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A log rapid analysis method, characterized in that: Applied to industrial computers, the method includes the following steps: Step S1, obtaining the operation log generated during the operation of the industrial computer, and preprocessing the operation log; wherein the preprocessing of the operation log in step S1 is specifically as follows: Step S11, uniformly process all the operation logs to determine whether the operation logs are legal logs; if the operation logs are legal logs, execute step S2; if the operation logs are illegal logs, record 0; wherein, in step S11, determine whether the operation logs are legal logs according to the thread id and the running time in the operation logs; Step S2, determining the type of the operation log, and parsing the operation log according to the preset rules corresponding to the type of the operation log; wherein, after obtaining the operation log, matching the corresponding preset rules through the IMatch matching structure, and creating a matching object through the IMatchFactory matching factory interface; then, according to the thread id in the operation log, assigning the pre-processed operation log to different matching objects; Wherein, in the step S2, the type of the operation log is determined, and the operation log is parsed according to a preset rule, specifically, The type of the operation log is obtained according to the content in the operation log; the types of the operation log include complex types and simple types, and the operation log of the complex type is parsed by regular expressions; the operation log of the simple type is parsed by program code; wherein, the parsing of the operation log according to preset rules further includes: obtaining a data row of the operation log, determining whether the data row is the first record, and if so, determining whether the first record includes an end feature string, and if so, adding the first record to an illegal log list; when the data row is not the first record, determining whether the data row includes a start feature string, and if so, counting the time consumed and adding it to the time consumption list; The time list includes the average time, the median time, and the maximum continuous time exceeding the threshold, and the time list is accurate to milliseconds or microseconds; Step S3, displaying the parsing result of the parsed running log; Step S4, obtaining characteristic information of the required operation log, and matching the operation log that meets the characteristic information in the analysis result; Step S5: uploading the analysis result to the log server, and analyzing the analysis result through the log server.

2. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a log rapid analysis method as described in claim 1 is implemented.

3. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the method for rapid log analysis described in claim 1 is implemented.

Images