A log processing method, device and computer equipment for lua program interface

Abstract

The application relates to a log processing method and device for a lua program interface and computer equipment. Resource pre-configuration operation is performed based on a lua program interface to obtain a function call library; a log file to be parsed is received; a log parsing function is selected from the function call library, the log parsing function is called to parse the log file, and a log parsing result is obtained. The log processing method for the lua program interface selects a log parsing function from a function call library of a lua program based on a lua program interface to parse a log file and obtain a parsing result, so that the parsing result obtained based on the lua can be conveniently acquired by an application main program written in multiple languages such as C / C++, and the effect of quickly connecting the log parsing function to an outer application main program is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of log processing, in particular to a log processing method and device for a lua program interface and a computer device. BACKGROUND

[0002] System logs are text information that records software and hardware problems when a system is running in real time. System logs can be used to monitor event information that occurs when a system is running, such as a timestamp of an event, an IP address of a request, or a state of an abnormal task. The rich information contained in system logs can help system developers and maintenance personnel better understand system behavior and detect and locate system abnormalities during production.

[0003] With the development of networks, security log auditing has become an important problem in enterprise development decisions. The log analysis function in a log analysis system usually adopts regular analysis. However, with the increasing number of log types, the efficiency of regular analysis becomes slower, and it is difficult to debug and reuse. SUMMARY

[0004] Therefore, it is necessary to provide a log processing method for a lua program interface to solve the above technical problems.

[0005] In a first aspect, the present application provides a log processing method for a lua program interface, which comprises:

[0006] Performing a resource pre-configuration operation based on a lua program interface to obtain a function call library;

[0007] Receiving a log file to be parsed;

[0008] Selecting a log parsing function from the function call library, and calling the log parsing function to parse the log file to obtain a log parsing result.

[0009] In one embodiment, the performing of the resource pre-configuration operation based on the lua program interface to obtain the function call library comprises:

[0010] Performing a resource pre-configuration operation based on a lua program interface to obtain a liblognorm function call library.

[0011] In one embodiment, the receiving of the log file to be parsed comprises:

[0012] Checking log version information at a regular time;

[0013] If the log version information is updated, actively receiving the log file to be parsed.

[0014] In one embodiment, the log parsing function is selected from the function call library, the log parsing function is called to parse the log file, and a log parsing result is obtained, including:

[0015] A log message and a parsing identifier are extracted from the log file.

[0016] The corresponding log parsing function is selected from the function call library according to the parsing identifier;

[0017] The log message is parsed according to the log parsing function, and a log parsing result is obtained.

[0018] In one embodiment, the log processing method further includes:

[0019] A target field is extracted from the log parsing result.

[0020] It is determined whether the parsing is correctly completed according to the target field.

[0021] In one embodiment, it is determined whether the parsing is correctly completed according to the target field, including:

[0022] The target field is compared with a preset field template.

[0023] If the target field is consistent with the preset field template, it is determined that the parsing is correctly completed.

[0024] Secondly, the present application also provides a log processing device for a lua program interface, including:

[0025] A function call library generation module is configured to perform resource pre-configuration operation based on the lua program interface, and obtain a function call library.

[0026] A log receiving module is configured to receive a log file to be parsed.

[0027] A log parsing module is configured to select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain a log parsing result.

[0028] Thirdly, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0029] Resource pre-configuration operation is performed based on the lua program interface, and a function call library is obtained.

[0030] A log file to be parsed is received.

[0031] Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0032] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0033] Resource pre-configuration is performed based on the Lua program interface to obtain a function call library;

[0034] Receive the log file to be parsed;

[0035] Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0036] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0037] Resource pre-configuration is performed based on the Lua program interface to obtain a function call library;

[0038] Receive the log file to be parsed;

[0039] Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0040] Beneficial effects:

[0041] The above-described log processing method applicable to Lua program interfaces selects a log parsing function from the Lua program's function call library based on the Lua program interface to parse the log file and obtain the parsing result. This allows the parsing result obtained based on Lua to be easily obtained by application main programs written in various languages ​​such as C / C++, achieving the effect of quickly connecting the log parsing function to the outer application main program. Attached Figure Description

[0042] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0043] Figure 1 This is an application environment diagram for a logging method for a Lua program interface in one embodiment;

[0044] Figure 2This is a flowchart illustrating a logging method for a Lua program interface in one embodiment.

[0045] Figure 3 This is a flowchart illustrating the process of determining whether log parsing has been completed correctly in one embodiment.

[0046] Figure 4 This is a structural block diagram of a log processing device for a Lua program interface in one embodiment;

[0047] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the log processing method of the Lua program interface in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the log processing method for the Lua program interface in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0052] In one embodiment, such as Figure 2 As shown, a logging method for Lua program interfaces is provided. The logging method includes the following steps:

[0053] Step S20: Perform resource pre-configuration based on the Lua program interface to obtain the function call library.

[0054] The pre-configured function call library includes the liblognorm function, which can accelerate the subsequent log matching process.

[0055] Step S40: Receive the log file to be parsed.

[0056] A log file is a text file, typically containing many lines, each carrying a log message. Different devices create log messages in different formats. These log messages are difficult to interpret directly and require parsing to extract useful information. Log files containing log messages can be obtained in several ways, including reading them directly from a disk or transmitting them over a network.

[0057] Step S60: Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0058] Based on the Lua program interface, a log parsing function is selected from the Lua program's function call library to parse the log file and obtain the parsing result. This parsing result can be easily obtained by the main program written in C / C++, realizing the purpose of embedding the log parsing function as a subroutine into the main program.

[0059] In one embodiment, resource pre-configuration is performed based on the Lua program interface to obtain a function call library, i.e., step S20 includes:

[0060] Step S22: Perform resource pre-configuration based on the Lua program interface to obtain the liblognorm function call library.

[0061] In implementation, liblognorm is only responsible for building the DAG Parser, while the code for performing the matching is written in Lua. The core function of liblognorm is to build the DAG Parser, which saves computing resources by utilizing the DAG structure and can accelerate the log matching process.

[0062] In one embodiment, receiving the log file to be parsed, i.e., step S40 includes:

[0063] Step S42: Periodically check the log version information.

[0064] Step S44: If the log version information is updated, actively receive the log file to be parsed.

[0065] In practice, the prerequisite for accepting a log file to be parsed is whether the log file has been updated. An update to the log file is accompanied by an update to the log version information. If the log version information is updated, it indicates that the log file has also been updated, and subsequent parsing operations need to be performed on the updated log file. Therefore, after detecting an update to the log version information, it is necessary to actively accept the log file to be parsed.

[0066] In one embodiment, a log parsing function is selected from the function call library, and the log parsing function is called to parse the log file to obtain the log parsing result. That is, step S60 includes:

[0067] Step S62: Extract log messages and parsing identifiers from the log file.

[0068] Step S64: Select the corresponding log parsing function from the function call library according to the parsing identifier.

[0069] Step S66: Parse the log message according to the log parsing function to obtain the log parsing result.

[0070] In implementation, log messages and parsing identifiers are extracted from the log file. Based on the parsing identifiers, the corresponding loading function and log parsing function are selected from the function call library. Based on the loading function, log loading rules are retrieved from a pre-defined rule base, and the logs are loaded according to these rules. Finally, the log messages are parsed using the log parsing function to obtain the log parsing results.

[0071] Specifically, selecting a loading function from the function call library to load the parsing rules involves using the loading function to load the parse tree generated from the rule base into memory for subsequent searching of parsing algorithms that match the log file. The rule base contains all log schemes, reflecting the structure of the log message. The parse tree is a representation of the syntax analysis results, typically used to represent the grammatical structure of a language in a tree-like structure.

[0072] Once the parsing rules are successfully loaded, the log parsing function is called to find the parsing algorithm, and the log messages in the log file are matched with the parsing algorithm to obtain the log parsing result.

[0073] In one embodiment, such as Figure 3 As shown, the log processing method also includes:

[0074] Step S82: Extract the target field from the log parsing results;

[0075] Step S84: Determine whether the parsing was completed correctly based on the target field.

[0076] In practice, a detection mechanism based on the target field is proposed to determine whether log parsing has been completed correctly.

[0077] Specifically, step S84 includes:

[0078] Step S842: Compare the target field with the preset field template;

[0079] Step S844: If the target field has a field that matches the preset field template, then the parsing is determined to have been completed correctly.

[0080] In implementation, the target field is first extracted from the obtained log parsing results. The target field here can be a standard field that must exist in the standard log parsing results.

[0081] Next, the target field is compared with a preset field template containing many fields representing successfully parsed fields. If the target field and the preset field template have the same field, it is determined that the aforementioned log parsing operation has been completed correctly.

[0082] Specifically, the logging methods used for Lua program interfaces can be applied to parsers built on the liblognorm technology. liblognorm is a simple submodule of Rsyslog, providing syslog logging capabilities. The liblognorm function `ln_loadSamplesFromString` is used to construct the parse tree of a Directed Acyclic Graph (DAG), and the liblognorm function `ln_normalize` is used to perform log matching.

[0083] Based on the Lua program interface, resource pre-configuration operations are performed to obtain the function call library, which specifically includes:

[0084] Declare liblognorm in the Lua configuration table, map the ln_loadSamplesFromString function as a loading function, and map the ln_normalize function as a parsing function.

[0085] The code that determines whether the rules have been loaded successfully is as follows:

[0086]

[0087] The code for parsing the logs is as follows:

[0088]

[0089] The log parsing results are stored in a table and returned via lua.create_table().

[0090] By utilizing the DAG tree structure, the complexity of the algorithm for finding parsing rules can be reduced, the number of times the parser runs is saved, and the efficiency of finding rules for parsable logs can be improved. Parsing rules can be loaded using a Lua program interface, facilitating quick integration into the main program of the log analysis system.

[0091] Secondly, this application also provides a log processing device for Lua program interfaces, such as... Figure 4 As shown, the log processing device 90 includes:

[0092] The function call library generation module 92 is used to perform resource pre-configuration operations based on the Lua program interface to obtain the function call library.

[0093] The pre-configured function call library includes the liblognorm function, which can speed up the subsequent log matching process.

[0094] Log receiving module 94 is used to receive log files to be parsed.

[0095] A log file is a text file, typically containing many lines, each carrying a log message. Different devices create log messages in different formats. These log messages are difficult to interpret directly and require parsing to extract useful information. Log files containing log messages can be obtained in several ways, including reading them directly from a disk or transmitting them over a network.

[0096] The log parsing module 96 is used to select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0097] The log processing method proposed in this embodiment, based on the Lua program interface, selects a log parsing function from the function call library of the Lua program to parse the log file and obtain the parsing result. This parsing result can be easily obtained by the main program written in C / C++, thus achieving the purpose of embedding the log parsing function as a subroutine into the main program.

[0098] The modules in the aforementioned log processing device based on the Lua program interface can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0099] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data based on a log processing process using a Lua program interface. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a log processing method for a Lua program interface.

[0100] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0101] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0102] Step S20: Perform resource pre-configuration based on the Lua program interface to obtain the function call library;

[0103] The pre-configured function call library includes the liblognorm function, which can speed up the subsequent log matching process.

[0104] Step S40: Receive the log file to be parsed;

[0105] A log file is a text file, typically containing many lines, each carrying a log message. Different devices create log messages in different formats. These log messages are difficult to interpret directly and require parsing to extract useful information. Log files containing log messages can be obtained in several ways, including reading them directly from a disk or transmitting them over a network.

[0106] Step S60: Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0107] Based on the Lua program interface, a log parsing function is selected from the Lua program's function call library to parse the log file and obtain the parsing result. This parsing result can be easily obtained by the main program written in C / C++, realizing the purpose of embedding the log parsing function as a subroutine into the main program.

[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0109] Step S20: Perform resource pre-configuration based on the Lua program interface to obtain the function call library;

[0110] The pre-configured function call library includes the liblognorm function, which can speed up the subsequent log matching process.

[0111] Step S40: Receive the log file to be parsed;

[0112] A log file is a text file, typically containing many lines, each carrying a log message. Different devices create log messages in different formats. These log messages are difficult to interpret directly and require parsing to extract useful information. Log files containing log messages can be obtained in several ways, including reading them directly from a disk or transmitting them over a network.

[0113] Step S60: Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0114] Based on the Lua program interface, a log parsing function is selected from the Lua program's function call library to parse the log file and obtain the parsing result. This parsing result can be easily obtained by the main program written in C / C++, realizing the purpose of embedding the log parsing function as a subroutine into the main program.

[0115] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0116] Step S20: Perform resource pre-configuration based on the Lua program interface to obtain the function call library;

[0117] The pre-configured function call library includes the liblognorm function, which can speed up the subsequent log matching process.

[0118] Step S40: Receive the log file to be parsed;

[0119] A log file is a text file, typically containing many lines, each carrying a log message. Different devices create log messages in different formats. These log messages are difficult to interpret directly and require parsing to extract useful information. Log files containing log messages can be obtained in several ways, including reading them directly from a disk or transmitting them over a network.

[0120] Step S60: Select a log parsing function from the function call library, call the log parsing function to parse the log file, and obtain the log parsing result.

[0121] Based on the Lua program interface, a log parsing function is selected from the Lua program's function call library to parse the log file and obtain the parsing result. This parsing result can be easily obtained by the main program written in C / C++, realizing the purpose of embedding the log parsing function as a subroutine into the main program.

[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A log processing method for Lua program interface, characterized in that, The log processing method comprises: The resource pre-configuration operation based on the lua program interface obtains a function call library; the function call library comprises a liblognorm function call library; The resource pre-configuration operation comprises: The liblognorm function call library is declared in the configuration table of the lua; An ln_loadSamplesFromString function in the liblognorm function call library is mapped as a loading function; the ln_loadSamplesFromString function is used to construct a directed acyclic graph parsing tree; An ln_normalize function in the liblognorm function call library is mapped as a parsing function; the ln_normalize function is used to perform log matching; A log file to be parsed is received; A log parsing function is selected from the function call library, the log parsing function is called to parse the log file, and a log parsing result is obtained.

2. The log processing method for lua program interface according to claim 1, wherein, The log file to be parsed is received, which comprises: The log version information is checked at a time; If the log version information is updated, the log file to be parsed is actively received.

3. The log processing method for Lua program interface according to claim 1, wherein, The log parsing function is selected from the function call library, the log parsing function is called to parse the log file, and a log parsing result is obtained, which comprises: A log message and a parsing identifier are extracted from the log file; The corresponding log parsing function is selected from the function call library according to the parsing identifier; The log message is parsed according to the log parsing function, and a log parsing result is obtained.

4. The log processing method for lua program interface according to claim 1, wherein, The log processing method further comprises: A target field is extracted from the log parsing result; It is judged whether the parsing is correctly completed according to the target field.

5. The log processing method for lua program interface according to claim 4, wherein, The target field is compared with a preset field template; If the target field and the preset field template have a consistent field, it is judged that the parsing has been correctly completed. The log processing device comprises:

6. A log processing apparatus for a lua program interface, characterized by, A function call library generation module is configured to perform resource pre-configuration operation based on a lua program interface to obtain a function call library; the function call library comprises a liblognorm function call library; The resource pre-configuration operation comprises: The liblognorm function call library is declared in the configuration table of the lua; An ln_loadSamplesFromString function in the liblognorm function call library is mapped as a loading function; the ln_loadSamplesFromString function is used to construct a directed acyclic graph parsing tree; An ln_normalize function in the liblognorm function call library is mapped as a parsing function; the ln_normalize function is used to perform log matching; A log receiving module is configured to receive a log file to be parsed; ​ A log analysis module is configured to select a log analysis function from the function call library, invoke the log analysis function to analyze the log file, and obtain a log analysis result. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The computer program, when executed by the processor, implements the steps of the log processing method for a lua program interface according to any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the log processing method for a lua program interface according to any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the log processing method for a lua program interface according to any one of claims 1 to 5. The computer program, when executed by the processor, implements the steps of the log processing method for a lua program interface according to any one of claims 1 to 5.

Images