EVTX format system log query method and system based on byte stream

Abstract

The present application discloses a system log query method and system in EVTX format based on byte stream, which mainly relates to the field of log query technology, and is used to solve the problem that when operating log files, the interface provided by Microsoft on the Windows platform does not support additional files and cross-Linux platforms. In addition, the use of third-party tools to operate log files requires secondary development based on the third-party tools, and the process is relatively cumbersome. It includes: determining the open file address or disk device address based on the data source; obtaining the platform type corresponding to the parsed data, and registering the runtime library corresponding to the system type; opening the parsed data, sequentially reading the byte stream corresponding to the parsed data, determining that the byte stream with the identification header of the evtx file is successfully matched, calling the evtx file storage format parsing program, and parsing and obtaining several log files from the successfully matched byte stream; and returning several log files to the user end through the callback function.

Description

Technical Field

[0001] The present application relates to the technical field of log query, and in particular to a method and system for querying system logs in EVTX format based on byte stream. Background Art

[0002] The evtx file is a new log file format adopted by Microsoft starting from Windows NT 6.0. Based on these logs, the operation dynamics of the target host can be understood. The logs specifically include system logs, application logs and security logs. They are created by the Windows Event Viewer, contain a list of events recorded by Windows, and are saved in a proprietary XML format.

[0003] Since evtx is a log file storage format customized by Microsoft, if software developers want to read local evtx format files, they can only call the API provided by Microsoft on the Windows platform to read the operating system's operation log. The API provided by Microsoft can only read evtx format files generated by the current host, and cannot read evtx format files copied from other hosts. In addition, since it is customized for Windows, there is no effective API for evtx file parsing on the Linux platform. In addition, third-party tools can be used to perform preliminary analysis of system operation logs, but software developers cannot directly operate the log files themselves. If they want to customize a set of system operation log analysis software, they need to conduct secondary development based on third-party tools.

[0004] Therefore, there is an urgent need for a system log query method and system in EVTX format based on byte stream to solve the problem that when operating log files, calling the API provided by Microsoft on the Windows platform only supports log files generated by the current system itself, does not support additional files and cross-Linux platforms. In addition, when using third-party tools to operate log files, secondary development is required based on the third-party tools, which makes the process more cumbersome. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present application provides a system log query method and system in the EVTX format based on a byte stream to solve the problem that when operating log files, calling the API provided by Microsoft on the Windows platform only supports log files generated by the current system itself, does not support additional files and cross-Linux platforms. In addition, when using third-party tools to operate log files, secondary development is required based on the third-party tools, which makes the process more cumbersome.

[0006] In a first aspect, the present application provides an EVTX format system log query method based on a byte stream, the method comprising: obtaining parsed data through a preset externally exposed API; wherein the parsed data includes additional files and cross-platform files that are not automatically generated; obtaining the platform type corresponding to the parsed data, so that the preset externally exposed API registers the runtime library corresponding to the system type; wherein the platform type is at least divided into: windows platform, linux platform, the runtime library is divided into dll runtime library and so runtime library, and the runtime library at least includes an execution program and a log detail callback; through the execution program, the parsed data is opened, the byte stream corresponding to the parsed data is sequentially read, and then it is determined in turn whether the read byte stream has an identification header of the evtx file; it is determined that the byte stream with the identification header of the evtx file is successfully matched, the evtx file storage format parsing program is called, and a number of log files are obtained by parsing the successfully matched byte stream; through the callback function, the number of log files are returned to the user end.

[0007] Furthermore, the parsed data is obtained through a preset externally exposed API, specifically including: obtaining the full path of the non-automatically generated additional file source or the disk number of the cross-platform file source through the externally exposed API.

[0008] Furthermore, the parsed data is opened and the byte stream corresponding to the parsed data is sequentially read, specifically including: when the parsed data is an additional file that is not automatically generated, the byte stream is read in the writing order of the additional file that is not automatically generated; when the parsed data is a cross-platform file, the disk number corresponding to the cross-platform file is opened, and the byte stream is read in the storage order.

[0009] Furthermore, the runtime library also includes a status information callback; before opening the parsed data by executing the program, the method also includes: when an error or parsing failure occurs, and the parsed data cannot be opened, the information that the parsed data cannot be opened is reported to the user through the pre-registered status information callback.

[0010] Furthermore, the so runtime library also includes a progress callback; when sequentially reading the byte stream corresponding to the parsed data, the method also includes: reporting the current progress of reading the parsed data to the user in real time through the pre-registered progress callback.

[0011] Furthermore, the so runtime library includes a file write-back callback. After calling the evtx file storage format parser and parsing to obtain several log files from the successfully matched byte stream, the method also includes: writing the several log files back to a preset directory, triggering the pre-registered file write-back callback, and reporting the absolute path and writing status of the currently written-back several log files to the user.

[0012] Furthermore, the method also includes: reporting the running result to the user in an asynchronous form to inform the user of the current running result; wherein the running result is divided into parsing success or parsing failure.

[0013] In the second aspect, the present application provides an EVTX format system log query system based on byte stream, and the system includes: a data determination module, which is used to obtain parsed data through a preset API exposed to the outside; wherein the parsed data includes non-automatically generated additional files and cross-platform files; an externally exposed header file module, which is used to obtain the platform type corresponding to the parsed data, so that the preset externally exposed API registers the runtime library corresponding to the system type; wherein the platform type is at least divided into: windows platform, linux platform, the runtime library is divided into dll runtime library and so runtime library, and the runtime library at least includes an execution program and a log detail callback; the runtime library, and the runtime library includes: dll runtime library and so runtime library, which is used to open the parsed data through the execution program, sequentially read the byte stream corresponding to the parsed data, and then determine in turn whether the read byte stream has an identification header of the evtx file; determine that the byte stream with the identification header of the evtx file is successfully matched, call the evtx file storage format parsing program, and parse and obtain several log files from the successfully matched byte stream; through the callback function, return the several log files to the user end.

[0014] Furthermore, the so runtime library also includes a progress callback unit; which is used to report the current progress of reading and parsing data to the user in real time through the pre-registered progress callback.

[0015] Furthermore, the so runtime library includes a file write-back callback unit, which is used to write back several log files to a preset directory and trigger a pre-registered file write-back callback to report the absolute path and writing status of the current several log files to the user.

[0016] Those skilled in the art can understand that the present application has at least the following beneficial effects:

[0017] The present application proposes a system log query method and system based on byte stream in EVTX format, which obtains parsed data through a preset externally exposed API, and realizes reading parsed data of non-automatically generated additional file sources and cross-platform file sources; by obtaining the platform type corresponding to the parsed data, the preset externally exposed API registers the runtime library corresponding to the system type, where the runtime library is divided into dll runtime library (supporting Windows platform file parsing) and so runtime library (supporting Linux platform file parsing), and uses the execution program and log detailed callback built into the runtime library to realize opening the parsed data (extra file or cross-platform file) by running the corresponding runtime library, and sequentially reading the byte stream corresponding to the parsed data. After the identification header of the evtx file is successfully matched, several log files are parsed from the successfully matched byte stream, and the log files corresponding to the additional files or cross-platform files are effectively obtained. The problem that when operating log files, calling the API provided by Microsoft on the Windows platform only supports the log files generated by the current system itself, and does not support additional files and cross-linux platforms. In addition, the present application does not involve the use of third-party tools, which avoids the need for secondary development based on third-party tools when operating log files using third-party tools, and the problem of a relatively cumbersome process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:

[0019] Figure 1 This is a flow chart of a system log query method in EVTX format based on byte stream provided in an embodiment of the present application.

[0020] Figure 2 It is a schematic diagram of the overall structure of the evtx file provided in the embodiment of the present application.

[0021] Figure 3 It is a schematic diagram of the internal structure of an EVTX format system log query system based on byte stream provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.

[0023] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0024] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0025] The present application embodiment provides a system log query method in EVTX format based on byte stream, such as Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps:

[0026] Step 110: Obtain parsed data through a preset externally exposed API.

[0027] It should be noted that the parsed data includes additional files and cross-platform files that are not automatically generated. The parsed data is data containing evtx format files.

[0028] This step can be specifically as follows:

[0029] Through the exposed API, get the full path of the non-automatically generated additional file source or the disk number of the cross-platform file source.

[0030] Step 120: Obtain the platform type corresponding to the parsed data, and register the runtime library corresponding to the system type of the preset externally exposed API.

[0031] It should be noted that the platform types are at least divided into: Windows platform and Linux platform, and the runtime library is divided into dll (Dynamic Link Library) runtime library and so (Shared Object) runtime library, and the runtime library at least includes the execution program and log detailed callback. The dll runtime library supports Windows platform file parsing, and the so runtime library supports Linux platform file parsing.

[0032] The specific method for obtaining the platform type corresponding to the parsed data can be obtained through external input.

[0033] Step 130, by executing the program, opening the parsed data, sequentially reading the byte stream corresponding to the parsed data, and then determining in turn whether the read byte stream has the identification header of the evtx file; determining that the byte stream with the identification header of the evtx file is successfully matched, calling the evtx file storage format parsing program, parsing from the successfully matched byte stream to obtain several log files; and returning the several log files to the user end through the callback function.

[0034] It should be noted that the structure of the evtx format file is as follows Figure 2 As shown, it contains an evtx file header, and the size of the evtx file header is fixed at 4K (4096 bytes). The first 8 bytes "ElfFile\x00" are the identification header of the evtx file. By comparing the byte stream offset in the parsed data with the identification header, it is determined whether the current file is an evtx format file.

[0035] Among them, the parsed data is opened and the byte stream corresponding to the parsed data is read sequentially, which can be specifically:

[0036] When the parsed data is an additional file that is not automatically generated, the byte stream is read in the order in which the additional file is written; when the parsed data is a cross-platform file, the disk number corresponding to the cross-platform file is opened and the byte stream is read in the order in which it is stored.

[0037] The specific process of parsing and obtaining a number of log files from the successfully matched byte stream may be: a person skilled in the art may parse and obtain the log files according to the evtx file storage format standard.

[0038] For example, the file header size of evtx is fixed at 4K (4096 bytes). The first 8 bytes "ElfFile\x00" are the identification header of the evtx file. The byte stream offset process is to compare with the identification header to determine whether the current file is an evtx format file. The offset 0x2A bit records the block data contained in the current file. Immediately after the file header, the block needs to be further parsed. The structure of the evtx format file is as follows: Figure 2 As shown in the figure, the evtx format file contains an identification header and several evtx data blocks (evtx blocks). The evtx data block consists of a block header and several logs. The fixed size of each block is 64KB (65536). The excess logs are saved to the next block. "ElfChnk\x00" is the block identification bit. During the byte stream offset process, the logs in each evtx block are read in sequence according to the block identification bit.

[0039] In addition, this application can also report status information.

[0040] The specific process is:

[0041] The runtime library also includes a status information callback; before the program is executed and the parsed data is opened, when an error or a parsing failure occurs and the parsed data cannot be opened, the pre-registered status information callback is used to report to the user the information that the parsed data cannot be opened.

[0042] It should be noted that the status information callback is an existing status information callback mechanism, which is directly used in this application without involving any modification of the underlying logic.

[0043] In addition, in order to facilitate users to obtain the reading progress of disk data, the so runtime library of this application also includes a progress callback; when sequentially reading the byte stream corresponding to the parsed data, the progress of the current reading of the parsed data is reported to the user in real time through the pre-registered progress callback.

[0044] It should be noted that the progress callback is an existing progress callback mechanism, which is used directly in this application without modifying the underlying logic. In addition, due to the large amount of data on the disk, progress reporting is supported at specified time intervals.

[0045] In addition, if the user sets the need to write back the retrieved evtx file, the background will write back the evtx format file that is hit and parsed correctly to the directory specified by the user, and trigger the write-back callback registered by the user, and report the absolute path and writing status of the current write-back file to the user.

[0046] Specifically, the so runtime library includes a file write-back callback. After calling the evtx file storage format parser and parsing to obtain several log files from the successfully matched byte stream, it writes the several log files back to the preset directory and triggers the pre-registered file write-back callback to report the absolute path and writing status of the currently written log files to the user.

[0047] It should be noted that the file write-back callback is an existing file write-back callback mechanism, which is directly used in this application without involving any modification of the underlying logic.

[0048] In addition, the method further includes: reporting the operation result to the user in an asynchronous form to inform the user of the current operation result; wherein the operation result is divided into parsing success or parsing failure.

[0049] Based on the above description, this solution supports two ways to check evtx format files. One is based on files, extracting the complete data of the file in the form of byte streams, without distinguishing file types, and supporting cross-system parsing. You only need to set the complete path of the data to be parsed through the API preset by this application. After the parsing is completed, if the incoming file matches the file format of evtx, the extracted log details can be reported to the user for display and analysis. Another way is to directly specify the disk number, set the disk number to be retrieved through the API exposed to the outside, and open the corresponding disk for deep traversal retrieval after getting the disk number specified by the user. In this way, even if the corresponding log file on the disk is deleted or damaged, the information can be extracted as long as the corresponding storage block is not overwritten. At the same time, if the user needs, the evtx file information extracted from the disk can be restored, saved as an evtx format file and placed in the directory specified by the user for the user to review and conduct subsequent problem analysis and positioning.

[0050] besides, Figure 3 An EVTX format system log query system based on byte stream is provided in the embodiment of the present application. Figure 3 As shown, the system provided in the embodiment of the present application mainly includes:

[0051] The data determination module 210 is used to obtain parsed data through a preset API exposed to the outside.

[0052] The parsed data includes additional files that are not automatically generated and cross-platform files.

[0053] The header file module 220 exposed to the outside is used to obtain the platform type corresponding to the parsed data, so as to enable the runtime library 230 corresponding to the preset API registration system type exposed to the outside.

[0054] The platform types are at least divided into: windows platform and linux platform, the runtime library 230 is divided into dll runtime library and so runtime library, and the runtime library 230 at least includes an execution program and a log detail callback.

[0055] The runtime library 230 includes: a dll runtime library and a so runtime library, which is used to open the parsed data by executing a program, sequentially read the byte stream corresponding to the parsed data, and then determine in turn whether the read byte stream has an identification header of the evtx file; determine that the byte stream with the identification header of the evtx file is successfully matched, call the evtx file storage format parsing program, and parse the successfully matched byte stream to obtain a number of log files; and return the number of log files to the user end through a callback function.

[0056] The so runtime library also includes a progress callback unit, which is used to report the current progress of reading and parsing data to the user in real time through the pre-registered progress callback.

[0057] The so runtime library includes a file write-back callback unit, which is used to write back several log files to a preset directory and trigger the pre-registered file write-back callback to report the absolute path and writing status of the current log files to the user.

[0058] It should be noted that the runtime library 230 is dynamically loaded when the executable program to be applied is running, and there is no need to pay attention to the details of parsing the evtx format log. During the operation, the byte stream is read out from the address or disk device in the order of data writing, and the read byte stream is matched with the identification header of the evtx file. If it does not match, continue to read the next byte. If it matches, enter the parsing process. The parsing basis comes from the official definition of the evtx file storage format standard.

[0059] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.

Claims

1. A byte stream-based EVTX format system log query method, characterized in that: The method comprises: Obtain parsed data through a preset externally exposed API; the parsed data includes additional files and cross-platform files that are not automatically generated; Specifically, it includes: obtaining the full path of the non-automatically generated additional file source or the disk number of the cross-platform file source through the exposed API; Obtain the platform type corresponding to the parsed data, so that the preset externally exposed API registers the runtime library corresponding to the system type; wherein the platform type is at least divided into: Windows platform, Linux platform, the runtime library is divided into dll runtime library and so runtime library, and the runtime library at least includes an execution program and a log detailed callback; By executing the program, the parsed data is opened, the byte stream corresponding to the parsed data is read sequentially, and then it is determined in turn whether the read byte stream has the identification header of the evtx file; it is determined that the byte stream with the identification header of the evtx file is successfully matched, and the evtx file storage format parsing program is called to parse the successfully matched byte stream to obtain several log files; through the callback function, the several log files are returned to the user end.

2. The EVTX format system log query method based on byte stream according to claim 1, characterized in that: Open the parsed data and read the byte stream corresponding to the parsed data in sequence, including: When the parsed data is an additional file that is not automatically generated, the byte stream is read in the order in which the additional file is written; When the parsed data is a cross-platform file, open the disk number corresponding to the cross-platform file and read the byte stream in the order in which it is stored.

3. The EVTX format system log query method based on byte stream according to claim 1, characterized in that: The runtime library also includes status information callbacks; Before opening and parsing the data by executing the program, the method further includes: When an error or parsing failure occurs, and the parsed data cannot be opened, the pre-registered status information callback is used to report the failure to open the parsed data to the user.

4. The EVTX format system log query method based on byte stream according to claim 1, characterized in that: The so runtime library also includes progress callbacks; When sequentially reading the byte stream corresponding to the parsed data, the method further includes: The progress of reading and parsing data is reported to the user in real time through the pre-registered progress callback.

5. The EVTX format system log query method based on byte stream according to claim 1, characterized in that: so runtime library includes file writeback callbacks, After calling the evtx file storage format parsing program and parsing the successfully matched byte stream to obtain a number of log files, the method further includes: Writes back several log files to the preset directory and triggers the pre-registered file write-back callback to report the absolute path and writing status of the currently written log files to the user.

6. The EVTX format system log query method based on byte stream according to claim 1, characterized in that: The method further comprises: Report the running results to the user in an asynchronous manner to inform the user of the current running results; The operation result is divided into parsing success or parsing failure.

7. A byte stream-based EVTX format system log query system, characterized in that: The system comprises: The data determination module is used to obtain the parsed data through a preset externally exposed API; wherein the parsed data includes non-automatically generated additional files and cross-platform files; specifically including: obtaining the full path of the non-automatically generated additional file source or the disk number of the cross-platform file source through the externally exposed API; The header file module exposed to the outside is used to obtain the platform type corresponding to the parsed data, so that the runtime library corresponding to the preset externally exposed API registration system type is used; wherein the platform type is at least divided into: Windows platform, Linux platform, the runtime library is divided into dll runtime library and so runtime library, and the runtime library at least includes an execution program and a log detailed callback; The runtime library includes: a dll runtime library and a so runtime library, which is used to open and parse data by executing a program, sequentially read the byte stream corresponding to the parsed data, and then determine in turn whether the read byte stream has an identification header of the evtx file; determine that the byte stream with the identification header of the evtx file is successfully matched, call the evtx file storage format parsing program, and parse the successfully matched byte stream to obtain several log files; and return the several log files to the user end through a callback function.

8. The EVTX format system log query system based on byte stream according to claim 7, characterized in that: The so runtime library also includes a progress callback unit; Used to report the current progress of reading and parsing data to the user in real time through the pre-registered progress callback.

9. The EVTX format system log query system based on byte stream according to claim 7, characterized in that: so runtime library includes a file write-back callback unit, Used to write back several log files to the preset directory and trigger the pre-registered file write-back callback to report the absolute path and writing status of the current log files to the user.

Images