A log auditing method, device, medium and product

Abstract

This application relates to the technical field of network security, and in particular, to a log auditing method, device, medium, and product. The method includes: obtaining a log set, and based on the log set, performing log type division to obtain a log subset corresponding to each log type. Then, obtaining the log format corresponding to each log type, and according to the log format corresponding to each log type, unifying the format of each log subset to obtain a formatted log subset corresponding to each log subset. Finally, performing log parsing on each formatted log subset to obtain a log auditing result. Unifying log files of the same type into the same format facilitates subsequent parsing of the log files after format unification, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

Description

Technical Field

[0001] This application relates to the technical field of network security, and in particular, to a log auditing method, device, medium, and product. Background Art

[0002] With the rapid development of information technology and the deepening of enterprise informatization construction, various information systems and applications play an increasingly important role in enterprise operations. A large amount of log data is generated during the operation of these systems and applications, recording important information such as user operations, system status, and abnormal events. Log data is of great significance for enterprise security auditing, fault troubleshooting, performance optimization, etc.

[0003] However, due to the different log formats of different systems and applications and the lack of unified standards and specifications, inconsistent log formats may cause some key security information in the logs to be hidden or ignored, increasing the risk of security vulnerabilities. At the same time, processing log data in different formats increases the complexity of data parsing, which may lead to errors in log data parsing or omission of important information.

[0004] Therefore, how to solve the above technical problems is an urgent problem for those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a log auditing method, device, medium, and product to solve at least one of the above technical problems.

[0006] The above invention purpose of this application is achieved through the following technical solutions:

[0007] In a first aspect, this application provides a log auditing method, adopting the following technical solution:

[0008] A log auditing method includes:

[0009] Obtain a log set, and based on the log set, perform log type division to obtain a log subset corresponding to each log type, where the log set includes: multi-type log data respectively corresponding to multiple systems;

[0010] Obtain the log format corresponding to each log type, and according to the log format corresponding to each log type, unify the format of each log subset to obtain a formatted log subset corresponding to each log subset;

[0011] Perform log parsing on each formatted log subset to obtain a log auditing result.

[0012] By adopting the above technical solution, a log set is obtained, and log type division is performed based on the log set to obtain a log subset corresponding to each log type. Then, the log format corresponding to each log type is obtained, and according to the log format corresponding to each log type, the format of each log subset is unified to obtain a formatted log subset corresponding to each log subset. Finally, log parsing is performed on each formatted log subset to obtain a log audit result. Unifying log files of the same type into the same format facilitates subsequent parsing of the unified-format log files, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

[0013] In a preferred example, the present application can be further configured as follows: The step of unifying the format of each log subset according to the log format corresponding to each log type to obtain a formatted log subset corresponding to each log subset includes:

[0014] Performing format key analysis based on a target log format to determine format key information, where the format key information includes: log key fields, field position relationships, and log audit periods, and the target log format is any one of the log formats;

[0015] Extracting and arranging fields from the target log subset corresponding to the target log format according to the format key information to obtain a target formatted log subset, where the target formatted log subset is any one of the formatted log subsets.

[0016] In a preferred example, the present application can be further configured as follows: The step of performing log parsing on each formatted log subset to obtain a log audit result includes:

[0017] Obtaining distributed framework information, where the distributed framework information includes: node names corresponding to the cluster, the number of nodes, and memory distribution;

[0018] Based on the number of nodes in the distributed framework information, each formatted log subset is divided into blocks to obtain a log data block corresponding to each node name;

[0019] Controlling each node in the distributed framework to perform log parsing on the log data blocks in parallel to obtain a log audit result.

[0020] In a preferred example, the present application can be further configured as follows: After performing log parsing on each formatted log subset to obtain a log audit result, it further includes:

[0021] When the log audit result includes log anomalies, then based on the log audit result and the formatted log subset, anomaly information analysis is performed to obtain anomaly operators, anomaly operation devices, and anomaly operation contents;

[0022] Combining the abnormal records based on the abnormal operator, the abnormal operation device, and the abnormal operation content to obtain abnormal records indexed by the abnormal operator;

[0023] Analyzing measures based on the abnormal records to obtain emergency measures.

[0024] In a preferred example, the present application can be further configured as: after analyzing measures based on the abnormal records to obtain emergency measures, it further includes:

[0025] Performing first visual analysis based on the log audit result to obtain an overall audit chart;

[0026] Performing second visual analysis based on the abnormal records indexed by the abnormal operator to obtain an abnormal personnel audit chart;

[0027] Obtaining a first abnormal record and a second abnormal record before and after implementing the emergency measure, and performing third visual analysis based on the first abnormal record and the second abnormal record to obtain a measure effect audit chart, where the first abnormal record is the abnormal record indexed by the abnormal operator before implementing the emergency measure, and the second abnormal record is the abnormal record indexed by the abnormal operator after implementing the emergency measure.

[0028] In a preferred example, the present application can be further configured as: after obtaining the log set, it further includes:

[0029] Performing multi-dimensional verification based on the log data in the log set, where the multi-dimensional verification includes: data integrity verification, timestamp verification, and data source verification;

[0030] When the multi-dimensional verification fails, a log data anomaly warning is generated; when the multi-dimensional verification is successful, the log set is uploaded to the blockchain for storage to prevent unauthorized personnel from tampering with and deleting the log data.

[0031] In a second aspect, the present application provides an electronic device, adopting the following technical solution:

[0032] At least one processor;

[0033] A memory;

[0034] At least one application program, where at least one application program is stored in the memory and is configured to be executed by at least one processor, and the at least one application program is configured to: execute the above-mentioned log audit method.

[0035] In a third aspect, the present application provides a computer-readable storage medium, adopting the following technical solution:

[0036] A computer-readable storage medium stores a computer program thereon. When the computer program is executed on a computer, the computer is caused to execute the log auditing method described above.

[0037] In a fourth aspect, the present application provides a computer program product, adopting the following technical solution:

[0038] A computer program product includes a computer program. When the computer program is executed by a processor, the above log auditing method is implemented.

[0039] In summary, the present application includes at least one of the following beneficial technical effects:

[0040] Obtain a log set, and based on the log set, perform log type division to obtain a log subset corresponding to each log type. Then, obtain the log format corresponding to each log type, and according to the log format corresponding to each log type, unify the format of each log subset to obtain a formatted log subset corresponding to each log subset. Finally, perform log parsing on each formatted log subset to obtain a log auditing result. Unifying log files of the same type into the same format facilitates subsequent parsing of the unified format log files, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

[0041] Obtain distributed framework information, and based on the number of nodes in the distributed framework information, divide each formatted log subset into blocks to obtain a log data block corresponding to each node name. Then, control each node in the distributed framework to perform log parsing on the log data block in parallel to obtain a log auditing result. When facing a large number of log files that need to perform log parsing operations, utilize the computing power of multiple nodes in the distributed framework to significantly reduce the overall time of log parsing and improve the processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flowchart of a log auditing method according to an embodiment of the present application;

[0043] Figure 2 is a structural diagram of a log auditing device according to an embodiment of the present application;

[0044] Figure 3 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following is a further detailed description of the present application in conjunction with Figures 1 to 3 to further illustrate the present application.

[0046] This specific embodiment is only an interpretation of the present application and does not limit the present application. After reading this specification, those skilled in the art can make modifications to this embodiment without creative contributions as needed, but as long as it is within the scope of the present application, it is protected by the patent law.

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without making creative efforts belong to the scope of protection of the present application.

[0048] In addition, the term "and / or" in this document is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0049] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings of the specification.

[0050] The embodiment of the present application provides a log auditing method, which is executed by an electronic device. The electronic device can be a server or a terminal device. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods. The embodiment of the present application does not make any restrictions here, as Figure 1 shown, the method includes step S101, step S102, and step S103, where:

[0051] Step S101: Obtain a log set, and perform log type division based on the log set to obtain a log subset corresponding to each log type. Among them, the log set includes: multi-type log data respectively corresponding to multiple systems.

[0052] For the embodiments of the present application, the log set is composed of multiple types of log files corresponding to each system. That is, the electronic device obtains the log data of each system through wireless transmission. Since there are multiple types of logs, and the information and data formats concerned by different types of log data are different, therefore, based on the log set, the log types are divided to obtain a log subset corresponding to each log type. Among them, the log subset is composed of log data of the same log type. The log types include but are not limited to: system logs, application logs, security logs, etc. Of course, other types of log types can also be included, such as network logs, device logs, etc. For the processing method of log type division, the embodiments of the present application do not make any further limitations. In one feasible way, by checking the file name of the log file in the log set or the specific identifier in the log data, the log files are classified according to the identifier. For example, when the log file includes security-related identifiers such as "auth" and "access denied", the log type is determined to be a security log; the log including the keyword "system" in the log file can be classified as a system log, and the log file including the keyword "app" is classified as an application log.

[0053] Step S102: Obtain the log format corresponding to each log type, and unify the format of each log subset according to the log format corresponding to each log type to obtain a formatted log subset corresponding to each log subset.

[0054] For the embodiments of the present application, corresponding log types are set for the log files of each log type in the electronic device, so as to unify the log files of the same type into the same format, which is convenient for subsequent parsing of the log files with the unified format, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

[0055] Specifically, obtain the log format corresponding to each log type. The log format details the log keyword fields, field position relationships, and log audit periods corresponding to each log format. Of course, other relevant information can also be included in the log format, and the embodiments of the present application do not make any further limitations. Then, unify the format of each log subset according to the log format corresponding to each log type to obtain a formatted log subset corresponding to each log subset.

[0056] Step S103: Perform log parsing on each formatted log subset to obtain a log audit result.

[0057] For the embodiments of the present application, log parsing is uniformly performed on log files of the same log type in the formatted log subsets, so as to obtain the log audit results of each formatted log subset and the overall log audit results corresponding to the log set. For the process of log parsing, event types that require key attention are screened according to security policies, and event classification is performed based on the field information in the log files after unified formatting. For example, error events, warning events, normal events, etc. Then, machine learning and statistical analysis methods are used to perform pattern recognition on error events and warning events to discover potential anomalies or violations, so as to obtain the audit results corresponding to the log file. Of course, in the case of a large number of log files, single-threaded log parsing cannot meet the requirements of high-speed log parsing. Therefore, using multi-threaded or distributed processing methods for log parsing can improve the speed of log parsing.

[0058] It can be seen that in the embodiments of the present application, a log set is obtained, and log type division is performed based on the log set to obtain log subsets corresponding to each log type. Then, the log format corresponding to each log type is obtained, and according to the log format corresponding to each log type, the format of each log subset is unified to obtain the formatted log subset corresponding to each log subset. Finally, log parsing is performed on each formatted log subset to obtain the log audit results. Unifying log files of the same type into the same format facilitates subsequent parsing of the log files after unified formatting, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

[0059] Further, in order to simplify the complexity of subsequent log parsing and improve the parsing efficiency, in the embodiments of the present application, according to the log format corresponding to each log type, the format of each log subset is unified to obtain the formatted log subset corresponding to each log subset, including:

[0060] Perform format key analysis based on the target log format to determine format key information, where the format key information includes: log key fields, field position relationships, and log audit periods, and the target log format is any one of the log formats;

[0061] Extract and arrange fields from the target log subset corresponding to the target log format according to the format key information to obtain the target formatted log subset, where the target formatted log subset is any one of the formatted log subsets.

[0062] For the embodiments of the present application, by unifying the formats of log subsets from different sources and different formats, log data of the same log type follows the same structure and specifications, simplifies the complexity of subsequent log parsing, and improves the parsing efficiency.

[0063] Specifically, based on the target log format, perform format key analysis to determine format key information, where the format key information includes: log key fields, field position relationships, and log audit cycles, and the target log format is any one of the log formats. For example, for the log format corresponding to system logs, the log key fields include: timestamp (used to record the time when a system event occurs), event type (identifying the nature of the system event, such as start, shutdown, error), event description (detailed description of the system event, including involved components, error codes, etc.), system component (identifying the system component or module where the event occurs), user information (information about the user or process performing the operation); the field position relationship is: the timestamp is usually at the front of the log entry as the primary information, the event type follows the timestamp for quick event classification, the event description is in a later position to provide detailed context, and the system component and user information are arranged in sequence after the event description. For the log format corresponding to application logs, the log key fields include: event timestamp (recording the time when an application event occurs), application module (identifying the application module or function to which the event belongs), event level (such as INFO, WARN, ERROR, etc., indicating the severity of the event), request details (such as HTTP request information, operation parameters, etc.), response result (the result or return information after the operation is executed); the field position relationship is: the timestamp is at the beginning of the log entry, the application module and event level follow immediately, for quickly locating problems, and the request details and response result are arranged in sequence after the event level. For the log format corresponding to security logs, the log key fields include: timestamp (recording the time when a security event occurs), user identity (the identity of the user or entity performing the operation), event type (identifying the nature of the security event, such as login, access control, etc.), source and target (recording the source address and target address where the event occurs), security policy (used to mark the involved security policy or rule); the field position relationship is: the timestamp and security event type are usually in prominent positions in the log entry, and key information such as user identity, source and target addresses follows immediately, and the security policy and event description are in a later position to provide a basis for detailed analysis. The log audit cycle is preset by technical personnel and stored in the electronic device, and users can also set it according to their needs. In this regard, the embodiments of the present application will not be further limited. Furthermore, according to the format key information, perform field extraction and arrangement on each log file in the target log subset corresponding to the target log format to obtain the target formatted log subset.

[0064] It can be seen that in the embodiments of the present application, format key analysis is performed based on the target log format to determine format key information, and then, according to the format key information, field extraction and arrangement are performed on the target log subset corresponding to the target log format to obtain the target formatted log subset. By unifying the formats of log subsets from different sources and different formats, log data of the same log type follows the same structure and specification, simplifying the complexity of subsequent log parsing and improving the parsing efficiency.

[0065] Furthermore, in order to reduce the overall time of log parsing and improve the processing efficiency, in the embodiments of the present application, log parsing is performed on each formatted log subset to obtain log audit results, including:

[0066] Obtain distributed framework information, where the distributed framework information includes: the node name, the number of nodes, and the memory distribution corresponding to the cluster;

[0067] Based on the number of nodes in the distributed framework information, each formatted log subset is chunked to obtain a log data chunk corresponding to each node name;

[0068] Control each node in the distributed framework to perform log parsing on the log data chunks in parallel to obtain log audit results.

[0069] For the embodiments of the present application, when facing a large number of log files that require log parsing operations, the computing capabilities of multiple nodes in the distributed framework are utilized to significantly reduce the overall time of log parsing and improve the processing efficiency. Therefore, the electronic device is connected to the management interface of the distributed framework to collect detailed information about the cluster, and extract the node name of each node in the cluster, the number of nodes in the distributed framework, and the memory distribution of each node. Then, based on the number of nodes in the distributed framework information, all formatted log subsets are chunked according to the data volume to obtain multiple log data chunks, where the number of log data chunks is the same as the number of nodes. At the same time, after the chunking operation, it is also necessary to evaluate the size of the log data chunks, that is, a data chunk size threshold is preset in the electronic device. When the log data chunk is smaller than the data chunk size threshold, each node in the distributed framework is controlled to perform log parsing on the log data chunk in parallel; when the log data chunk is larger than the data chunk size threshold, the size of the log data chunk is adjusted so that the size of the log data chunk meets the requirements. Of course, when dividing the size of the log data chunks, the memory and computing capabilities of the nodes can also be considered. Furthermore, using the task scheduling mechanism of the distributed framework, the log data chunks are distributed to the corresponding nodes, and each node in the distributed framework is controlled to perform log parsing on the log data chunk in parallel. The node uses its own computing resources to efficiently process the log data. Finally, after each node completes the log parsing, the generated log audit results are returned to the aggregation node, which is convenient for the aggregation node to collect the audit results corresponding to all nodes to obtain the log audit results corresponding to all log files. Preferably, during the process of controlling each node to perform log parsing on the log data chunk in parallel, the number of threads or the load of the computing nodes can also be dynamically adjusted according to the complexity of the parsing task and the situation of the computing resources; ensuring that the parsing process can make full use of the available resources while avoiding resource waste.

[0070] It can be seen that in the embodiments of the present application, the distributed framework information is obtained, and based on the number of nodes in the distributed framework information, each formatted log subset is chunked to obtain a log data chunk corresponding to each node name. Then, each node in the distributed framework is controlled to perform log parsing on the log data chunk in parallel to obtain the log audit results. When facing a large number of log files that require log parsing operations, the computing capabilities of multiple nodes in the distributed framework are utilized to significantly reduce the overall time of log parsing and improve the processing efficiency.

[0071] Furthermore, in order to enhance the security of the system to ensure the continuity and normal operation of the business, in the embodiments of the present application, after performing log parsing on each formatted log subset to obtain the log audit results, it further includes:

[0072] When the log audit result includes log anomalies, anomaly information analysis is performed based on the log audit result and the formatted log subset to obtain the anomalous operator, the anomalous operation device, and the anomalous operation content;

[0073] Based on the anomalous operator, the anomalous operation device, and the anomalous operation content, anomaly record combination is performed to obtain anomaly records indexed by the anomalous operator;

[0074] Based on the anomaly records, measure analysis is performed to obtain emergency measures.

[0075] For the embodiments of the present application, after log parsing, accurate positioning of the log entries with anomalies helps to quickly understand the background and causes of anomalies, provides a direction for subsequent problem solving, enhances the security of the system, and ensures the continuity and normal operation of the business. Therefore, when the anomaly log entries marked as anomalies are filtered out from the log audit result, based on the anomaly marks in the log audit result, anomaly information extraction is performed in the formatted log subset to obtain the anomalous operator, the anomalous operation device (such as IP address, MAC address, etc.), and the anomalous operation content (such as the executed command, the accessed resource, etc.). Furthermore, based on the anomalous operator, the anomalous operation device, and the anomalous operation content, anomaly record combination is performed to obtain anomaly records indexed by the anomalous operator, and the anomaly records contain complete anomaly information for subsequent analysis and processing of the anomalies in the logs. At the same time, the anomaly records established with the anomalous operator as the index facilitate querying all the anomaly records of a certain operator and help to identify the operators who frequently have anomalies. Then, based on the anomaly records, measure analysis is performed to obtain emergency measures, where the emergency measures include but are not limited to: modifying permission settings, updating system configurations, strengthening security monitoring, etc. Specifically, corresponding emergency measures can be determined for the anomalous operator, the anomalous operation device, and the anomalous operation content respectively. For example, for the anomalous operator, by restricting the account login of the anomalous operator to ensure system security; for the anomalous operation device, isolating the anomalous operation device from the system to prevent the anomalous device from causing further impact on the system; for the anomalous operation content, performing statistical analysis on the anomalous operation content to identify the patterns and trends of anomalies, which helps to discover potential security threats.

[0076] It can be seen that in the embodiment of the present application, when the log audit result includes a log anomaly, the anomaly information is analyzed based on the log audit result and the formatted log subset to obtain the anomalous operator, the anomalous operation device, and the anomalous operation content. Then, the anomaly records are combined based on the anomalous operator, the anomalous operation device, and the anomalous operation content to obtain the anomaly records indexed by the anomalous operator, and the emergency measures are obtained based on the anomaly records. Precise positioning of the log entries with anomalies helps to quickly understand the background and reasons for the anomalies, provides a direction for subsequent problem-solving, improves the security of the system, and ensures the continuity and normal operation of the business.

[0077] Further, in order to improve work efficiency and accuracy, in the embodiment of the present application, after obtaining the emergency measures based on the anomaly records, it further includes:

[0078] Performing a first visual analysis based on the log audit result to obtain an overall audit chart;

[0079] Performing a second visual analysis based on the anomaly records indexed by the anomalous operator to obtain an anomalous personnel audit chart;

[0080] Obtaining the first anomaly record and the second anomaly record before and after the execution of the emergency measures, and performing a third visual analysis based on the first anomaly record and the second anomaly record to obtain a measure effect audit chart, where the first anomaly record is the anomaly record indexed by the anomalous operator before the execution of the emergency measures, and the second anomaly record is the anomaly record indexed by the anomalous operator after the execution of the emergency measures.

[0081] For the embodiments of this application, the log audit result is the audit situation of all log files in the system, including the statistical data of normal logs and abnormal logs, and can comprehensively and completely reflect the running status and security risks of the system as a whole. Therefore, based on the log audit result, a first visual analysis is performed to obtain an overall audit chart. The overall audit chart can be a bar chart, a line chart, or a pie chart to display the quantity, proportion, or trend of various types of logs, so as to quickly understand the overall log audit situation. At the same time, a second visual analysis is performed on the abnormal records indexed by abnormal operators to obtain an abnormal personnel audit chart. The abnormal personnel audit chart can be a scatter plot, a heat map, or a Sankey diagram, etc., which is used to display information such as the number of abnormal operators, the abnormal frequency, and the abnormal type. The abnormal personnel audit chart helps to quickly identify potential internal threats and take targeted measures, thereby reducing the probability of security incidents. At the same time, the first abnormal record and the second abnormal record before and after the implementation of the emergency measure are obtained, and a third visual analysis is performed based on the first abnormal record and the second abnormal record to obtain a measure effect audit chart. By comparing the two sets of data of the first abnormal record and the second abnormal record, the impact of the emergency measure on abnormal operators and abnormal records is analyzed. For the measure effect audit chart, it can be a comparison bar chart, a line chart, or an area chart to represent the comparison of the two sets of data, and to display the change in the number of abnormal operators and the reduction of abnormal records before and after the implementation of the emergency measure. The comparative analysis method helps to evaluate the effectiveness of the emergency measure, timely discover and improve the deficiencies in the measure, thereby improving the overall security level of the system. The multi-faceted visual display is convenient for in-depth understanding of the security status of the system, the behavior patterns of abnormal operators, and the effects of emergency measures, and then optimizing the security management measures. Compared with text analysis, the work efficiency and accuracy are greatly improved.

[0082] It can be seen that in the embodiments of this application, a first visual analysis is performed based on the log audit result to obtain an overall audit chart; a second visual analysis is performed on the abnormal records indexed by abnormal operators to obtain an abnormal personnel audit chart; at the same time, the first abnormal record and the second abnormal record before and after the implementation of the emergency measure are obtained, and a third visual analysis is performed based on the first abnormal record and the second abnormal record to obtain a measure effect audit chart. The multi-faceted visual display is convenient for in-depth understanding of the security status of the system, the behavior patterns of abnormal operators, and the effects of emergency measures, and then optimizing the security management measures. Compared with text analysis, the work efficiency and accuracy are greatly improved.

[0083] Further, in the embodiments of this application, after obtaining the log set, it further includes:

[0084] Performing multi-dimensional verification based on the log data in the log set, where the multi-dimensional verification includes: data integrity verification, timestamp verification, and data source verification;

[0085] When the multi-dimensional verification fails, a log data anomaly warning is generated; when the multi-dimensional verification is successful, the log set is uploaded to the blockchain for storage to prevent unauthorized personnel from tampering with and deleting the log data.

[0086] For the embodiments of the present application, in order to ensure the accuracy of subsequent log parsing, multi-dimensional verification is first performed after obtaining the log set, effectively identifying and filtering out potentially abnormal and incorrect data, ensuring that the log data on which the parsing process is based is true and reliable, and uploading the verified log set to the blockchain for storage, ensuring the immutability and traceability of the log data.

[0087] Specifically, multi-dimensional verification is performed based on the log data in the log set. For data integrity verification, after receiving the log set, the electronic device uses a hash function to perform a hash calculation on each log data in the log set to obtain the corresponding hash value, and compares the calculated hash value with the pre-stored correct hash value. If the two are consistent, it indicates that the data integrity verification is passed; otherwise, it indicates that the data integrity verification fails. For timestamp verification, it is checked whether the timestamp of each log data in the log set is within the preset timestamp range. When all are within the timestamp range, it indicates that the timestamp verification is passed; otherwise, it indicates that the timestamp verification fails. For data source verification, it is checked whether the data source information (such as IP address, device identifier, etc.) of each log data in the log set is legal and trustworthy, that is, the data source information is compared with the known list of secure data sources to determine the finiteness of the data source. When the data source information is in the list of secure data sources, it indicates that the data source verification is passed; otherwise, it indicates that the data source verification fails. Then, based on the results corresponding to the data integrity verification, timestamp verification, and data source verification respectively, the multi-dimensional verification result is determined. That is, only when all three aspects of verification are successful is the multi-dimensional verification determined to be successful; otherwise, the multi-dimensional verification is determined to fail. Furthermore, if any link in the multi-dimensional verification process fails, the anomaly warning mechanism is immediately triggered to generate a log data anomaly warning and notify relevant personnel by sending emails or text messages, etc., so that relevant personnel can quickly locate the problem and handle it. When the multi-dimensional verification is successful, it means that the data in the log set is complete, true, and trustworthy, and the log set is uploaded to the blockchain for storage, ensuring the immutability and traceability of the log data.

[0088] It can be seen that in the embodiments of the present application, multi-dimensional verification is performed based on the log data in the log set. When the multi-dimensional verification fails, a log data anomaly warning is generated; when the multi-dimensional verification is successful, the log set is uploaded to the blockchain for storage to prevent unauthorized personnel from tampering with and deleting the log data.

[0089] The above embodiments introduce a log auditing method from the perspective of the method flow. The following embodiments introduce a log auditing device from the perspective of virtual modules or virtual units. For details, see the following embodiments.

[0090] An embodiment of the present application provides a log auditing device. As Figure 2 shown, the log auditing device may specifically include:

[0091] A log type division module 210, configured to obtain a log set and perform log type division based on the log set to obtain a log subset corresponding to each log type, where the log set includes: multi-type log data corresponding to multiple systems respectively;

[0092] A format unification module 220, configured to obtain the log format corresponding to each log type and unify the format of each log subset according to the log format corresponding to each log type to obtain a formatted log subset corresponding to each log subset;

[0093] A log parsing module 230, configured to perform log parsing on each formatted log subset to obtain a log auditing result.

[0094] For the embodiment of the present application, a log set is obtained, and log type division is performed based on the log set to obtain a log subset corresponding to each log type. Then, the log format corresponding to each log type is obtained, and the format of each log subset is unified according to the log format corresponding to each log type to obtain a formatted log subset corresponding to each log subset. Finally, log parsing is performed on each formatted log subset to obtain a log auditing result. Unifying log files of the same type into the same format facilitates subsequent parsing of the unified format log files, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

[0095] In a possible implementation manner of the embodiment of the present application, when the format unification module 220 performs unifying the format of each log subset according to the log format corresponding to each log type to obtain a formatted log subset corresponding to each log subset, it is configured to:

[0096] Perform format key analysis based on the target log format to determine format key information, where the format key information includes: log key fields, field position relationships, and log auditing periods, and the target log format is any one of the log formats;

[0097] Extract and arrange fields from the target log subset corresponding to the target log format according to the format key information to obtain a target formatted log subset, where the target formatted log subset is any one of the formatted log subsets.

[0098] In a possible implementation of the embodiment of the present application, when the log parsing module 230 performs log parsing on each formatted log subset to obtain a log audit result, it is used for:

[0099] Obtain distributed framework information, where the distributed framework information includes: the node names corresponding to the cluster, the number of nodes, and the memory distribution;

[0100] Based on the number of nodes in the distributed framework information, divide each formatted log subset into blocks to obtain log data blocks corresponding to each node name;

[0101] Control each node in the distributed framework to perform log parsing on the log data blocks in parallel to obtain a log audit result.

[0102] In a possible implementation of the embodiment of the present application, the log audit device further includes:

[0103] An exception recording module, which is used to, when the log audit result includes a log exception, analyze the exception information based on the log audit result and the formatted log subset to obtain the exception operator, the exception operation device, and the exception operation content;

[0104] Perform exception record combination based on the exception operator, the exception operation device, and the exception operation content to obtain an exception record indexed by the exception operator;

[0105] Perform measure analysis based on the exception record to obtain an emergency measure.

[0106] In a possible implementation of the embodiment of the present application, the log audit device further includes:

[0107] Perform first visual analysis based on the log audit result to obtain an overall audit chart;

[0108] A visualization display module, which is used to perform second visual analysis based on the exception record indexed by the exception operator to obtain an exception personnel audit chart;

[0109] Obtain the first exception record and the second exception record before and after executing the emergency measure, and perform third visual analysis based on the first exception record and the second exception record to obtain a measure effect audit chart, where the first exception record is the exception record indexed by the exception operator before executing the emergency measure, and the second exception record is the exception record indexed by the exception operator after executing the emergency measure.

[0110] In a possible implementation of the embodiment of the present application, the log audit device further includes:

[0111] A multi-dimensional verification module is used to perform multi-dimensional verification based on the log data in the log set. Among them, the multi-dimensional verification includes: data integrity verification, timestamp verification, and data source verification;

[0112] When the multi-dimensional verification fails, a log data anomaly warning is generated; when the multi-dimensional verification is successful, the log set is uploaded to the blockchain for storage to prevent unauthorized personnel from tampering with and deleting the log data.

[0113] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of a log auditing device described above can refer to the corresponding process in the foregoing method embodiments, and will not be described in detail here.

[0114] In an embodiment of the present application, an electronic device is provided, such as Figure 3 shown. Figure 3 The electronic device 300 shown includes: a processor 301 and a memory 303. Among them, the processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may further include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation to the embodiments of the present application.

[0115] The processor 301 may be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in combination with the disclosure of the present application. The processor 301 may also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0116] The bus 302 may include a path for transmitting information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 3It is only represented by a thick line, but it does not mean that there is only one bus or one type of bus.

[0117] The memory 303 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0118] The memory 303 is used to store the application program code for implementing the solution of this application, and is controlled by the processor 301 to execute. The processor 301 is used to execute the application program code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0119] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The shown electronic device is only an example and should not bring any limitations to the functions and usage scope of the embodiments of this application.

[0120] The embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When it runs on a computer, it enables the computer to execute the corresponding content in the foregoing method embodiments.

[0121] An embodiment of the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the method in any of the above embodiments. Compared with the related art, in the embodiment of the present application, a log set is obtained, and log type division is performed based on the log set to obtain a log subset corresponding to each log type. Then, the log format corresponding to each log type is obtained, and according to the log format corresponding to each log type, the format of each log subset is unified to obtain a formatted log subset corresponding to each log subset. Finally, log parsing is performed on each formatted log subset to obtain a log audit result. Unifying log files of the same type into the same format facilitates subsequent parsing of the log files after format unification, reduces the complexity of data parsing, and improves the accuracy of log data parsing.

[0122] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0123] The above are only some implementation manners of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A log auditing method, characterized in that, Including: Obtain a log set, and perform multi-dimensional verification based on the log data in the log set, where the multi-dimensional verification includes: data integrity verification, timestamp verification, and data source verification; When any verification fails in the multi-dimensional verification, a log data anomaly warning is generated; When the multi-dimensional verification is all successful, the log set is uploaded to the blockchain for storage to prevent unauthorized personnel from tampering with and deleting the log data, and based on the log set, log type classification is performed to obtain a log subset corresponding to each log type, where the log set includes: multi-type log data corresponding to multiple systems; the log types include system logs, application logs, security logs, network logs, and device logs; Obtain the log format corresponding to each log type, and the format key information of the log format includes log key fields, field position relationships, and log audit periods; For any log format, perform format key analysis based on the log format to determine the format key information, and according to the format key information, perform field extraction and arrangement on the log subset corresponding to the log format to obtain the formatted log subset corresponding to the log subset; Obtain distributed framework information, where the distributed framework information includes: node names, node quantities, and memory distributions corresponding to the clusters; Based on the node quantity and the memory and computing capabilities of each node in the distributed framework information, each formatted log subset is chunked to obtain a log data chunk corresponding to each node name and distributed to the corresponding node; Control each node in the distributed framework to dynamically adjust the number of threads according to the task complexity and computing resources, and concurrently perform log parsing on the log data chunks and return the corresponding log audit results to the summary node to obtain the log audit results corresponding to all log files.

2. The log auditing method according to claim 1, characterized in that, After performing log parsing on each formatted log subset to obtain the log audit results, it further includes: When the log audit results include log anomalies, based on the log audit results and the formatted log subset, perform anomaly information analysis to obtain the anomalous operator, anomalous operation device, and anomalous operation content; Based on the anomalous operator, the anomalous operation device, and the anomalous operation content, perform anomaly record combination to obtain an anomaly record indexed by the anomalous operator; Based on the anomaly record, perform measure analysis to obtain emergency measures.

3. The log auditing method according to claim 2, wherein After performing measure analysis based on the anomaly record to obtain emergency measures, it further includes: Based on the log audit results, perform first visual analysis to obtain an overall audit chart; Based on the anomaly record indexed by the anomalous operator, perform second visual analysis to obtain an anomalous personnel audit chart; Obtain the first abnormal record and the second abnormal record before and after implementing the emergency measures, and conduct a third visual analysis based on the first abnormal record and the second abnormal record to obtain an audit chart of the measure effect, where the first abnormal record is the abnormal record indexed by the abnormal operator before implementing the emergency measures, and the second abnormal record is the abnormal record indexed by the abnormal operator after implementing the emergency measures.

4. An electronic device, characterized in that, Including: At least one processor; A memory; At least one application program, where at least one application program is stored in the memory and is configured to be executed by at least one processor, and the at least one application program is configured to: execute the log auditing method according to any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed on a computer, the computer is made to execute the log auditing method according to any one of claims 1 to 3.

6. A computer program product, characterized in that, Including a computer program, and the computer program is executed by a processor to execute the log auditing method according to any one of claims 1 to 3.

Images