Log management method and system

The logs of the big data system are obtained and classified through interfaces, which solves the storage and processing problems caused by the increase in log information, and realizes efficient management and monitoring of log data.

CN119046471AInactive Publication Date: 2024-11-29HUANENG INFORMATION TECH CO LTD +1

Patent Information

Application Number
CN202411546537.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-01
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

As the big data system runs, log information gradually increases. Direct analysis and management log information is invalid and the system memory usage continues to increase, making it difficult to effectively process and store.

Method used

The logs of different modules are obtained through the interface, and the analysis results are stored in the storage tool according to the log classification analysis level, including determining the log target, configuring the log proxy tool interface, preprocessing the log, analyzing the event severity and classifying, recording the log level to storage system.

Benefits of technology

It realizes the accuracy and efficient processing of log data, helps to monitor and respond to system events more effectively, reduces system memory usage and improves the efficiency of log management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046471A_ABST
    Figure CN119046471A_ABST
Patent Text Reader

Abstract

The invention provides a log management method and system, and belongs to the technical field of system design. The method comprises the steps of 1, determining a target of a log, defining a log type according to the target, and setting basic parameters of the log; 2, configuring an interface of a log agent tool according to the basic parameters of the log, and capturing an original log of a data source through the interface; 3, preprocessing the original log to obtain a standard log, and analyzing detailed information of the standard log to obtain an event of the data source; 4, the severity of the event is analyzed according to the analysis model, and the level of the event is obtained according to preset level classification; and 5, defining the level of the log according to the level of the event, and recording the log analysis result in a storage system. The efficiency of performance analysis is improved, and troubleshooting is facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of system design, and in particular to a log management method and system. Background Art

[0002] At present, with the continuous development of big data, log information is system information that will inevitably be generated when different modules of the data system are running. These system information show the status of different modules of the system; however, as the system continues to run, the log information is increasing. Usually, directly analyzing and managing the log information will take a long time to be effective, and the system memory usage is also increasing.

[0003] Therefore, the present invention proposes a log management method and system. Summary of the invention

[0004] The present invention provides a log management method and system, which are used to obtain logs of different modules by using an interface, analyze the level of the logs according to log classification, and store the analysis results in a storage tool.

[0005] In one aspect, the present invention provides a log management method, comprising: Step 1: Determine the log target and define the log type according to the target, and set the basic parameters of the log; Step 2: Configure the interface of the log agent tool according to the basic parameters of the log, and capture the original log of the data source through the interface; Step 3: Preprocess the original log to obtain a standard log, and analyze the detailed information of the standard log to obtain the event of the data source; Step 4: Analyze the severity of the event according to the analysis model, and obtain the level of the event according to the preset level classification; Step 5: Define the level of the log according to the level of the event, and record the log analysis result in the storage system.

[0006] On the other hand, determine the log goal and define the log type based on the goal, and set the basic parameters of the log, including: Determine the purpose of the log based on the purpose of connecting to the target data source; According to the purpose of the log, match the log purpose-log type mapping table to obtain the log type; The basic parameters of the log are set according to the log type and the target data source.

[0007] On the other hand, according to the basic parameters of the log, configure the interface of the log agent tool and capture the original log of the data source through the interface, including: Select a log proxy tool based on the log type, test and match the log proxy tool with the target data source to determine compatibility, and generate a compatibility report result; Based on the compatibility report result, the incompatible versions are adjusted to ensure that the compatibility of the log proxy tool and the data source meets the preset standard; Configure the log proxy tool according to the basic parameters of the log and access the interface of the target data source based on the transmission protocol; Generate a log rotation strategy to obtain the original log array of the target data source in batches through the interface.

[0008] On the other hand, the original logs are preprocessed to obtain standard logs, including: The original log array is split into fields according to preset standard fields, all logs and their basic information are obtained, and the log IDs of all logs are obtained; Convert the timestamps corresponding to all logs into a unified format and sort them in logical chronological order to obtain a log list; Use the prompting model to decompose any unit log in the log list into structured fields; ; Wherein, F represents the matching degree between any prompting standard and the unit log, represents the matching degree between the i-th word of the unit log and the prompting standard, T represents the minimum split matching standard, represents the matching degree between the i-1th word of the unit log and the prompting standard, exp() represents the exponential function, Indicates the minimum matching degree between any word in the unit log and the prompt standard, Indicates the highest matching degree between any word in the unit log and the prompt standard, Indicates the average matching degree between any vocabulary of the unit log and the prompting standard; Split out all structured fields of all unit logs according to the prompting criteria and matching degree; The fields and format requirements of the preset log are determined to generate a standard template, the structured fields are inserted into the standard template to obtain the standard log of the unit log, and a mapping relationship between the log id and the standard log list is constructed.

[0009] On the other hand, detailed information of the standard log is analyzed to obtain events of the data source, including: Creating identification event rules according to the target of the log of the data source and configuring them into the event analysis tool; By using an event analysis tool to parse any standard log and extract multiple sub-events, the sub-events are clustered into unit clusters according to the time logical order and event type, and the unit clusters are aggregated into the data source event report based on the format of the event report.

[0010] On the other hand, the severity of the event is analyzed according to the analysis model, and the level of the event is obtained according to the preset level classification, including: defining severity levels of the incidents and presetting thresholds for each severity level; Extract all event features of all event reports, substitute any event feature into the analysis model, and obtain a single severity of the event feature; According to the types of different event features, the event feature type-weight mapping table is matched to obtain the weight of the event feature; The severity of the event report is obtained by combining the single severity of all event features and their weights, the level of the event report is obtained according to the preset threshold of the corresponding severity level, and a mapping relationship between the log id and the event report is constructed.

[0011] On the other hand, the level of the log is defined according to the level of the event, and the log analysis result is recorded in the storage system, including: Create mapping rules between event report level and log level, build mapping table and configure it in log storage; After receiving the log ID, the log storage obtains the event report level corresponding to the log ID and converts it into the log level based on the mapping table; Obtain a list of standard logs corresponding to the log id, and aggregate multiple standard logs into one standard log message according to the log timestamp; A log entry of the log level is generated, standard log information is inserted into the log entry and stored in a storage system of the log storage.

[0012] In another aspect, the present invention provides a log management system, comprising: Basic module: determine the log target and define the log type according to the target, and set the basic parameters of the log; Interface module: configures the interface of the log agent tool according to the basic parameters of the log, and captures the original log of the data source through the interface; Event module: pre-processes the original log to obtain a standard log, and analyzes the detailed information of the standard log to obtain the event of the data source; Analysis module: analyzes the severity of the event according to the analysis model, and obtains the level of the event according to the preset level classification; Storage module: defines the level of the log according to the level of the event, and records the log analysis result into the storage system.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a log management method and system, which are used to obtain logs of different modules by using an interface, analyze the level of the logs according to log classification, and store the analysis results in a storage tool. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 It is a flow chart of a log management method provided by an embodiment of the present invention; Figure 2 It is a structural diagram of a log management system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Embodiment 1: like Figure 1 As shown, a log management method provided by an embodiment of the present invention includes: Step 1: Determine the log target and define the log type according to the target, and set the basic parameters of the log; Step 2: Configure the interface of the log agent tool according to the basic parameters of the log, and capture the original log of the data source through the interface; Step 3: Preprocess the original log to obtain a standard log, and analyze the detailed information of the standard log to obtain the event of the data source; Step 4: Analyze the severity of the event according to the analysis model, and obtain the level of the event according to the preset level classification; Step 5: Define the level of the log according to the level of the event, and record the log analysis result in the storage system.

[0018] In this embodiment, the log is a data file that records information such as system operation, event processing, and user activities, and is used to track and audit system behavior, troubleshoot errors, and monitor performance.

[0019] In this embodiment, the goal is, for example, performance monitoring, security auditing, or troubleshooting.

[0020] In this embodiment, the log type refers to a method of classifying logs according to recording targets and contents, such as error logs, warning logs, information logs, etc.

[0021] In this embodiment, the basic parameters refer to the core elements for setting log records, such as log format, record frequency, storage path, filtering rules, etc.

[0022] In this embodiment, the log agent tool is an application program used to collect and forward log data.

[0023] In this embodiment, the interface refers to the interaction point between the log agent tool and the data source or other system.

[0024] In this embodiment, the data source refers to the system, application or device that generates the original log, which can be a server, network device, application, operating system, etc.

[0025] In this embodiment, the original log is log data directly captured from the data source without being modified or processed.

[0026] In this embodiment, preprocessing refers to the preliminary processing steps performed on the original log data before further analysis or processing of the log data, including: data cleaning, format conversion, data parsing and other steps.

[0027] In this embodiment, the standard log is the original log data that has been pre-processed and converted into a log in a unified and standardized format.

[0028] In this embodiment, detailed information refers to specific content extracted from the standard log, including: timestamp, event type, source IP address, etc.

[0029] In this embodiment, an event is a specific activity record extracted from a log that reflects a critical operation abnormality that occurs in the system.

[0030] In this embodiment, the analysis model is a tool used to evaluate and classify the severity of events in the log, and conducts in-depth analysis of the events based on preset rules or data-driven algorithms to determine their impact and priority.

[0031] In this embodiment, the severity indicates the urgency and importance of the event.

[0032] In this embodiment, the preset level classification refers to the result of classifying events according to predefined standards during the log analysis process.

[0033] In this embodiment, the level refers to the classification of the event, reflecting the severity of the event.

[0034] In this embodiment, the log analysis result is information that has been processed and classified, and describes the type and severity of the event and its corresponding log level.

[0035] In this embodiment, the storage system refers to a system used to store, manage and retrieve log data.

[0036] The working principle and beneficial effects of the above technical solution are: by clarifying the log target, configuring tools, capturing and preprocessing logs, analyzing the severity of events and classifying them, and finally recording logs. This process ensures the accuracy and efficient processing of log data, which helps to monitor and respond to system events more effectively.

[0037] Embodiment 2: Based on the above embodiment 1, the target of the log is determined and the log type is defined according to the target, and the basic parameters of the log are set, including: Determine the purpose of the log based on the purpose of connecting to the target data source; According to the purpose of the log, match the log purpose-log type mapping table to obtain the log type; The basic parameters of the log are set according to the log type and the target data source.

[0038] In this embodiment, the purpose of docking refers to the specific goals and requirements for integration or connection with the target data source, such as monitoring and troubleshooting, security and compliance, performance optimization, etc.

[0039] In this embodiment, the uses include: troubleshooting, monitoring, security auditing, performance optimization, business analysis, etc.

[0040] In this embodiment, the log purpose-log type mapping table is a table indicating the mapping relationship between log purpose and log type.

[0041] The working principle and beneficial effect of the above technical solution are: by determining the purpose of the log and matching the log type, setting basic parameters to connect to the target data source. This process ensures the accurate configuration and effective connection of the log, optimizes the relevance and practicality of the log records, and improves the accuracy of data monitoring and analysis.

[0042] Embodiment 3: On the basis of the above-mentioned embodiment 2, according to the basic parameters of the log, the interface of the log agent tool is configured, and the original log of the data source is captured through the interface, including: Select a log proxy tool based on the log type, test and match the log proxy tool with the target data source to determine compatibility, and generate a compatibility report result; Based on the compatibility report result, the incompatible versions are adjusted to ensure that the compatibility of the log proxy tool and the data source meets the preset standard; Configure the log proxy tool according to the basic parameters of the log and access the interface of the target data source based on the transmission protocol; Generate a log rotation strategy to obtain the original log array of the target data source in batches through the interface.

[0043] In this embodiment, compatibility refers to the ability of different systems, tools or components to work together seamlessly.

[0044] In this embodiment, the compatibility report result is a summary and analysis report of the compatibility test between the log proxy tool and the target data source, including: format, protocol, function, version and other aspects.

[0045] In this embodiment, the preset standard refers to specific requirements or conditions that are expected to be met when performing compatibility testing between the log proxy tool and the target data source.

[0046] In this embodiment, the basic parameter configuration includes: log format, log level, collection frequency, target address, authentication information, etc.

[0047] In this embodiment, the transmission protocol refers to the rules and standards for sending and receiving data on the network, including: HTTP / HTTPS, TCP / UDP, etc.

[0048] In this embodiment, the log rotation strategy refers to a mechanism for managing log files to prevent the log files from being too large, including: time-based rotation, size-based rotation, quantity-based rotation, archiving, compression, and other types.

[0049] In this embodiment, the original log array refers to a set of log data obtained from the target data source without being processed or parsed.

[0050] The working principle and beneficial effects of the above technical solution are: by testing and adjusting the compatibility of the log proxy tool, configuring parameters, accessing the data source interface, and using a rotation strategy to obtain logs in batches, the log collection process is optimized, and the compatibility and collection efficiency of the tool and data source are improved.

[0051] Embodiment 4: Based on the above embodiment 1, the original log is preprocessed to obtain a standard log, including: The original log array is split into fields according to preset standard fields, all logs and their basic information are obtained, and the log IDs of all logs are obtained; Convert the timestamps corresponding to all logs into a unified format and sort them in logical chronological order to obtain a log list; Use the prompting model to decompose any unit log in the log list into structured fields; ; Wherein, F represents the matching degree between any prompting standard and the unit log, represents the matching degree between the i-th word of the unit log and the prompting standard, T represents the minimum split matching standard, represents the matching degree between the i-1th word of the unit log and the prompting standard, exp() represents the exponential function, Indicates the minimum matching degree between any word in the unit log and the prompt standard, Indicates the highest matching degree between any word in the unit log and the prompt standard, Indicates the average matching degree between any vocabulary of the unit log and the prompting standard; Split out all structured fields of all unit logs according to the prompting criteria and matching degree; The fields and format requirements of the preset log are determined to generate a standard template, the structured fields are inserted into the standard template to obtain the standard log of the unit log, and a mapping relationship between the log id and the standard log list is constructed.

[0052] In this embodiment, the preset standard fields refer to a set of predefined field names and format requirements.

[0053] In this embodiment, field splitting refers to the process of parsing and separating original log data according to preset standard fields.

[0054] In this embodiment, the log id is an identifier used to uniquely identify each log record.

[0055] In this embodiment, the timestamp is a value or mark indicating a specific time point, and is recorded in the log entry in the form of date and time.

[0056] In this embodiment, the time logical order refers to sorting the logs according to the actual time order of the occurrence of events.

[0057] In this embodiment, the log list refers to a set of log records sorted in chronological order.

[0058] In this embodiment, the prompting model is a natural language processing technology used to decompose text into structured fields.

[0059] In this embodiment, the unit log refers to each independent log record in the log list.

[0060] In this embodiment, the structured field is a unit field of specific and clear information extracted from the original log.

[0061] In this embodiment, the standard template The working principle and beneficial effects of the above technical solution are: splitting the original log and converting it into structured fields, generating a standard log according to the prompting standard, and finally establishing a mapping relationship between the log ID and the standard log, thereby improving the standardization and efficiency of log processing.

[0062] Embodiment 5: Based on the above embodiment 4, the detailed information of the standard log is analyzed to obtain the events of the data source, including: Creating identification event rules according to the target of the log of the data source and configuring them into the event analysis tool; By using an event analysis tool to parse any standard log and extract multiple sub-events, the sub-events are clustered into unit clusters according to the time logical order and event type, and the unit clusters are aggregated into the data source event report based on the format of the event report.

[0063] In this embodiment, the identification event rule is a logic rule used to automatically detect and extract specific events from log data.

[0064] In this embodiment, the event analysis tool is software for automatically identifying, analyzing, and reporting events from log data.

[0065] In this embodiment, a sub-event is a smaller, more specific event extracted from a larger event, such as a main event: system crash, sub-event 1: database connection failure, and sub-event 2: application abnormal termination.

[0066] In this embodiment, clustering is a data analysis technology used to group objects in a data set so that the objects in the same group have as high similarity as possible with each other and as low similarity as possible with objects in other groups.

[0067] In this embodiment, the unit cluster refers to a group formed by grouping data through a clustering algorithm.

[0068] In this embodiment, the event report is a report format generated based on data analyzed by the event analysis tool, and is used to summarize and display the results of the event analysis.

[0069] The working principle and beneficial effects of the above technical solution are: by setting event identification rules, parsing standard logs to extract and cluster sub-events, and generating data source event reports, the accuracy of event analysis and the systematization of reports are improved.

[0070] Embodiment 6: On the basis of the above-mentioned embodiment 1, the severity of the event is analyzed according to the analysis model, and the level of the event is obtained according to the preset level classification, including: defining severity levels of the incidents and presetting thresholds for each severity level; Extract all event features of all event reports, substitute any event feature into the analysis model, and obtain a single severity of the event feature; According to the types of different event features, the event feature type-weight mapping table is matched to obtain the weight of the event feature; The severity of the event report is obtained by combining the single severity of all event features and their weights, the level of the event report is obtained according to the preset threshold of the corresponding severity level, and a mapping relationship between the log id and the event report is constructed.

[0071] In this embodiment, the preset threshold is a specific numerical limit for defining the severity level of the event, for example: low level: severity score in the range of 0-3, medium level: severity score in the range of 4-6, high level: severity score in the range of 7-10.

[0072] In this embodiment, the event feature is a specific attribute used to describe the nature and impact of the event, such as event type, occurrence time, error code, trigger condition, etc.

[0073] In this embodiment, a single severity refers to a specific value of the severity of an event represented by an event feature obtained after substituting the event feature into the analysis model.

[0074] In this embodiment, the feature type-weight mapping table is a table representing the mapping relationship between event feature types and corresponding weights.

[0075] In this embodiment, the weight represents the importance of each event feature in calculating the overall severity of the event report.

[0076] The working principle and beneficial effects of the above technical solution are: defining the event severity level and threshold, extracting event features and analyzing their severity, calculating feature weights to determine the overall severity of the event report. Finally, building a mapping relationship between log ID and event report, improving the accuracy of event classification and response priority setting.

[0077] Embodiment 7: Based on the above embodiment 1, the level of the log is defined according to the level of the event, and the log analysis result is recorded in the storage system, including: Create mapping rules between event report level and log level, build mapping table and configure it in log storage; After receiving the log ID, the log storage obtains the event report level corresponding to the log ID and converts it into the log level based on the mapping table; Obtain a list of standard logs corresponding to the log id, and aggregate multiple standard logs into one standard log message according to the log timestamp; A log entry of the log level is generated, standard log information is inserted into the log entry and stored in a storage system of the log storage.

[0078] In this embodiment, the mapping rule is a rule that defines how to convert the event report level into the log level in the log storage system.

[0079] In this embodiment, the mapping table is a data structure that defines the relationship between the event report level and the log level.

[0080] In this embodiment, the log storage is a tool for recording, storing and managing log information.

[0081] In this embodiment, a log entry is a basic unit recorded in the log storage, and is used to describe the status information of a specific event, including: a timestamp, a log level, an event description, and the like.

[0082] The working principle and beneficial effects of the above technical solution are: the event report level is converted into the log level through the mapping rule, and the standard log is integrated into a piece of information according to the timestamp. Finally, the log entry is generated and stored, which improves the efficiency and consistency of log classification and storage.

[0083] Embodiment 8: like Figure 2 As shown, a log management system provided by an embodiment of the present invention includes: Basic module: determine the log target and define the log type according to the target, and set the basic parameters of the log; Interface module: configures the interface of the log agent tool according to the basic parameters of the log, and captures the original log of the data source through the interface; Event module: pre-processes the original log to obtain a standard log, and analyzes the detailed information of the standard log to obtain the event of the data source; Analysis module: analyzes the severity of the event according to the analysis model, and obtains the level of the event according to the preset level classification; Storage module: defines the level of the log according to the level of the event, and records the log analysis result into the storage system.

[0084] The working principle and beneficial effects of the above technical solution are: by clarifying the log target, configuring tools, capturing and preprocessing logs, analyzing the severity of events and classifying them, and finally recording logs. This process ensures the accuracy and efficient processing of log data, which helps to monitor and respond to system events more effectively.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A log management method, characterized in that: include: Step 1: Determine the log target and define the log type according to the target, and set the basic parameters of the log; Step 2: Configure the interface of the log agent tool according to the basic parameters of the log, and capture the original log of the data source through the interface; Step 3: Preprocess the original log to obtain a standard log, and analyze the detailed information of the standard log to obtain the event of the data source; Step 4: Analyze the severity of the event according to the analysis model, and obtain the level of the event according to the preset level classification; Step 5: Define the level of the log according to the level of the event, and record the log analysis result in the storage system.

2. A log management method according to claim 1, characterized in that: Determine the log's goal and define the log type based on the goal, and set the basic parameters of the log, including: Determine the purpose of the log based on the purpose of connecting to the target data source; According to the purpose of the log, match the log purpose-log type mapping table to obtain the log type; The basic parameters of the log are set according to the log type and the target data source.

3. A log management method according to claim 2, characterized in that: Configure the interface of the log proxy tool according to the basic parameters of the log, and capture the original log of the data source through the interface, including: Select a log proxy tool based on the log type, test and match the log proxy tool with the target data source to determine compatibility, and generate a compatibility report result; Based on the compatibility report result, the incompatible versions are adjusted to ensure that the compatibility of the log proxy tool and the data source meets the preset standard; Configure the log proxy tool according to the basic parameters of the log and access the interface of the target data source based on the transmission protocol; Generate a log rotation strategy to obtain the original log array of the target data source in batches through the interface.

4. A log management method according to claim 1, characterized in that: The original log is preprocessed to obtain the standard log, including: The original log array is split into fields according to preset standard fields, all logs and their basic information are obtained, and the log IDs of all logs are obtained; Convert the timestamps corresponding to all logs into a unified format and sort them in logical chronological order to obtain a log list; Use the prompting model to decompose any unit log in the log list into structured fields; ; Wherein, F represents the matching degree between any prompting standard and the unit log, represents the matching degree between the i-th word of the unit log and the prompting standard, T represents the minimum split matching standard, represents the matching degree between the i-1th word of the unit log and the prompting standard, exp() represents the exponential function, Indicates the minimum matching degree between any word in the unit log and the prompt standard, Indicates the highest matching degree between any word in the unit log and the prompt standard, Indicates the average matching degree between any vocabulary of the unit log and the prompting standard; Split out all structured fields of all unit logs according to the prompting criteria and matching degree; The fields and format requirements of the preset log are determined to generate a standard template, the structured fields are inserted into the standard template to obtain the standard log of the unit log, and a mapping relationship between the log id and the standard log list is constructed.

5. A log management method according to claim 4, characterized in that: Analyze the detailed information of the standard log to obtain the events of the data source, including: Creating identification event rules according to the target of the log of the data source and configuring them into the event analysis tool; By using an event analysis tool to parse any standard log and extract multiple sub-events, the sub-events are clustered into unit clusters according to the time logical order and event type, and the unit clusters are aggregated into the data source event report based on the format of the event report.

6. A log management method according to claim 1, characterized in that: The severity of the event is analyzed according to the analysis model, and the level of the event is obtained according to the preset level classification, including: defining severity levels of the incidents and presetting thresholds for each severity level; Extract all event features of all event reports, substitute any event feature into the analysis model, and obtain a single severity of the event feature; According to the types of different event features, the event feature type-weight mapping table is matched to obtain the weight of the event feature; The severity of the event report is obtained by combining the single severity of all event features and their weights, the level of the event report is obtained according to the preset threshold of the corresponding severity level, and a mapping relationship between the log id and the event report is constructed.

7. A log management method according to claim 1, characterized in that: Define the log level according to the level of the event, and record the log analysis results in the storage system, including: Create mapping rules between event report level and log level, build mapping table and configure it in log storage; After receiving the log ID, the log storage obtains the event report level corresponding to the log ID and converts it into the log level based on the mapping table; Obtain a list of standard logs corresponding to the log id, and aggregate multiple standard logs into one standard log message according to the log timestamp; A log entry of the log level is generated, standard log information is inserted into the log entry and stored in a storage system of the log storage.

8. A log management system, characterized in that: include: Basic module: determine the log target and define the log type according to the target, and set the basic parameters of the log; Interface module: configures the interface of the log agent tool according to the basic parameters of the log, and captures the original log of the data source through the interface; Event module: pre-processes the original log to obtain a standard log, and analyzes the detailed information of the standard log to obtain the event of the data source; Analysis module: analyzes the severity of the event according to the analysis model, and obtains the level of the event according to the preset level classification; Storage module: defines the level of the log according to the level of the event, and records the log analysis result into the storage system.

Citation Information

Patent Citations

  • Log correlation analysis method based on time axis events

    CN108964995A

  • Log-based early warning method and device, server and storage medium

    CN112636957A

  • Methods and systems for processing log data

    US20190286628A1

Cited By

  • Method, system and device for detecting newly added bad block of SSD (Solid State Disk)

    CN121075404A