Alarm generation method and device based on log data, equipment, medium and product

Abstract

The invention provides an alarm generation method and device based on log data, equipment, a medium and a product. Relates to the technical field of data processing. The method comprises the following steps: acquiring multi-source log data, and extracting a key index of a target business scene from the multi-source log data; determining an abnormal fluctuation index according to the key index and the historical baseline corresponding to each index; matching a corresponding alarm strategy based on the abnormal fluctuation index, wherein the alarm strategy comprises a threshold condition, a time window and a service scene; generating a corresponding alarm event in response to the abnormal fluctuation index meeting the corresponding alarm strategy; and determining a corresponding notification channel according to the priority of the alarm event so as to report the alarm event and the alarm recovery event through the notification channel. According to the method, a composite logic alarm strategy and a closed-loop notification mechanism are introduced, so that the technical breakthrough of log processing from passive recording to active decision support is realized.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device, medium and product for generating alarms based on log data. Background Technology

[0002] In modern enterprise IT systems, cloud computing platforms, and distributed business architectures, log data is a core observation carrier reflecting system operating status, business performance, and security events. As business complexity increases, logs exhibit multi-source heterogeneous characteristics, encompassing various types of logs from applications, servers, middleware, etc., and the data volume is growing exponentially. The business trends and potential risks hidden in logs are crucial for system operation and maintenance decisions, especially in industries with extremely high stability requirements such as finance and telecommunications, where there is an urgent need to realize the deep value transformation of log data.

[0003] Currently, the core processes of the log processing solution include log collection and standardization, data supplementation and filtering, and log reporting. Multi-source logs are collected using tools such as Filebeat, parsed, and mapped to a unified field format to eliminate heterogeneity; missing fields are supplemented based on network asset information, and low-quality logs such as duplicates and incorrectly formatted logs are filtered out; finally, the logs are distributed to storage or analysis systems through APIs, message queues, and other channels to complete initial cleaning and transmission.

[0004] However, insufficient value mining of logs makes it difficult to assist in operation and maintenance decisions; alarm strategies lack flexibility and accuracy, static threshold judgment cannot adapt to dynamic business scenarios, and are prone to false alarms or missed alarms due to instantaneous fluctuations. Summary of the Invention

[0005] The alarm generation method, apparatus, device, medium, and product based on log data provided in this application are intended to address the problems of insufficient log value mining and simplistic alarm strategies.

[0006] In a first aspect, embodiments of this application provide an alarm generation method based on log data, the method comprising:

[0007] Acquire multi-source log data and extract key indicators for the target business scenario from the multi-source log data, including operational indicators and business indicators;

[0008] The abnormal fluctuation indicators are determined based on the key indicators and the historical baselines corresponding to each indicator.

[0009] Based on the abnormal fluctuation index, a corresponding alarm strategy is matched. The alarm strategy includes threshold conditions, time windows, and business scenarios.

[0010] In response to the abnormal fluctuation index meeting the corresponding alarm policy, a corresponding alarm event is generated. The alarm event includes the target service, the alarm reason, the abnormal fluctuation index, and the corresponding log data link.

[0011] The corresponding notification channel is determined based on the priority of the alarm event, so that the alarm event and alarm recovery event are reported through the notification channel.

[0012] In one possible implementation, the steps for generating and updating the historical baseline include:

[0013] Obtain normal operation data of key indicators for a continuous preset period under the target business scenario;

[0014] The average value, variance of fluctuation, and peak range are calculated based on the normal operation data to construct an initial historical baseline;

[0015] Periodically acquire the latest normal operation data at the current moment;

[0016] The initial historical baseline is iteratively updated based on the latest normal operation data.

[0017] In one possible implementation, after generating the corresponding alarm event, the method further includes:

[0018] Based on the type of alarm event, the identifier of the target service, and the target business scenario, an adaptive suppression parameter is determined, which includes a suppression time window and an alarm frequency threshold.

[0019] When the number of times an alarm event of the same type for the same target service is triggered within the suppression time window reaches the alarm frequency threshold, short-term high-frequency alarms are merged into a continuous alarm state, and the matching conditions of the suppression rules are recorded. The matching conditions of the suppression rules include alarm trigger frequency, associated indicators, and time window.

[0020] In one possible implementation, acquiring multi-source log data includes:

[0021] Based on the corresponding log collection requirements, a preset collection tool is used to acquire multi-source log data through a breakpoint resume mechanism.

[0022] The multi-source log data is standardized, and business tags and environment tags are added. The standardization process includes log format unification, field mapping, invalid data filtering, and duplicate log deduplication.

[0023] The deduplication of duplicate logs includes retaining only one log data when the log hash values ​​from the same source are the same and the generation time difference does not exceed a preset time threshold, while recording the number of deduplicated logs and their source information.

[0024] In one possible implementation, it also includes:

[0025] The recipient's current work status is obtained through the notification channel, including online status, task load, and historical response speed.

[0026] When the alarm event is of a preset priority level, if the recipient does not respond to the first notification within a preset time period, the system switches to a preset backup notification channel and notifies the backup recipient group.

[0027] In one possible implementation, the alarm event further includes troubleshooting suggestions, and the method further includes:

[0028] Obtain alarm handling solutions of the same type as the alarm event from the historical alarm event database;

[0029] Based on the link context information and original log characteristics corresponding to the abnormal fluctuation indicators, investigation steps are extracted from the alarm handling scheme.

[0030] Based on the priority of the alarm events and the target business scenario, the alarm events are embedded after being sorted according to the priority of the investigation steps, so that one-click jump to the relevant investigation tools or configuration pages can be made.

[0031] Secondly, embodiments of this application provide an alarm generation device based on log data, comprising:

[0032] The acquisition module is used to acquire multi-source log data and extract key indicators of the target business scenario from the multi-source log data. The key indicators include operational indicators and business indicators.

[0033] The determination module is used to determine abnormal fluctuation indicators based on the key indicators and the historical baselines corresponding to each indicator.

[0034] The matching module is used to match the corresponding alarm strategy based on the abnormal fluctuation index. The alarm strategy includes threshold conditions, time windows and business scenarios.

[0035] The generation module is used to generate a corresponding alarm event in response to the abnormal fluctuation index meeting the corresponding alarm policy. The alarm event includes the target service, alarm reason, abnormal fluctuation index and corresponding log data link.

[0036] The determining module is further configured to determine the corresponding notification channel based on the priority of the alarm event, so as to report the alarm event and alarm recovery event through the notification channel.

[0037] Thirdly, embodiments of this application provide an alarm generation device based on log data, including: a memory and a processor;

[0038] The memory stores computer-executed instructions;

[0039] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0041] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0042] The alarm generation method, apparatus, device, medium, and product based on log data provided in this application acquire multi-source log data, enabling a comprehensive understanding of system operation and business activities. This avoids information gaps from single data sources, and comprehensive analysis can uncover potential correlations, providing rich evidence for accurate anomaly detection. Key indicators of the target business scenario reflect the system's operation and results under specific business conditions, eliminating irrelevant data interference and making subsequent analysis and processing more efficient and accurate. Historical baselines are statistically derived from key indicator data during normal system operation; comparison with these baselines clearly determines whether current indicators are abnormal, providing an objective reference for anomaly detection. By comparing and identifying abnormal fluctuation indicators, problems can be quickly and accurately located, providing early warnings and improving system reliability and stability. Linking alarm strategies with business scenarios allows for the formulation of rules based on business characteristics, making alarms more targeted and practical. Time window settings consider the time characteristics of indicators, avoiding false alarms due to instantaneous fluctuations and improving accuracy. Alarm strategies include multi-dimensional information such as threshold conditions, time windows, and business scenarios, allowing for flexible settings to meet diverse needs and ensure timely and effective alarms. The generated alarm events include information such as the target service, cause, abnormal indicators, and log links, providing comprehensive details for operations and maintenance personnel to quickly locate problems, analyze root causes, and improve fault handling efficiency. Appropriately selecting notification channels can improve alarm handling efficiency, enabling operations and maintenance personnel to respond promptly, reducing the impact of faults on business operations, ensuring business continuity and stability, and allowing different alarm events to have independently configured notification channels to adapt to individual needs. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] Figure 1An application scenario diagram of an alarm generation method based on log data provided in this application;

[0045] Figure 2 A flowchart illustrating an alarm generation method based on log data provided in an embodiment of this application;

[0046] Figure 3 A flowchart illustrating an alarm generation method based on log data, provided in another embodiment of this application;

[0047] Figure 4 A schematic diagram of the structure of an alarm generation device based on log data provided in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the structure of an alarm generation device based on log data provided in an embodiment of this application.

[0049] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0050] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0051] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0052] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0053] In modern enterprise-level IT systems, cloud computing platforms, and distributed business architecture environments, log data is a core observation carrier that accurately reflects system operating status, business processing performance, and security events. With the continuous expansion of business scale and the increasing complexity of architecture, log data exhibits significant multi-source heterogeneity, covering various data sources such as applications, servers, and middleware, and its volume is growing exponentially. For industries such as finance and telecommunications, which have extremely high requirements for system stability, the business operation trends and potential risks contained in logs are key evidence supporting operational decisions; therefore, it is urgent to achieve in-depth mining and value transformation of log data. Current mainstream log processing solutions revolve around three main stages: log collection and standardization, data supplementation and filtering, and log reporting. Specifically, multi-source logs are collected using dedicated tools such as Filebeat, parsed, and mapped to a unified field format to eliminate heterogeneity between logs from different sources; missing fields in the logs are supplemented based on network asset-related information, and low-quality logs such as duplicates and incorrectly formatted logs are filtered out; finally, the processed logs are distributed to designated storage or analysis systems through transmission channels such as APIs and message queues, completing the initial cleaning and transmission process of the logs. However, the existing solutions still have obvious shortcomings: on the one hand, the value mining of log data is not deep enough, and the effective information in it is not fully extracted, making it difficult to provide strong support for operation and maintenance decisions; on the other hand, the alerting strategy lacks flexibility and accuracy, and the judgment method that relies on static thresholds cannot adapt to dynamically changing business scenarios, and is very prone to false alarms or missed alarms due to instantaneous fluctuations in the system.

[0054] Therefore, when facing technical problems in existing technologies, since existing solutions only complete preliminary data cleaning and transmission, they are insufficient to assist in operation and maintenance decisions. Key indicators, however, accurately reflect the system's operational status and core business characteristics. Therefore, we first acquire multi-source log data, separating operational and business indicators for the target business scenario. By focusing on key indicators, we realize the value transformation of log data, providing accurate data support for subsequent alarms. To reduce the susceptibility of static thresholds to instantaneous fluctuations, historical baselines are constructed for each key indicator. Comparison between indicators and baselines identifies real abnormal fluctuations, avoiding the limitations of single static judgments. Simultaneously, a composite alarm strategy incorporating threshold conditions, time windows, and business scenarios is designed. Combined with multi-dimensional condition matching, this improves the flexibility and accuracy of alarm judgment, adapting to dynamic business needs. Addressing the issues of lack of associated tracing information, low efficiency in operation and maintenance investigation, and lack of closed-loop management, when generating alarm events, key information such as the target service, alarm cause, abnormal indicators, and log data links are integrated to assist in rapid tracing. At the same time, corresponding notification channels are assigned according to alarm priority, and alarm recovery events are reported synchronously, forming a complete closed loop of alarm triggering, response, and recovery, thereby improving operation and maintenance efficiency.

[0055] Figure 1This is a diagram illustrating an application scenario for implementing the log data-based alarm generation method provided in this application, such as... Figure 1 As shown in the diagram, the scenario diagram corresponding to the alarm generation method based on log data provided in this application includes: server 101, policy database 102, and mobile terminal 103. Among them, server 101 is deployed with various applications and middleware.

[0056] Optionally, the alarm generation device based on log data can be integrated into server 101.

[0057] Specifically, the log data-based alarm generation device obtains multi-source log data from server 101 in real time, including application, server, and middleware logs. It extracts key indicators for the target business scenario from the multi-source log data and then determines abnormal fluctuation indicators based on these key indicators and their corresponding historical baselines. The log data-based alarm generation device matches corresponding alarm policies from the policy database 102 based on the abnormal fluctuation indicators. In response to the abnormal fluctuation indicators meeting the corresponding alarm policies, a corresponding alarm event is generated. Finally, the log data-based alarm generation device determines the corresponding notification channel based on the priority of the alarm event, such as... Figure 1 The mobile terminal 103 in the middle can report alarm events and alarm recovery events through notification channels.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0059] Figure 2 This is a flowchart illustrating an embodiment of the alarm generation method based on log data provided in this application, as shown below. Figure 2 As shown, the execution entity in this embodiment is an alarm generation device based on log data. This device can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc, or through a physical device integrating or installing the relevant computer program, such as a chip or an alarm generation device based on log data. The alarm generation device based on log data can be a server, a server cluster, etc. The alarm generation method based on log data provided in this embodiment includes the following steps:

[0060] S201. Obtain multi-source log data and extract key indicators for the target business scenario from the multi-source log data. Key indicators include operational indicators and business indicators.

[0061] Multi-source log data refers to a collection of logs from different data sources in an enterprise IT system, such as application logs, server logs, and middleware logs.

[0062] Among them, the target business scenario refers to a specific business scope defined according to the operation and maintenance requirements, such as payment business and transfer business in the financial transaction system.

[0063] Key metrics refer to core data dimensions extracted from log data that accurately reflect the system's operational status and business performance. These include operational metrics such as service response time, number of errors, and API call success rate; and business metrics such as order volume, payment success rate, and user registration volume.

[0064] Specifically, based on the different data source types in the system architecture, corresponding log collection tools, such as Filebeat, are adapted to establish collection connections with each data source. Dedicated collection rules are configured for different log storage locations, such as local files and network streams. During collection, a breakpoint resumption mechanism is enabled. If the network is interrupted or the collection tool stops abnormally, collection can resume from the breakpoint after recovery, avoiding data loss. Simultaneously, the collected log data undergoes batch compression. After collection is complete, log data from all sources is aggregated to a unified data receiving node.

[0065] Furthermore, the aggregated multi-source log data undergoes preprocessing, including standardizing log formats and field standards, transforming heterogeneous logs from different data sources into a unified structured format, and automatically filtering invalid logs. Based on a predefined target business scenario, log data relevant to that business is selected. For the selected business-related logs, specific indicator data is extracted according to the classification rules of operational and business indicators. When extracting operational indicators, information such as service response time, error code occurrence count, and API call return status are parsed from the logs, and derived indicators such as API call success rate and error rate are obtained through statistical calculations. When extracting business indicators, business fields such as order number, payment status, and registered user ID are identified from the logs, and core business data such as order volume, successful payment count, and registration volume per unit time are statistically obtained, ultimately forming a set of key indicators for the target business scenario.

[0066] S202. Determine abnormal fluctuation indicators based on key indicators and the historical baselines corresponding to each indicator.

[0067] The historical baseline refers to the normal fluctuation range and characteristic benchmark of a key indicator formed through statistical analysis based on historical data of the key indicator over a continuous normal operating period.

[0068] Among them, abnormal fluctuation indicators refer to indicators whose real-time monitoring values ​​of key indicators deviate from their corresponding historical baselines, and whose deviation magnitude or duration reaches the preset judgment criteria.

[0069] Specifically, based on the extracted key indicators, they are categorized and matched according to indicator type and target business scenario, and each key indicator is associated with its corresponding historical baseline. It is then determined whether the current indicator value falls within the normal fluctuation range of the historical baseline. For example, if the historical baseline shows a normal payment success rate range of 99.5%-100%, and the current success rate is 98.8%, it exceeds this range. The trend of the current indicator is analyzed to see if it is consistent with the normal trend of the historical baseline. For example, if the historical baseline shows a steady increase in order volume during the morning peak period, and the current order volume drops sharply, the trend is abnormal. The deviation of the current indicator from the historical baseline average is calculated. For example, if the historical baseline average number of errors is 5 times / hour, and the current number of errors is 20 times / hour, the deviation is far beyond the normal range.

[0070] Furthermore, based on preset anomaly detection rules, the above comparison results are comprehensively judged. If the current value of the indicator exceeds the normal range of the historical baseline, or the trend change or fluctuation amplitude does not conform to the normal pattern, and this state continues for a preset duration, such as 3 minutes, then the indicator is determined to be an abnormal fluctuation indicator.

[0071] Optionally, during the comparison process, key information such as the current value of the indicator, the normal range of the baseline, the deviation magnitude and duration are recorded simultaneously to provide a complete basis for anomaly determination.

[0072] Optionally, if a key indicator is a new indicator, i.e. there is no historical operating data, then collect the normal operating data of the indicator in the initial operating phase, such as 7 days, to quickly generate a temporary baseline as a transition, and automatically switch to the formal historical baseline after accumulating enough historical data.

[0073] S203. Match alarm strategies based on abnormal fluctuation indicators. Alarm strategies include threshold conditions, time windows, and business scenarios.

[0074] Alarm strategy refers to a set of pre-defined, flexibly configurable judgment rules for identifying abnormal situations that require operational intervention.

[0075] Among them, threshold conditions refer to the quantitative judgment criteria in the alarm strategy, including static numerical thresholds, such as error rate > 3% and call volume > 1000 times / minute; and dynamic trend thresholds, such as the number of errors doubling within 5 minutes and response time continuously increasing by more than 50% within 10 minutes.

[0076] The time window refers to the continuous observation period set in the alarm strategy, such as 10 minutes, which is used to filter out false judgments caused by instantaneous fluctuations. Only when the abnormal fluctuation indicator continuously meets the threshold condition within the period is it determined to meet the alarm strategy.

[0077] Understandably, a centralized strategy library is pre-built, including multiple alarm strategies preset for different business scenarios. Each strategy clearly indicates the corresponding business scenario, threshold conditions, time window, and applicable indicator type. Specifically, based on the identified abnormal fluctuation indicators, the target business scenario and indicator type are first extracted, such as the business indicator - payment success rate. Then, the strategy library is searched based on these attributes to filter out candidate alarm strategies that match the business scenario and indicator type of the abnormal fluctuation indicator. The duration for which the abnormal fluctuation indicator meets the preliminary abnormal conditions is statistically analyzed and compared with the time window in the candidate strategy. For example, if the candidate strategy's time window is 10 minutes, it needs to be confirmed whether the abnormal state has lasted for 10 minutes. Only if the duration meets the time window requirement can the next step of verification be performed. Depending on the threshold type of the candidate strategy, if it is a static numerical threshold, the real-time statistical value of the abnormal fluctuation indicator is compared with the threshold size, such as comparing an error rate of 3.5% with a threshold of >3%. If it is a dynamic trend threshold, the trend of the indicator's change within the time window is analyzed to see if it meets the threshold requirements, such as comparing an increase in the number of errors from 10 to 25 within 5 minutes with a doubling of the threshold. If all verifications are satisfied, the abnormal fluctuation indicator is determined to have successfully matched the corresponding alarm strategy.

[0078] Optionally, for alarm strategies containing complex logic, such as error rate > 3% and call volume > 1000 times / minute, response timeout or continuous increase in error count, after completing single-condition verification, first verify whether each sub-threshold condition in the complex strategy is met, and then make a comprehensive judgment based on the logic operators preset by the strategy.

[0079] S204. In response to abnormal fluctuation indicators meeting the corresponding alarm policy, generate a corresponding alarm event. The alarm event includes the target service, alarm reason, abnormal fluctuation indicator, and corresponding log data link.

[0080] Among them, an alarm event refers to a structured notification carrier containing complete abnormal information generated after the abnormal fluctuation index meets the alarm strategy.

[0081] The target service refers to the specific business service or technical component that triggers the alarm, such as payment interface service or user registration service.

[0082] The log data link is an accessible address pointing to the original log related to the exception. Through this link, you can directly jump to the complete log record of the time when the exception occurred.

[0083] Specifically, once the abnormal fluctuation metric passes the alarm policy matching verification, the specific service triggering the alarm is identified through the business identifier associated with the abnormal fluctuation metric, such as service name, interface ID, and component number. Combining the type of the abnormal fluctuation metric, its deviation from the historical baseline, and the core conditions of the matching alarm policy, a clear and easy-to-understand alarm reason description is generated, clearly explaining why the alarm was triggered. Key data such as metric name, real-time value, historical baseline normal range, deviation magnitude, and duration of the anomaly are recorded to fully present the abnormal metric status. Based on the timestamp of the anomaly, the target service identifier, and the business scenario tag, the corresponding original log set is retrieved, and a unique, accessible log data link is generated.

[0084] Furthermore, according to a preset unified format, the collected information is structured and assembled to form standardized alarm events. The alarm cause, complete details of abnormal fluctuation indicators are filled in sequentially, log data links are attached, and a summary of the system context when the anomaly occurs is added to generate the corresponding alarm event.

[0085] S205. Determine the corresponding notification channel based on the priority of the alarm event, so as to report the alarm event and alarm recovery event through the notification channel.

[0086] Among them, alarm event priority refers to the level standard that classifies alarms according to their impact on system operation and business operations.

[0087] Notification channels refer to the carriers used to push alarm-related information to operations and maintenance personnel, including instant messaging such as DingTalk and WeChat Work; communication such as SMS and telephone; and platform such as in-site messages of the operations and maintenance management platform.

[0088] Among them, alarm recovery events refer to structured information generated to inform operations and maintenance personnel that the problem has been resolved after the abnormal fluctuation indicators return to the normal range of the historical baseline and the system resumes normal operation.

[0089] Optionally, priority determination rules can be preset to clarify the priority levels corresponding to different impact scenarios. For example, high priority means that the anomaly directly leads to the interruption of core business, the unavailability of critical system functions, or the deviation of the abnormal indicator from the baseline by more than 50% and continues to expand, which may cause significant losses. Medium priority means that non-core business is affected and the abnormal indicator deviates from the baseline by 30%-50%. Low priority means that only some functions or non-critical indicators are abnormal and the indicator deviates from the baseline by less than 30%, with minimal impact on business operations.

[0090] Understandably, based on the above rules, the importance of the target service, the deviation of abnormal fluctuation indicators, and the scope of business impact in the alarm event are extracted to comprehensively determine the priority of the alarm event.

[0091] Specifically, once the priority of an alarm event is determined, the mapping rule base is retrieved to match the corresponding set of notification channels, and the corresponding operations and maintenance recipient group is extracted. The core information of the alarm event is then formatted and pushed according to the matched notification channel. After pushing, the push time, recipient, and channel status are recorded. If a push fails on a certain channel, it automatically switches to a backup channel of the same priority for supplementary pushing.

[0092] Furthermore, continuously monitor abnormal fluctuation indicators. When an indicator continuously meets the normal range of returning to the historical baseline for a preset duration, generate an alarm recovery event, including information such as the target service, alarm recovery time, and the current status of the indicator. Match the alarm event to the corresponding notification channel according to its priority, push it to the same recipient group, and simultaneously update the alarm event status to "recovered".

[0093] The alarm generation method based on log data provided in this application acquires multi-source log data, enabling a comprehensive understanding of system operation and business activities. This avoids information gaps from single data sources, and comprehensive analysis can uncover potential correlations, providing rich evidence for accurate anomaly detection. Key indicators of the target business scenario reflect the system's operation and results under specific business conditions, eliminating irrelevant data interference and making subsequent analysis and processing more efficient and accurate. Historical baselines are statistically derived from key indicator data during normal system operation; comparison with these baselines clearly determines whether current indicators are abnormal, providing an objective reference for anomaly detection. By comparing and identifying abnormal fluctuation indicators, problems can be quickly and accurately located, providing early warnings and improving system reliability and stability. Linking alarm strategies with business scenarios allows for the formulation of rules based on business characteristics, making alarms more targeted and practical. Time window settings consider the time characteristics of indicators, avoiding false alarms due to instantaneous fluctuations and improving accuracy. Alarm strategies include multi-dimensional information such as threshold conditions, time windows, and business scenarios, allowing for flexible settings to meet diverse needs and ensure timely and effective alarms. The generated alarm events include information such as the target service, cause, abnormal indicators, and log links, providing comprehensive details for operations and maintenance personnel to quickly locate problems, analyze root causes, and improve fault handling efficiency. Appropriately selecting notification channels can improve alarm handling efficiency, enabling operations and maintenance personnel to respond promptly, reducing the impact of faults on business operations, ensuring business continuity and stability, and allowing different alarm events to have independently configured notification channels to adapt to individual needs.

[0094] As an optional implementation, based on the above embodiments, the steps for generating and updating the historical baseline include:

[0095] Obtain normal operation data of key indicators for a continuous preset period under the target business scenario;

[0096] The average value, variance of fluctuation, and peak range are calculated based on normal operation data to construct an initial historical baseline;

[0097] Periodically acquire the latest normal operation data at the current moment;

[0098] The initial historical baseline is updated iteratively based on the latest normal operation data.

[0099] Among them, normal operation data refers to the valid data generated by key indicators under the target business scenario during the period when the system is fault-free and the business is running smoothly.

[0100] The preset period refers to the pre-set duration of continuous data collection, such as 7 days. The duration can be flexibly adjusted according to the characteristics of the business scenario.

[0101] Iterative updates refer to periodically incorporating the latest normal operation data based on the initial historical baseline, and dynamically adjusting the average value, variance, and peak range of the baseline to ensure that the baseline always remains consistent with the current business operation status.

[0102] Specifically, the boundaries of the target business scenario are clearly defined, such as e-commerce order payment business and corresponding key indicators, such as payment success rate and payment interface response time. Relevant log data for this business scenario is then selected. Invalid data is removed using a data filtering mechanism, and the aforementioned valid data is continuously collected according to a preset period. By summarizing all valid data within the period and taking the arithmetic mean, a baseline value for the normal operation of the indicator is obtained. Based on the average value, the deviation of each data point from the average value is statistically analyzed to determine the dispersion range of normal fluctuations in the indicator. The maximum and minimum values ​​of the data within the period are extracted to form the extreme value range for normal operation of the indicator. The average value, variance, and peak range are integrated to form the initial historical baseline of this key indicator.

[0103] Furthermore, a baseline update cycle is set, such as every day at midnight. At the update node, operational data for key indicators of the target business scenario from the previous cycle are collected. A filtering mechanism is then used to remove faulty periods, invalid data, and transient outliers, retaining only the latest operational data. If significant business changes occur during the update cycle, such as system upgrades or business process adjustments, the data collection time is extended.

[0104] Furthermore, the newly collected normal operation data is merged with the original preset periodic data to recalculate the average value, variance of fluctuation, and peak range of the indicators. Based on the recalculation results, the initial historical baseline is adjusted to generate an updated historical baseline.

[0105] Optionally, machine learning models (such as LSTM or Prophet) can be introduced to model historical indicator data, dynamically predict the normal baseline range, and automatically adjust the baseline threshold based on real-time business load. LSTM neural networks can be used to perform time-series modeling of historical indicator data, such as error rate and call latency, to predict the normal baseline range. The baseline threshold is then dynamically adjusted based on real-time business load, such as transaction volume and system resource utilization. Abnormal fluctuations are identified by comparing the deviation between real-time indicators and the dynamic baseline, such as using Z-score.

[0106] The alarm generation method based on log data provided in this application constructs an initial historical baseline by acquiring continuous, preset-period normal operation data of key indicators under the target business scenario. This accurately reflects the normal fluctuation range of business indicators, providing a solid and reliable data foundation for the historical baseline. The initial historical baseline is constructed by calculating the average value, variance, and peak range based on the normal operation data. The combination of these three factors comprehensively and accurately characterizes the normal features of the indicators, resulting in a high accuracy of the initial historical baseline. A dynamic update mechanism is established by periodically acquiring the latest normal operation data at the current moment and iteratively updating the initial historical baseline based on this data. This allows the method to keep up with changes in business and systems, adapt to new operating states, and avoid misjudgments due to outdated baselines.

[0107] As an optional implementation, after generating the corresponding alarm event based on the above embodiments, the method further includes:

[0108] The adaptive suppression parameters are determined based on the type of alarm event, the identifier of the target service, and the target business scenario. The adaptive suppression parameters include the suppression time window and the alarm frequency threshold.

[0109] When the number of times a short-term, high-frequency alarm event of the same type is triggered within the suppression time window reaches the alarm frequency threshold, the short-term, high-frequency alarms are merged into a continuous alarm state, and the matching conditions of the suppression rules are recorded. The matching conditions of the suppression rules include the alarm trigger frequency, related indicators, and time window.

[0110] Among them, the adaptive suppression parameter refers to the suppression rule configuration item that is dynamically matched according to the characteristics of the alarm event, which can adapt to the alarm patterns of different business scenarios and service types.

[0111] The suppression time window refers to a set continuous observation period, such as 3 minutes, used to define the time range of short-term high-frequency alarms. Only the number of alarms of the same type for the same service within this window is counted.

[0112] The alarm frequency threshold refers to the maximum number of alarms allowed to be triggered within the suppression time window, such as 3 times.

[0113] Among them, the continuous alarm status refers to the unified alarm status identifier generated by merging the same type of alarms that are triggered frequently in a short period of time, rather than pushing each alarm individually, which simplifies the handling of operations and maintenance personnel.

[0114] Optionally, a suppression parameter configuration rule base can be constructed, pre-setting parameter benchmarks corresponding to different dimension combinations. For example, core fault alarms correspond to a shorter suppression time window, such as 3 minutes, and a lower frequency threshold, such as 3 times. Non-core fault alarms correspond to a longer time window, such as 10 minutes, and a higher frequency threshold, such as 5 times.

[0115] Specifically, after an alarm event is generated, its alarm type, target service identifier, and target business scenario are extracted. The configuration rule base is retrieved, and the corresponding suppression time window and alarm frequency threshold are dynamically matched to form the adaptive suppression parameters specific to the alarm event.

[0116] Furthermore, using suppression time windows as units, the number of times the same target service and the same type of alarm are triggered is counted in real time. When the number of alarm triggers within the suppression time window reaches or exceeds the alarm frequency threshold, the suppression logic is immediately activated. Subsequent single alarms are stopped from being pushed to operations and maintenance personnel. Multiple alarms that have already been triggered and any subsequent alarms of the same type that may be triggered are merged into a continuous alarm status, generating a consolidated alarm notification.

[0117] Furthermore, the matching conditions of the suppression rules are recorded synchronously, including the current alarm trigger frequency, i.e., the actual number of triggers / suppression time window; the associated abnormal fluctuation indicators; the duration of the suppression time window and the alarm frequency threshold, forming a complete suppression trajectory.

[0118] Optionally, after the suppression time window ends, the suppression status of this type of alarm is lifted, and the normal alarm triggering and push mechanism is restarted. If the abnormal fluctuation indicator still has not returned to normal after the window ends, a new suppression time window will automatically start, and the above statistical and suppression logic will be repeated. If the indicator returns to the normal range after the window ends, there is no need to continue suppression, and if this type of alarm is triggered again later, it will be handled as a new independent event.

[0119] The alarm generation method based on log data provided in this application addresses the issue that during network and system operation, certain faults or anomalies may cause a large number of alarm events of the same type for the same target service to be triggered in a short period of time, forming an alarm storm. By setting adaptive suppression parameters, when the number of times alarm events of the same type for the same target service are triggered within the suppression time window reaches the alarm frequency threshold, short-term high-frequency alarms are merged into a continuous alarm state. This effectively prevents maintenance personnel from being overwhelmed by massive amounts of duplicate alarms, allowing them to focus on handling truly important alarms.

[0120] As an optional implementation, based on the above embodiments, acquiring multi-source log data includes:

[0121] Based on the corresponding log collection requirements, a preset collection tool is used to acquire multi-source log data through a breakpoint resume mechanism.

[0122] Standardize multi-source log data and add business and environment tags. Standardization includes unifying log formats, mapping fields, filtering invalid data, and deduplicating duplicate logs.

[0123] Deduplication of logs involves retaining only one log record when logs from the same source have the same hash value and the generation time difference does not exceed a preset time threshold. At the same time, the number of deduplicated logs and their source information are recorded.

[0124] Among them, the preset collection tool refers to the professional log collection software, such as Filebeat, that is pre-selected based on the collection scenario of multi-source logs, such as file logs and network logs.

[0125] Among them, business tags refer to the markers added to the logs to identify the business scope to which they belong, such as payment business, which are generated based on the business characteristics in the logs.

[0126] Among them, environment tags refer to the tags added to logs to identify the operating environment, such as production environment, test environment, and server configuration and deployment environment information generated based on the log source.

[0127] The preset time threshold refers to the standard used to define the time range of duplicate logs, such as 30 seconds.

[0128] Specifically, the corresponding preset collection tool is selected based on the source type and storage method of the logs. During the collection process, the collection status and progress of each log are recorded in real time. If a network interruption or abnormal tool stoppage occurs, the historical collection records are automatically read after the tool resumes operation, and collection continues from the last collected log.

[0129] Furthermore, for heterogeneous logs from different sources, the format is standardized to a preset format, and fields with the same meaning but different names are unified into standard field names. Core fields, such as service name, log level, and event description, are extracted from each source log and mapped to a unified field system according to preset field mapping rules. Log data with incorrect formatting, no practical meaning, or that does not meet business requirements is removed.

[0130] Furthermore, a hash calculation is performed on the core content of each log entry, such as the log body, service name, and event type, to generate a unique log hash value. Then, duplicate logs are determined based on the logic of the same source, identical hash value, and time difference. Two logs are considered duplicates if they meet the criteria of the same source, identical hash value, and a generation time difference not exceeding a preset time threshold. For logs determined to be duplicates, only the earliest generated one is retained, and the number of deduplicated logs, source information, and deduplication time are recorded.

[0131] Furthermore, after standardization, business feature information is extracted from the logs and matched with a pre-defined business tag mapping library to assign corresponding business tags to the logs. Based on the log collection source information, environment tags such as production environment are matched and added.

[0132] The alarm generation method based on log data provided in this application uses a preset collection tool to acquire multi-source log data based on corresponding log collection requirements through a breakpoint resume mechanism. This effectively addresses unexpected situations such as network instability and interruptions during collection, preventing data loss and ensuring the integrity and continuity of the acquired log data. Standardization processing of the acquired multi-source log data eliminates differences in format and structure between different sources, ensuring consistency and standardization. The deduplication function identifies and removes duplicate logs from the same source with identical hash values ​​and generation time differences not exceeding a preset time threshold, retaining only one log record and recording the number and source information of the deduplicated logs. This effectively reduces data redundancy, saves storage space, and improves data processing efficiency. Adding business and environment tags to the standardized multi-source log data allows for classification and management of log data according to business type and operating environment.

[0133] As an optional implementation, based on the above embodiments, it further includes:

[0134] Obtain the recipient's current work status through the notification channel. The work status includes online status, task load, and historical response speed.

[0135] When an alarm event is at a preset priority level, if the recipient does not respond to the first notification within a preset time period, the system switches to a preset backup notification channel and notifies the backup recipient group.

[0136] The preset duration refers to the initial notification response time limit set for the alarm, such as 10 minutes.

[0137] Among them, the preset backup notification channel refers to the backup push carrier configured for alarms with preset priority levels, such as DingTalk as the main channel and telephone as the backup channel.

[0138] Among them, the backup receiver group refers to a pre-designated backup maintenance personnel team that can handle the corresponding alarms.

[0139] Specifically, the system determines whether a recipient is online and capable of receiving messages by checking their login status on the operations and maintenance platform and online status via notification channels such as DingTalk / WeChat Work. It integrates with the operations and maintenance task management system to track the number of alarm events and operations and maintenance work orders that the recipient has currently claimed but not completed, categorizing them into load levels according to preset thresholds (e.g., 0-2 alarms for low load, 3-5 for medium load, and 6 or more for high load). The system extracts records of past handling of similar priority alarms from the alarm log database, calculating the average time from receiving the notification to the first response (e.g., clicking "viewed" or "processing"), establishing a historical response speed baseline (e.g., an average response time of 8 minutes).

[0140] Furthermore, when an alarm event is determined to be of a preset priority level, an initial notification is pushed through the original matching notification channel, and a response timer is started. The recipient's response status is monitored in real time within a preset duration. If the recipient completes the response operation within the timer period, the subsequent process is terminated. If no response is received after the timer expires, it is determined that the initial notification has not been responded to. The original notification channel is switched to a pre-configured backup channel, and the complete alarm event information is re-pushed. Simultaneously, a member list is extracted from the preset backup recipient group, and the alarm event is pushed to the backup recipient group through a combination of the original notification channel and the backup channel.

[0141] The alarm generation method based on log data provided in this application embodiment obtains the current working status of the recipient according to the notification channel, enabling a comprehensive understanding of the recipient's feasibility and efficiency in handling alarms. This allows for a more accurate determination of the effectiveness of the initial notification, providing a basis for subsequent decisions and ensuring that alarm information is delivered to the recipient in the most appropriate way. When an alarm event has a preset priority level, it indicates that the alarm is urgent and requires timely handling. If the recipient does not respond to the initial notification within a preset time period, the system quickly switches to a preset backup notification channel and notifies the backup recipient group. This effectively avoids delays caused by the recipient not seeing or handling the alarm in a timely manner, ensuring that urgent alarms receive attention and handling immediately, and reducing the impact of system failures on business operations.

[0142] As an optional implementation, based on the above embodiments, the alarm event also includes troubleshooting suggestions, and the method further includes:

[0143] Obtain alarm handling solutions of the same type as the alarm event from the historical alarm event database;

[0144] Based on the link context information and original log characteristics corresponding to the abnormal fluctuation indicators, investigation steps are extracted from the alarm handling scheme.

[0145] Based on the priority of alarm events and the target business scenario, alarm events are embedded after being sorted by the priority of the investigation steps, so that one-click jump to the relevant investigation tools or configuration pages can be made.

[0146] Among them, the troubleshooting suggestions refer to targeted troubleshooting guidelines generated based on historical processing experience and the current anomaly context. They include troubleshooting steps with priority order and tool / page jump entry points, and directly embed alarm events.

[0147] The historical alarm event database refers to a database that stores all past alarm events and their corresponding handling processes, covering information such as alarm type, abnormal indicators, handling solutions, troubleshooting steps, and resolution results.

[0148] Among them, the link context information refers to the details of the business call link related to the abnormal fluctuation index, including service dependencies, interface call order, data transmission path and the running status of each node.

[0149] Among them, the original log characteristics refer to the key information unique to the original log during the period when the exception occurred, such as specific error codes, key exception fields, and special error descriptions.

[0150] The priority of the investigation steps refers to the level of the investigation steps based on their efficiency, operational complexity, and business relevance (the higher the priority, the earlier they are executed).

[0151] Specifically, identify the alarm type, abnormal fluctuation index type, and target business scenario of the current alarm event, and use these as search keywords. Search the historical alarm event database, filter out historical alarm events that are completely identical or highly similar to the current alarm event's classification identifier, and extract the complete handling solutions corresponding to these historical events, including fault cause analysis, troubleshooting operations performed, final solutions, and effect feedback, forming a set of similar alarm handling solutions.

[0152] Furthermore, for the acquired set of similar alarm handling solutions, based on the link context information, troubleshooting steps related to the current business call link and service dependencies are filtered out. For example, if the current alarm is a payment interface call failure, and the link context shows a dependency on a database service, then steps related to checking the database connection status and verifying database permission configuration from historical solutions are extracted first. Then, combined with the characteristics of the original logs, targeted steps are further filtered. For example, if timeout error codes appear in the original logs, then troubleshooting steps related to timeouts, such as checking interface timeout parameter configuration and investigating network transmission latency, are extracted, and general steps irrelevant to the current log characteristics are eliminated, forming a core set of troubleshooting steps adapted to the current fault scenario.

[0153] Furthermore, steps that can directly verify core fault points, such as querying exception log details, are listed as the highest priority; simple and quick steps are listed as medium priority; and complex steps requiring cross-departmental collaboration are listed as low priority. Based on the priority of the current alarm event and the target business scenario, the core troubleshooting steps are sorted. The sorted troubleshooting steps are then embedded into the alarm event, with each step accompanied by a brief operation description and the corresponding troubleshooting tool / configuration page redirection entry, while also indicating the step priority.

[0154] The alarm generation method based on log data provided in this application retrieves alarm handling solutions of the same type as the current alarm event from the historical alarm event database. This provides valuable experience reference for operations and maintenance personnel, avoiding repeated exploration and trial and error, quickly locating the cause of the fault and taking corresponding solutions, thereby greatly shortening the fault handling time. Based on the link context information and original log characteristics corresponding to the abnormal fluctuation indicators, the troubleshooting steps are extracted from the alarm handling solution, ensuring that the extracted steps are highly relevant to the current alarm event. According to the priority of the alarm event and the target business scenario, the alarm events are embedded after being sorted by the priority of the troubleshooting steps, and one-click jump to the relevant troubleshooting tools or configuration pages is supported, providing great convenience for operations and maintenance personnel. Operations and maintenance personnel do not need to manually search for and open relevant tools or pages; they can directly enter the corresponding troubleshooting environment and quickly execute the troubleshooting steps by simply clicking the one-click jump button, further improving the efficiency of fault handling.

[0155] Figure 3 A flowchart illustrating another embodiment of the alarm generation method based on log data provided in this application is shown below. Figure 3 As shown, the alarm generation method based on log data provided in this embodiment includes the following steps:

[0156] S301. Obtain normal operation data of key indicators for a continuous preset period under the target business scenario.

[0157] S302. Calculate the average value, variance of fluctuation, and peak range based on normal operation data to construct an initial historical baseline.

[0158] S303. Obtain multi-source log data and extract key indicators for the target business scenario from the multi-source log data. Key indicators include operational indicators and business indicators.

[0159] S304. Determine abnormal fluctuation indicators based on key indicators and the historical baselines corresponding to each indicator.

[0160] S305. Alarm strategies are matched based on abnormal fluctuation indicators. The alarm strategies include threshold conditions, time windows, and business scenarios.

[0161] S306. In response to abnormal fluctuation indicators meeting the corresponding alarm policy, generate a corresponding alarm event. The alarm event includes the target service, alarm reason, abnormal fluctuation indicator, and corresponding log data link.

[0162] S307. Determine the adaptive suppression parameters based on the type of alarm event, the identifier of the target service, and the target business scenario. The adaptive suppression parameters include the suppression time window and the alarm frequency threshold.

[0163] S308. Determine the corresponding notification channel based on the priority of the alarm event, so as to report the alarm event and alarm recovery event through the notification channel.

[0164] S309. When the number of times an alarm event of the same type for the same target service is triggered within the suppression time window reaches the alarm frequency threshold, short-term high-frequency alarms are merged into a continuous alarm state, and the matching conditions of the suppression rules are recorded. The matching conditions of the suppression rules include alarm trigger frequency, associated indicators and time window.

[0165] S310: Periodically acquire the latest normal operation data at the current moment.

[0166] S311. Iteratively update the initial historical baseline based on the latest normal operation data.

[0167] In this embodiment, the implementation method and technical effect of S301-S311 are similar to those of the corresponding solutions in the above embodiments, and will not be repeated here.

[0168] Figure 4 A schematic diagram of the alarm generation device based on log data provided in this application is shown below. Figure 4 As shown, the alarm generation device 40 based on log data provided in this embodiment includes: an acquisition module 41, a determination module 42, a matching module 43, and a generation module 44.

[0169] The system includes: an acquisition module 41 for acquiring multi-source log data and extracting key indicators for the target business scenario from the multi-source log data, including operational and business indicators; a determination module 42 for determining abnormal fluctuation indicators based on the key indicators and their corresponding historical baselines; a matching module 43 for matching corresponding alarm policies based on the abnormal fluctuation indicators, including threshold conditions, time windows, and business scenarios; a generation module 44 for generating corresponding alarm events in response to abnormal fluctuation indicators meeting the corresponding alarm policies, including the target service, alarm cause, abnormal fluctuation indicator, and corresponding log data link; and a determination module 42 for determining the corresponding notification channel based on the priority of the alarm event, so as to report the alarm event and alarm recovery event through the notification channel.

[0170] The alarm generation device based on log data provided in this embodiment can execute... Figure 2 The implementation principles and technical effects of the methods shown are similar, and will not be repeated here.

[0171] Optionally, the alarm generation device based on log data provided in this embodiment further includes a calculation module and an update module.

[0172] Correspondingly, the acquisition module 41 is also used to acquire the normal operation data of key indicators for a continuous preset period under the target business scenario; the calculation module is used to calculate the average value, fluctuation variance and peak range based on the normal operation data to construct the initial historical baseline; the acquisition module 41 is also used to periodically acquire the latest normal operation data at the current moment; the update module is used to iteratively update the initial historical baseline based on the latest normal operation data.

[0173] Optionally, the alarm generation device based on log data provided in this embodiment further includes a merging module and a recording module.

[0174] Correspondingly, the determination module 42 is also used to determine adaptive suppression parameters based on the type of alarm event, the identifier of the target service, and the target business scenario. The adaptive suppression parameters include the suppression time window and the alarm frequency threshold. The merging module is used to merge short-term high-frequency alarms into a continuous alarm state when the number of triggers of the same type of alarm event for the same target service within the suppression time window reaches the alarm frequency threshold. The recording module is used to record the matching conditions of the suppression rules. The matching conditions of the suppression rules include the alarm trigger frequency, related indicators, and time window.

[0175] Optionally, the acquisition module 41, when acquiring multi-source log data, is specifically used for: using a preset acquisition tool to acquire multi-source log data based on the corresponding log acquisition requirements through a breakpoint resume mechanism; performing standardization processing on the multi-source log data and adding business tags and environment tags. The standardization processing includes log format unification, field mapping, invalid data filtering, and duplicate log deduplication; duplicate log deduplication includes retaining only one log data when logs from the same source have the same hash value and the generation time difference does not exceed a preset time threshold, while recording the number of duplicate logs and their source information.

[0176] Optionally, the alarm generation device based on log data provided in this embodiment further includes a switching notification module.

[0177] Correspondingly, the acquisition module 41 is also used to acquire the current working status of the recipient according to the notification channel. The working status includes online status, task load and historical response speed. The switching notification module is used to switch to the preset backup notification channel and notify the backup recipient group when the alarm event is of preset priority level and the recipient does not respond to the first notification within a preset time.

[0178] Optionally, the alarm generation device based on log data provided in this embodiment also includes an embedded module.

[0179] Correspondingly, the acquisition module 41 is also used to acquire alarm handling solutions of the same type as the alarm event from the historical alarm event library; extract investigation steps from the alarm handling solutions based on the link context information and original log characteristics corresponding to the abnormal fluctuation index; and embed the alarm event according to the priority of the alarm event and the target business scenario, and then embed the alarm event in order of priority of the investigation steps, so as to enable one-click jump to the relevant investigation tool or configuration page.

[0180] Figure 5 A schematic diagram of the alarm generation device based on log data provided in this application. Figure 5 As shown, the alarm generation device 50 based on log data provided in this embodiment includes a processor 51 and a memory 52. ​​The processor 51 and the memory 52 are connected via a bus and communicate with each other.

[0181] In the specific implementation process, the processor 51 executes the computer execution instructions stored in the memory 52, causing the processor 51 to perform the above-described method.

[0182] The specific implementation process of processor 51 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0183] In the above embodiments, it should be understood that the processor 51 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0184] The memory 52 may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0185] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0186] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0187] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0188] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0189] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0190] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0191] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0192] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0193] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0195] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for generating alarms based on log data, characterized in that, The method includes: Acquire multi-source log data and extract key indicators for the target business scenario from the multi-source log data, including operational indicators and business indicators; The abnormal fluctuation indicators are determined based on the key indicators and the historical baselines corresponding to each indicator. Based on the abnormal fluctuation index, a corresponding alarm strategy is matched. The alarm strategy includes threshold conditions, time windows, and business scenarios. In response to the abnormal fluctuation index meeting the corresponding alarm policy, a corresponding alarm event is generated. The alarm event includes the target service, the alarm reason, the abnormal fluctuation index, and the corresponding log data link. The corresponding notification channel is determined based on the priority of the alarm event, so that the alarm event and alarm recovery event are reported through the notification channel.

2. The method according to claim 1, characterized in that, The steps for generating and updating the historical baseline include: Obtain normal operation data of key indicators for a continuous preset period under the target business scenario; The average value, variance of fluctuation, and peak range are calculated based on the normal operation data to construct an initial historical baseline; Periodically acquire the latest normal operation data at the current moment; The initial historical baseline is iteratively updated based on the latest normal operation data.

3. The method according to claim 1, characterized in that, After generating the corresponding alarm event, the process also includes: Based on the type of alarm event, the identifier of the target service, and the target business scenario, an adaptive suppression parameter is determined, which includes a suppression time window and an alarm frequency threshold. When the number of times an alarm event of the same type for the same target service is triggered within the suppression time window reaches the alarm frequency threshold, short-term high-frequency alarms are merged into a continuous alarm state, and the matching conditions of the suppression rules are recorded. The matching conditions of the suppression rules include alarm trigger frequency, associated indicators, and time window.

4. The method according to claim 1, characterized in that, The acquisition of multi-source log data includes: Based on the corresponding log collection requirements, a preset collection tool is used to acquire multi-source log data through a breakpoint resume mechanism. The multi-source log data is standardized, and business tags and environment tags are added. The standardization process includes log format unification, field mapping, invalid data filtering, and duplicate log deduplication. The deduplication of duplicate logs includes retaining only one log data when the log hash values ​​from the same source are the same and the generation time difference does not exceed a preset time threshold, while recording the number of deduplicated logs and their source information.

5. The method according to claim 1, characterized in that, Also includes: The recipient's current work status is obtained through the notification channel, including online status, task load, and historical response speed. When the alarm event is of a preset priority level, if the recipient does not respond to the first notification within a preset time period, the system switches to a preset backup notification channel and notifies the backup recipient group.

6. The method according to claim 1, characterized in that, The alarm event also includes troubleshooting suggestions, and the method further includes: Obtain alarm handling solutions of the same type as the alarm event from the historical alarm event database; Based on the link context information and original log characteristics corresponding to the abnormal fluctuation indicators, investigation steps are extracted from the alarm handling scheme. Based on the priority of the alarm events and the target business scenario, the alarm events are embedded after being sorted according to the priority of the investigation steps, so that one-click jump to the relevant investigation tools or configuration pages can be made.

7. An alarm generation device based on log data, characterized in that, include: The acquisition module is used to acquire multi-source log data and extract key indicators of the target business scenario from the multi-source log data. The key indicators include operational indicators and business indicators. The determination module is used to determine abnormal fluctuation indicators based on the key indicators and the historical baselines corresponding to each indicator. The matching module is used to match the corresponding alarm strategy based on the abnormal fluctuation index. The alarm strategy includes threshold conditions, time windows and business scenarios. The generation module is used to generate a corresponding alarm event in response to the abnormal fluctuation index meeting the corresponding alarm policy. The alarm event includes the target service, alarm reason, abnormal fluctuation index and corresponding log data link. The determining module is further configured to determine the corresponding notification channel based on the priority of the alarm event, so as to report the alarm event and alarm recovery event through the notification channel.

8. An alarm generation device based on log data, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images