Evidence-driven hierarchical self-adaptive false alarm noise reduction method for operation and maintenance scenarios

Abstract

The application discloses an evidence-driven hierarchical self-adaptive false alarm noise reduction method for operation and maintenance scenarios, and belongs to the technical field of intelligent operation and maintenance and network security. The method comprises the following steps: 1) uniformly decoding, field mapping, abnormality cleaning and multi-modal feature construction are performed on multi-source alarms; 2) a lightweight model with uncertainty gating is used to perform three-way distribution of real attacks, business false alarms and difficult review of the alarms; 3) the difficult review type alarm is automatically associated with operation and maintenance context evidence, asset baseline and exposure surface evidence, similar event evidence and subsequent behavior verification evidence to form a structured evidence vector; and 4) the alarm structured features and the structured evidence vector are input into a large language model adapted to the field for review, and the large language model is updated through supervision and fine-tuning in combination with the final disposal result. The method can reduce the false alarm rate and the artificial disposal pressure, and can also take into account the accuracy, interpretability and reasoning cost.

Description

Technical Field

[0001] This invention relates to the fields of intelligent operation and maintenance and network security technology, and in particular to an evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios. Background Technology

[0002] In the operation and maintenance monitoring of modern large-scale, distributed IT systems, automated security detection and alarm systems are crucial for ensuring business continuity and security. Alarm noise reduction technology aims to identify and filter false alarms from massive amounts of monitoring data, thereby improving the effectiveness of alarms. With the rapid development of information technology, traditional alarm noise reduction methods have revealed a series of limitations and urgently need improvement.

[0003] Early alert handling heavily relied on static rules and thresholds. Operations personnel manually configured filtering rules based on experience, such as whitelists of specific source IPs and fixed frequency thresholds, to block known false alarm sources. While simple to implement, this approach lacked flexibility, failing to adapt to dynamic business changes or identify new false alarm patterns, resulting in a bloated rule base and high maintenance costs. This issue has been extensively discussed in numerous studies, highlighting the limitations of this approach in handling complex and dynamic environments (Bhukar K, Kumar H, Mahindru R, et al. Dynamic alertsuppression policy for noise reduction in aiops[C] / / Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice. 2024: 178-188.)(Yu Q, Zhao N, Li M, et al. A survey on intelligent management of alerts and incidents in IT services[J]. Journal of Network and Computer Applications, 2024, 224: 103842.).

[0004] To improve adaptability, researchers have introduced traditional machine learning methods. By extracting the temporal and statistical features of alarms, clustering (such as K-means, DBSCAN) or classification algorithms (such as decision trees, random forests) are used to group or distinguish alarms. Although machine learning-based alarm denoising methods can improve the system's adaptability to a certain extent, they still heavily rely on the quality of feature engineering and lack the ability to understand the deep semantic information and business context contained in alarm texts (Luo C, Lou JG, Lin Q, et al. Correlating events with time series for incident diagnosis[C] / / Proceedings of the 20th ACM SIGKDD internationalconference on Knowledge discovery and data mining. 2014: 1583-1592.)(Soldani J, Brogi A. Anomaly detection and failure root cause analysis in (micro)service-based cloud applications: A survey[J]. ACM Computing Surveys (CSUR),2022, 55(3): 1-39.).

[0005] With the development of natural language processing technology, deep learning-based text analysis methods have been applied to the parsing of alarm logs. This method further attempts to understand alarms at the text level, using recurrent neural networks (RNNs), convolutional neural networks (CNNs), or Transformer encoders to automatically extract features and classify alarm messages. While deep learning methods improve processing power, they also face challenges such as a lack of interpretability, reliance on large amounts of labeled data, and insufficient integration of business knowledge (Du M, Li F, Zheng G, et al. Deeplog: Anomaly detection and diagnosis from system logs through deep learning[C] / / Proceedings of the 2017 ACM SIGSACconference on computer and communications security. 2017: 1285-1298.).

[0006] In recent years, knowledge graphs have been introduced to model the relationships between entities and use graph reasoning to identify anomalies. This method can reveal the correlation between alarms through reasoning and effectively reduce false alarms. However, its construction, maintenance and updating still heavily rely on domain experts, and its ability to process unstructured alarm information is relatively weak (Chen P, Qi Y, Zheng P, et al. Causeinfer: Automatic and distributed performance diagnosis with hierarchicalcausality graph in large distributed systems[C] / / IEEE INFOCOM 2014-IEEEConference on Computer Communications. IEEE, 2014: 1887-1895.).

[0007] Recently, general-purpose artificial intelligence technologies, represented by Large Language Models (LLMs), have demonstrated powerful natural language understanding and reasoning capabilities. In the security field, there have been preliminary explorations of directly applying general-purpose LLMs (such as the GPT series) to tasks such as log analysis and vulnerability report interpretation. However, directly applying general-purpose LLMs without domain adaptation to operational alarm noise reduction still presents challenges, especially in handling specific business logic, internal system naming conventions, and real-time generation constraints (Yang S, Zheng X, Zhang X, et al. Large language models for network intrusiondetection systems: Foundations, implementations, and future directions[J]. arXiv preprint arXiv:2507.04752, 2025.). Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing operation and maintenance security alarm noise reduction technologies, which generally rely on static rule thresholds, single-model semantic judgment, or simple confidence filtering. These technologies have weaknesses such as weak adaptability to business changes and environmental drift, high risk of false alarms at the boundary, high cost of full inference of large models, and lack of verifiable evidence to support the judgment conclusions. This invention proposes an evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios.

[0009] This invention employs a technical path of "small-model uncertainty gating and triage—evidence generation and behavioral verification completion—large-model difficulty assessment with evidence constraints—large-model supervised fine-tuning," upgrading alarm assessment from relying solely on rule triggers or semantic judgments to consistency determination based on evidence chains. Furthermore, during operation, large-model supervised fine-tuning continuously enhances the model's adaptability to business changes, environmental drift, and evolving attack methods, achieving high-precision, low-cost, and interpretable / traceable false alarm reduction.

[0010] The objective of this invention is achieved through the following technical solution: an evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios, comprising the following steps:

[0011] (1) Alarm data preprocessing and multimodal feature construction: The original alarm data from security devices, log systems and monitoring platforms are preprocessed in a unified manner, including decoding, parsing, cleaning outliers and normalizing the format of the original alarm data; on this basis, alarm structured features for semantic understanding and pattern recognition are extracted from the alarm data. The alarm structured features include basic field features, time series statistical features, contextual features and alarm text semantic features, and a structured alarm sample dataset is constructed.

[0012] (2) Small model coarse screening of uncertainty gating: Based on the structured alarm input sample obtained in step (1), the first layer of lightweight alarm discrimination model is constructed to output the false alarm probability and uncertainty measure value of real-time alarm; according to the false alarm probability and uncertainty measure value, gating and hierarchical diversion are performed to divide the alarm into real attack type, business false alarm type and difficult review type.

[0013] (3) Construction of review input driven by evidence generation and behavior verification: For the difficult review alarms obtained in step (2), the evidence generation and behavior verification process is triggered to automatically associate and aggregate multi-source evidence elements. The multi-source evidence elements include operation and maintenance context evidence, asset baseline and exposure surface evidence, similar event evidence, and subsequent behavior verification evidence. The multi-source evidence elements come from multiple data sources, including the automatic association change management system, release pipeline system, asset list library, historical communication log library, historical disposal library, work order closed-loop library, and application audit log library. All evidence elements are standardized into structured evidence vectors to support subsequent review judgment.

[0014] (4) Large model verification and supervised fine-tuning adaptive update with evidence constraints: For difficult verification classes, the structured evidence vector obtained in step (3) and the alarm structured features obtained in step (1) are adapted to the input domain of the large language model, and only the difficult verification class alarms are evaluated a second time; and supervised fine-tuning samples are constructed to perform supervised fine-tuning adaptive update with evidence consistency constraints on the large language model.

[0015] Further, step (1) includes the following sub-steps:

[0016] (1.1) Unified decoding of multi-source alarms: alarms from security devices, log systems and monitoring platforms are collected and accessed. For different output protocols and encoding formats, a unified decoding mechanism is used to complete format recognition and message parsing, and alarms from various sources are unified into a set of original fields to achieve input consistency.

[0017] (1.2) Field parsing and standard schema mapping: Construct field mapping functions to map synonymous fields in different systems to standard fields under a unified standard schema; parse and extract nested alarms; use missing fields to fill missing fields and use missing flags in combination; and perform type validation and format calibration on key fields such as time, IP, port, and protocol to ensure that fields are comparable and statistically valid.

[0018] (1.3) Anomaly cleaning and format normalization: After completing schema alignment, this embodiment performs anomaly cleaning and format normalization on alarm data. Anomaly cleaning includes, but is not limited to, correcting, removing or marking records such as illegal IPs, port out of bounds, protocol anomalies, and timestamp drift. Format normalization includes unifying the time field to the agreed time zone and format, standardizing the protocol field to the standard enumeration, and unifying the expression of IPv4 / IPv6.

[0019] (1.4) Structured Sample Construction: Based on the cleaned alarm records, structured samples are constructed to form alarm structured features consisting of basic field features, time series statistical features, contextual features, and alarm text semantic features. The basic field features include at least alarm type, triggering rule, source address, target address, communication protocol, and port information. Next, a structured alarm sample dataset is constructed to provide high-quality input for subsequent domain annotation and large language model fine-tuning.

[0020] Furthermore, step (2) includes the following sub-steps:

[0021] (2.1) Lightweight model construction: Select a lightweight model as the first-layer discriminator and train it based on the structured alarm sample dataset constructed in step (1); define a unified output specification, and the model output shall include at least the false alarm probability and the uncertainty measure corresponding to the false alarm probability.

[0022] (2.2) Uncertainty measurement construction: To characterize the risk of boundary samples and out-of-distribution samples, an uncertainty measurement mechanism is constructed. The mechanism includes, but is not limited to, integrated divergence measurement, random inactivation sampling variance estimation, or probability uncertainty estimation based on confidence interval.

[0023] (2.3) Construction of three-stream gating strategy: Based on the false alarm probability and uncertainty metric, gating is performed to divide alarms into: a) alarms with false alarm probability not lower than the false alarm threshold and uncertainty lower than the uncertainty threshold, which are classified as business false alarms; b) alarms with false alarm probability lower than the false alarm threshold and uncertainty lower than the uncertainty threshold, which are classified as real attacks; c) alarms with uncertainty not lower than the uncertainty threshold, which are classified as difficult to verify and proceed to step (3).

[0024] Furthermore, step (3) includes the following sub-steps:

[0025] (3.1) Generation of Operation and Maintenance Context Evidence: Automatically associate data from the change management system and the release pipeline system, perform unified event mapping, time matching and relevance calculation on the association results, and form operation and maintenance context evidence elements.

[0026] (3.2) Asset baseline and exposure surface evidence generation: Based on the asset list library and historical communication log library, extract asset roles, environment types, exposure surface information and historical communication baselines, calculate the degree of deviation, novelty and port exposure surface matching results of the current alarm behavior relative to the historical baseline, and form asset baseline and exposure surface evidence elements.

[0027] (3.3) Generation of similar event evidence: retrieve similar handling samples from the historical handling database and the work order closed-loop database in terms of rules, assets, communication patterns and semantic summaries to the current alarm, extract their handling results and key evidence to form similar event evidence elements.

[0028] (3.4) Generation of subsequent behavior verification evidence: Select the corresponding verification strategy according to the alarm type and triggering rule, and verify and check the observable data within the preset time window after the alarm is triggered to form subsequent behavior verification evidence elements; the subsequent behavior verification evidence includes at least the abnormal status code sequence, the success sequence after authentication failure, port scanning signs, lateral access signs, and external connection signs.

[0029] (3.5) Standardization of structured evidence vectors: The evidence elements obtained in steps (3.1) to (3.4) are standardized and normalized to construct structured evidence vectors; the evidence vectors include at least the evidence content, evidence source identifier, evidence timestamp and evidence credibility score, which are used for subsequent large model evidence constraint reasoning and audit traceability.

[0030] Furthermore, step (4) includes the following sub-steps:

[0031] (4.1) Construction of evidence constraint input template: Construct a structured input template, organize the structured evidence vector and alarm structured features into the input of the large language model according to the predefined format, so that the model output must explicitly include evidence citation identifier and evidence consistency judgment information.

[0032] (4.2) Secondary analysis of difficult alarms: Secondary analysis is performed only on the domain-adapted large language model for difficult review alarms, and standardized analysis results are output. The standardized analysis results include at least alarm classification results, confidence level, evidence support list, counter-evidence / conflict list, supplementary evidence list and summary description. The model output is also subjected to parsing verification, evidence identification legality verification, conclusion evidence verifiability verification and key assertion citation constraint verification. When the verification fails, the analysis result is downgraded to require manual review.

[0033] (4.3) Determination of insufficient evidence and conflict and downgrade processing: Based on the structured evidence vector obtained in step (3), combined with the evidence type, evidence direction, credibility score and relevance score of each piece of evidence, calculate the strength of evidence supporting attack, the strength of evidence supporting false alarm and the degree of evidence conflict, so as to determine the insufficient evidence and the evidence conflict of the current difficult review alarm; when there is insufficient evidence, downgrade processing is performed, a supplementary evidence list is generated and the corresponding evidence is supplemented; when there is evidence conflict, downgrade processing is performed and conflict evidence items are generated.

[0034] (4.4) Fusion Decision Execution: Combining the false alarm probability and uncertainty measure value output by the first-layer lightweight alarm discrimination model, the judgment result of the large language model, the strength of evidence supporting attacks, the strength of evidence supporting false alarms, the degree of evidence conflict and the supplementary evidence situation, the final noise reduction and handling decision is output. The final noise reduction and handling decision includes reporting and handling, filtering, delayed reporting and transfer to manual review.

[0035] (4.5) Supervised fine-tuning of evidence consistency constraints: Combine the final denoising decision in step (4.4) to determine the final disposal label and construct a supervised fine-tuning sample; screen and sample the supervised sample for enhancement; perform supervised fine-tuning update on the large language model based on the sample, so that the model can stably learn evidence citation, consistency check and downgrade rejection strategies.

[0036] The technical solution of this invention can be summarized as follows:

[0037] 1. A method for alarm data preprocessing and multi-dimensional feature construction for operation and maintenance scenarios is proposed. By uniformly parsing, cleaning, normalizing, and time-aligning multi-source alarm data from security devices, log systems, monitoring platforms, and operation and maintenance systems, multi-dimensional features including basic field features, time-series statistical features, contextual features, and alarm text semantic features are extracted to construct structured alarm input samples and standardized input specifications. This provides a high-quality, reusable data foundation for subsequent hierarchical judgment and evidence construction.

[0038] 2. A small-model coarse screening and hierarchical traffic division mechanism for uncertainty gating is proposed. A first-layer lightweight discrimination model is constructed to output the false alarm probability and uncertainty metric for real-time alarms. Based on the false alarm probability, uncertainty and risk budget strategy, gating and hierarchical traffic division is performed to classify alarms into real attack categories, business false alarm categories and difficult-to-review categories.

[0039] 3. A review input construction method driven by evidence generation and behavior verification is proposed. For alarms with difficult review issues, the evidence generation and behavior verification process is triggered, automatically aggregating and standardizing multi-source evidence elements to form a structured evidence vector; the multi-source evidence elements include operational context evidence, asset baseline and exposure surface evidence, similar event evidence, and subsequent behavior verification evidence.

[0040] 4. A large-scale model review and fine-tuning mechanism is proposed. Structured evidence vectors, alarm structured features, and semantic text are input into a domain-adapted large language model. Secondary evaluation is performed only on alarms requiring complex review. The model output includes at least the alarm classification result, confidence score, evidence support list, rebuttal / conflict list, supplementary evidence list, and summary description. Downgrading is performed for insufficient or conflicting evidence, and the final noise reduction decision is generated by combining fusion rules. Furthermore, supervised fine-tuning samples are constructed to adaptively update the large model under supervised fine-tuning with evidence consistency constraints.

[0041] The beneficial effects of this invention lie in its ability to construct a hierarchical adaptive architecture of "lightweight small model uncertainty gating and diversion—evidence generation and behavior verification completion—large language model difficulty judgment under evidence constraints—large model supervised fine-tuning." This architecture enables low-cost, high-precision false alarm identification and automatic noise reduction for massive alarms in operation and maintenance scenarios without relying on extensive manual rule maintenance. Furthermore, it elevates the judgment process from simple rule triggering or semantic judgment to consistency judgment based on verifiable evidence chains, thereby significantly reducing the false alarm rate and the burden of manual handling. Moreover, by constructing supervised samples and performing supervised fine-tuning of the large language model with evidence consistency constraints, the model continuously adapts to changes in business, environmental drift, and attack evolution. This is particularly suitable for the intelligent identification and filtering of security false alarms in complex and heterogeneous business scenarios such as government cloud and enterprise IT. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the overall framework of the method of the present invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0044] The core technology of this invention is to introduce a large language model into the field of operation and maintenance alarm noise reduction, and combine it with domain fine-tuning, prompt word engineering and inference post-processing mechanism to accurately identify and automatically filter false alarms in complex operation and maintenance scenarios, thereby significantly reducing the false alarm rate.

[0045] like Figure 1 As shown, this invention proposes an evidence-driven hierarchical adaptive false alarm denoising method for operation and maintenance scenarios, including the following steps:

[0046] (1) Alarm Data Preprocessing and Multimodal Feature Construction: The purpose of this step is to unify multi-source heterogeneous alarms into trainable and inferable structured samples, and to construct multi-dimensional features containing semantic, pattern, and contextual information. Specifically, it includes the following sub-steps:

[0047] (1.1) Unified Decoding of Multi-Source Alarm Data: In this embodiment, alarm data from security devices, logging systems, and monitoring platforms are first collected and accessed. Addressing the differences in output protocols and encoding formats among different sources, a unified decoding module is used to identify and parse the alarm messages. These formats include, but are not limited to, Syslog, CEF, JSON, XML, and key-value pair text. After decoding, alarms from each source are uniformly organized into a set of original fields. This process achieves consistency in the input of multi-source heterogeneous alarms, providing a unified entry point for subsequent standardization and modeling.

[0048] (1.2) Field parsing and standard schema mapping: To solve the problem of inconsistent field naming and semantics across different systems, this embodiment establishes a field mapping function. This process maps synonymous fields from various sources to standard fields under a unified schema. For example, it maps source_Address and src_ip to src, and signature and rule_Id to rule_name. For messages with nested structures, it recursively parses and extracts key fields, writing them into a unified field set. For missing fields, it uses a combination of missing field filling and missing field flags; that is, when a field... Set missing flag when missing Its definition is Indicates missing, This indicates the existence of data, thus preserving data quality information without disrupting the sample structure; at the same time, type validation and format calibration are performed on fields such as time, IP, port, and protocol to ensure that the fields are comparable and statistically valid.

[0049] (1.3) Anomaly cleaning and format normalization: After completing schema alignment, this embodiment performs anomaly cleaning and format normalization on alarm data. Anomaly cleaning includes, but is not limited to, correcting, removing or marking records such as illegal IPs, port out of bounds, protocol anomalies, and timestamp drift. Format normalization includes unifying the time field to the agreed time zone and format, standardizing the protocol field to the standard enumeration, and unifying the expression of IPv4 / IPv6.

[0050] (1.4) Structured Sample Construction: In this embodiment, multi-dimensional features are extracted from the cleaned and aggregated alarm records to form structured samples. Basic field characteristics Composed of fields such as rule_name, attack_type, req_head, req_body, and src_ip; time series statistical characteristics. Statistical results were obtained within a window of time. Contextual features. By constructing context collections Obtain, among which This indicates the target alarm sample currently pending processing. This represents historical alarm samples used for correlation determination with the current alarm sample. Indicates the current alarm sample The timestamp of the occurrence, Indicates historical alarm samples The timestamp of the occurrence, This indicates the preset context-related time window length, therefore the condition This means that only historical alarms that occurred before the current alarm and fall within the specified time window will be retained; This represents the association key extraction function, used to extract one or more field values ​​from alarm samples for association determination. These field values ​​can be configured as dst_ip, src_ip, asset_id, user_id, or other identifier fields that can characterize alarm objects or entity relationships; alarm text semantic features. It consists of templated text and key field summaries, used to characterize the semantics of alarm descriptions and clues for handling. Finally, various features are combined to obtain... And write it into the structured alarm sample dataset. This is used for subsequent step two, domain labeling and fine-tuning of the large language model.

[0051] (2) Uncertainty Gating: In this embodiment, the lightweight discriminator adopts the LightGBM binary classification model based on Gradient Boosting Decision Tree (GBDT), and constructs an uncertainty metric through "bootstrap ensemble". The model takes the structured alarm sample x obtained in step (1) as input and outputs the false alarm probability. and uncertainty And perform gating and routing. Specifically, this includes the following sub-steps:

[0052] (2.1) Lightweight model construction: In this embodiment, a lightweight discriminator is trained using the binary classification objective function of LightGBM. For any input sample The model outputs the false alarm probability. :

[0053]

[0054] in, This represents the feature vector of the input structured alarm sample. This indicates the category label corresponding to the sample. This indicates that the sample belongs to the false alarm category. This indicates that the sample belongs to the category of real attacks. Indicates that in a given input sample The conditional probability that the sample is classified as a false alarm under the given conditions. Lightweight discriminator The output false positive probability score, This indicates that the input sample is processed by LightGBM. The calculated original scoring function, This represents the Sigmoid mapping function, used to map the original scores to probability values ​​in the interval (0, 1). Weighted cross-entropy loss is used during the training phase to reduce the risk of "real attack classes" being misclassified as false positives.

[0055]

[0056] Where N is the number of training samples, and i represents the index of the training sample. Let represent the input feature vector of the i-th training sample. This represents the true class label corresponding to the i-th training sample. This represents the false positive probability output by the lightweight discriminator for the i-th training sample. This represents the category weight corresponding to the false positive sample. This represents the class weights corresponding to real attack samples. In this embodiment, w0=3.0 and w1=1.0 (i.e., assigning higher weights to the real attack class with y=0) to achieve a more conservative filtering strategy. The key hyperparameters of the model are fixed as follows: num_leaves=64, learning_rate=0.05, n_estimators=500, subsample=0.8, colsample_bytree=0.8, min_data_in_leaf=50, max_depth=-1.

[0057] To make the probability output more usable, this embodiment uses isotonic regression with equal binning on the validation set. Perform probability calibration to obtain the calibrated probability. This is used for subsequent gating and diversion.

[0058] (2.2) Uncertainty Measure Construction: To characterize the risk of boundary samples and out-of-distribution samples, this embodiment uses a bootstrap ensemble to construct the uncertainty measure. Specifically, with a fixed ensemble size K=7, bootstrap sampling with replacement is performed on the training set to obtain K training subsets. A LightGBM sub-model is independently trained on each subset. , 1≤k≤K. For any input We obtain K predicted probabilities:

[0059]

[0060] In this embodiment, the ensemble mean is used as the final false alarm probability:

[0061]

[0062] And let variance be the uncertainty u:

[0063]

[0064] When u is large, it indicates that the sub-models have significant differences in their judgment of the sample, and the sample is more likely to be at the threshold boundary, requiring a difficult review process rather than direct filtering.

[0065] (2.3) Construction of three-stream gating strategy: This embodiment sets a false alarm threshold. Uncertainty threshold The gating and traffic diversion rules are as follows:

[0066] Business false alarm type: when If the error occurs, it is classified as a false alarm and will be subject to automatic noise reduction processing (such as filtering / aggregation / downgrading), and audit information will be recorded.

[0067] Real attack type: When If the attack is deemed a genuine attack, it will proceed to the reporting / handling process.

[0068] Difficult review cases: When If the case is deemed difficult to review, it proceeds to step (3) evidence generation and behavior verification.

[0069] (3) Construction of verification input driven by evidence generation and behavior verification: This step is triggered when the gating and diversion output in step (2) is a difficult verification class. The goal is to automatically gather multi-source verifiable evidence around the difficult verification class alarm and standardize the evidence elements into a structured evidence vector E, so as to provide deterministic input for the large model verification of evidence constraints in step (4).

[0070] (3.1) Generation of Operational Context Evidence: In this embodiment, operational context evidence is derived from the Change Management System (CMS) and the Release Pipeline System (RPS). The CMS primarily stores data related to change requests, approvals, execution processes, and affected objects; the RPS primarily stores data related to release tasks, release times, release objects, and release results. The evidence generator first performs a structured query on the aforementioned systems based on the asset identifier (asset_id), service identifier (service_id), and alarm time (t0). The query results are uniformly mapped to a set of change events. Each change event It should include at least: event ID, event type, assets / services involved, and event time range. Operator, Change Summary, Rollback Flag fields.

[0071] For deterministic filtering and ranking, this embodiment defines a change relevance score. Based on time proximity Asset matching degree Type matching degree Linear combination yields:

[0072]

[0073] Time proximity is achieved using exponential decay:

[0074]

[0075] in, This is the timestamp of the current alarm occurrence. This is a time decay constant used to control the effect of time difference on the rate of score decay; in this embodiment, it is set to 30 minutes.

[0076] Asset matching degree Defined as follows: if the asset_id involved in 'e' is the same as the alarm asset_id, then the value is 1; if they are different but the service_id is the same, then the value is 0.6; otherwise, the value is 0. Type matching degree. Fixed mapping based on event type: Release / Scaling Up / Down / Policy Change / Configuration Distribution use 1.0 / 0.8 / 0.8 / 0.6 respectively. Evidence generator is reserved. Events with a value of ≥0.4 are used as evidence of change, and the top 5 events are output for review.

[0077] (3.2) Asset Baseline and Exposure Surface Evidence Generation: In this embodiment, the sources of asset baseline and exposure surface evidence are limited to the Asset Inventory Repository (AIR) and the Historical Communication Log Repository (HCLR). AIR primarily stores data related to asset attributes, asset environment, service affiliation, and exposure surface overview; HCLR primarily stores data related to network communication behavior, communication time, communication frequency, and traffic statistics. For each alarm, the target asset identifier is set as asset_id, the triggering rule as rule_name, the communication protocol as proto, the target port as dst_port, and the alarm occurrence time as... The evidence generator first reads the static profile corresponding to the asset_id from AIR, including the asset environment, asset role, business affiliation, and allowed exposed port set; then it extracts historical communication samples from HCLR that have the same combination of features (asset_id, rule_name, proto, dst_port) as the current alarm. To ensure the comparability of baseline comparisons, this embodiment constructs a historical baseline window based on the key K=(asset_id, rule_name, proto, dst_port). The Take a set of historical samples from the past 14 days that fall within the same time period as the current alarm occurrence; preferably, the same time period is defined as... The range is ±1 hour centered on the corresponding hour.

[0078] To generate baseline evidence for assets, this embodiment calculates the deviation of continuous statistics. Taking the alarm trigger frequency *r* per unit time as an example, let the historical average... Standard deviation exist The above estimate indicates the degree of deviation in this case. Defined as:

[0079]

[0080] in, For smoothing, used to avoid When the denominator is 0, the denominator is 0. Therefore, ε = 0.001 is preferred.

[0081] when A value ≥3 is considered a significant deviation and generates "baseline deviation evidence". For discrete behavioral patterns (historical occurrences of the dst_port, proto combination), this embodiment further defines a novelty index nov to characterize whether the current (dst_port, proto) combination belongs to historically unseen behavior. Novelty index nov: If (dst_port, proto) in If a value has never appeared in the table, then nov=1; otherwise, nov=0.

[0082] To generate evidence of exposure, this embodiment reads the set of allowed exposed ports from AIR. And define the exposure result: if dst_port If the alarm triggers an alert, then exposure = ALLOW; otherwise, exposure = DENY. ALLOW indicates that the port involved in the current alarm is within the allowed exposure range; DENY indicates that the port involved in the current alarm is not within the allowed exposure range.

[0083] (3.3) Similar Event Evidence Generation: This embodiment retrieves similar events from the Historical Case Repository (HCR) and the Ticket Closure Repository (TCR). The HCR primarily stores the handling results, handling basis, and historical evidence summaries of completed alarms; the TCR primarily stores the ticket flow process, handling actions, review records, and final conclusions. Each record h in the similar event repository includes: alarm key field, handling tag, handling basis, key evidence summary, and final conclusion. To ensure determinism, this embodiment adopts a hybrid retrieval scheme of "sparse retrieval + dense vector retrieval + cross-coding rearrangement," and fixes the model and parameters as follows:

[0084] Sparse retrieval: BM25 (k1=1.2, b=0.75), indexed fields are rule_name, alert_type, asset_id, dst_port, and text summary;

[0085] Dense Vector Retrieval: Using bge-m3 as the text vectorization model, the query and candidate event summary are encoded into a 1024-dimensional vector, and retrieved using cosine similarity and FAISS (IVF) index;

[0086] Cross-encoding reordering: The bge-reranker-large cross-encoder is used to reorder the Top-50 candidates and output the final Top-5 similar events.

[0087] The construction rule for the query text q is fixed as: q = concat(rule_name, alert_type, asset_role, proto, dst_port, text_summary). The candidate comprehensive score s(h) is defined as the normalized weighted sum of the BM25 score and the vector similarity:

[0088]

[0089] in Linear normalization is applied to [0,1]. After obtaining the Top-50, a rearrangement model is used to output the rearrangement scores. , and according to The ranking is based on the top 5. Evidence elements for similar events include: similar event ID, similarity score, historical handling tags, key evidence summary, and the set of evidence types cited.

[0090] (3.4) Generation of Subsequent Behavior Verification Evidence: In this embodiment, the verification strategy is selected according to the alarm type and triggering rules, and the verification data source is limited to the Application Audit Log Repository (AALR). The AALR mainly stores data related to request access, authentication processes, port activity, and application-side behavior trajectories. The verification window is fixed at a preset time window ΔT after the alarm is triggered. Different types of alarms use the following deterministic indicators to generate subsequent behavior verification evidence:

[0091] (i) Web injection type: Count the number of 5xx responses N5 and the number of 2xx successful responses N2 under the same (src_ip,dst_ip,uri), and calculate the suspicious success rate. .when When the value is ≥0.3 and multiple payload variants exist, generate evidence of an "abnormal status code sequence";

[0092] (ii) Brute-force authentication: Count the number of failures in the IAM log. Number of times "success after failure" ,definition .when ≥20 and When the value is ≥0.05, evidence of a "success sequence after authentication failure" is generated.

[0093] (iii) Port scanning / probing: Counting the number of unique destination ports from the same source to the same target in NetFlow. With connection rate, defined .when Evidence of "port scanning indications" is generated when the value is ≥20 and the rate exceeds the baseline mean +3.

[0094] (iv) Lateral movement / abnormal access: Within the window, calculate the size of the target asset set |DstSet| and the size of the access identity set |UserSet|, and calculate the spread = |DstSet|. When spread ≥ 5 or cross-network segment access occurs, generate evidence of "lateral access indication";

[0095] (v) External connection indicators: Count the number of abnormal domain name resolutions / new destination IP connections in the outbound logs. And calculate novelty. (If it hasn't appeared in the past 7 days, it's 1). When ≥3 and When =1, evidence of "external connection" is generated.

[0096] The above indicators are all judged using fixed thresholds and form structured verification results. The verification results also include an "Evidence Direction" field: SUPPORT_ATTACK (supports real attacks) or SUPPORT_FP (supports business false alarms). For example, if the evidence is changed... High and baseline deviation not significant ( <2), and subsequent behavioral verification does not show any signs of successful exploitation, then the direction of the verification evidence is SUPPORT_FP.

[0097] (3.5) Standardization of structured evidence vectors: In this embodiment, the evidence elements obtained from (3.1) to (3.4) are standardized into evidence vectors. Each piece of evidence Use a fixed schema:

[0098] evidence_id: Unique identifier for evidence (UUID);

[0099] evidence_type: Evidence type ∈ {CHANGE, BASELINE, SIMILAR_CASE, BEHAVIOR_VERIFY};

[0100] source_id: Identifier of evidence source (CMS, RPS, AIR, HCLR, HCR, TCR, AALR);

[0101] time_range: ;

[0102] content: Structured content (JSON, containing key fields and a summary);

[0103] direction: Evidence direction ∈ {SUPPORT_ATTACK, SUPPORT_FP, NEUTRAL};

[0104] confidence: The confidence score of evidence c∈[0,1];

[0105] relevance: The relevance score to the current alarm, r∈[0,1];

[0106] hash: Content fingerprint, used for deduplication and auditing.

[0107] The credibility score c is calculated using a deterministic weighted average.

[0108]

[0109] For source credibility constants: 0.9 / 0.85 / 0.8 for CMS / CI-CD / CMDB, 0.75 for WAF / IAM, 0.8 for EDR, and 0.7 for NetFlow. For time proximity (same form as (3.1), τ = 30 minutes); The percentage of missing fields in the content; consistency is used to characterize the consistency with other evidence, set to 1 if it does not conflict with highly credible evidence, otherwise set to 0. The relevance score r is set for change / baseline / behavioral verification evidence respectively. min(1, / 3), and the values ​​of indicators such as scan_score after normalization to [0,1].

[0110] To avoid redundancy, the evidence deduplication rule is fixed as follows: if evidence_type is the same as hash, only the record with the largest confidence is retained, and the remaining records are kept as historical versions but are not included in vector E. The final output evidence vector E and the alarm structured sample x are passed to the evidence constraint input template construction module in step (4).

[0111] (4) Large-scale model verification and supervised fine-tuning adaptive update of evidence constraints: In step (3), the system has automatically gathered multi-source verifiable evidence around the alarms of difficult verification, and standardized the evidence elements into structured evidence vectors. This step, based on this, performs "large-scale model verification and supervised fine-tuning adaptive update with evidence constraints." This embodiment provides deterministic model selection, input / output specifications, inference parameters, evidence consistency verification rules, downgrade / rejection criteria, fusion decision-making, and supervised fine-tuning training schemes.

[0112] (4.1) Construction of evidence constraint input template: This embodiment uses a fixed structured input template. The large language model input is organized, and the template consists of three parts: A. Alarm structured field x (from step (1)); B. Evidence list E (from step (3)); C. Output constraint G. To ensure the consistency of the input order, the evidence list is arranged in ascending lexicographical order of evidence_id.

[0113] (a) Input template fields (example) are as follows:

[0114] ALERT: rule_name, alert_type, asset_id, asset_role, env, src_ip, dst_ip, proto, dst_port, time=t0, time series summary, context summary, text_summary;

[0115] EVIDENCE: List {evidence_id, evidence_type, source_id, time_range, direction, confidence, relevance, content} item by item;

[0116] CONSTRAINT: Requires output to be strict JSON; the evidence_id in the output can only come from the evidence set of this input; a "list of supporting evidence / conflicts / supplementary evidence" must be provided.

[0117] (b) The output JSON Schema is fixed to the following set of fields:

[0118] decision: ∈{ATTACK, FALSE_POSITIVE, REVIEW}, representing a real attack, a false alarm, and a need for manual review, respectively;

[0119] confidence: ∈[0,1], representing the confidence level of decision;

[0120] evidence_support: A list of supporting evidence, each item is {evidence_id, why};

[0121] evidence_conflict: A list of counter-evidence / conflicts, each item being {evidence_id, why};

[0122] missing_evidence: List of missing evidence (array, elements are the types of evidence or data sources that need to be supplemented);

[0123] reasoning_summary: A summary of no more than 120 words (no key assertions without cited evidence are allowed).

[0124] (4.2) Secondary analysis of difficult alarms: The large language model in this embodiment uses Qwen3-30B-A3B-Instruct as the basic model. Based on the data of this invention, After supervising and fine-tuning, a domain-adaptive model is obtained. (See (4.5)). The inference phase only invokes alarms that are difficult to verify. This avoids the full inference cost of large models.

[0125] To ensure output stability, reproducibility, and audit consistency, this embodiment uses the following fixed inference parameters: temperature=0.2, top_p=0.9, top_k=20, max_new_tokens=512, repeat_penalty=1.05. The system prompt is fixed as: "You are an operations and maintenance security alarm analysis expert. Please output strict JSON under evidence constraints. Do not fabricate evidence or output additional text."

[0126] After the model is output, the system performs hard validation and downgrade correction. The validation rules are fixed as follows:

[0127] R1 (Resolvability): The output must be valid JSON and conform to the schema; otherwise, set decision=REVIEW, confidence=0, and add "requires manual review" to missing_evidence.

[0128] R2 (Evidence ID Validity): All evidence_ids appearing in the output must belong to ID(E); if an illegal reference exists, set decision=REVIEW and confidence=0.

[0129] R3 (Conclusion can be evidenced): When decision ∈ {ATTACK, FALSE_POSITIVE}, evidence_support must be non-empty; otherwise, set decision = REVIEW.

[0130] R4 (Key Assertion Reference Constraint): If key assertions such as "successful exploitation / external link / lateral diffusion / credential hit" appear in the reasoning_summary, there must be corresponding BEHAVIOR_VERIFY or SIMILAR_CASE evidence supporting them in evidence_support; otherwise, set decision=REVIEW.

[0131] (4.3) Determination of Insufficient Evidence and Conflict, and Downgrading: This embodiment, based on the structured evidence vector E generated in step (3), combines the evidence type, evidence direction, credibility score, and relevance score of each piece of evidence to definitively determine whether the current difficult review alarm has "insufficient evidence" or "conflicting evidence." Let the credibility of each piece of evidence k be... , direction is Define the strength of evidence supporting an attack. Strength of evidence supporting false positives :

[0132]

[0133]

[0134] (a) Criterion for insufficient evidence: when max( , If the value is less than 0.8, it is considered insufficient evidence, and `decision=REVIEW` must be output. A minimum supplementary evidence set `missing_evidence` must be generated according to the deterministic rules.

[0135] If CHANGE-type evidence is missing, supplement it with change orders, release records, maintenance windows, configuration distribution records, scaling records, or policy change records corresponding to the current asset_id, service_id, and t0 from the change management system and release pipeline system. If BASELINE-type evidence is missing, supplement it with asset roles, environment types, allowed exposed port sets, communication baseline samples for the same time period in the past 14 days, and exposure surface snapshots from the asset inventory database and historical communication log database. If SIMILAR_CASE-type evidence is missing, supplement it with historical samples similar to the current alarm in rule name, alarm type, asset characteristics, communication mode, and semantic summary from the historical disposal database and work order closed-loop database, expanding the search time window or relaxing the recall threshold if necessary. If BEHAVIOR_VERIFY-type evidence is missing, supplement it with the corresponding verification data source according to the current alarm type.

[0136] (b) Criterion for Conflict of Evidence: Define the degree of conflict as conflict = min( , When conflict ≥ 0.6, it is considered that there is a substantial conflict in the highly credible evidence. Decision = REVIEW must be output, and evidence_conflict must list at least one piece of highly credible evidence supporting the attack and one piece of highly credible evidence supporting the false positive. The reasons for the conflict must also be given, including at least: the factual judgment dimension to which the conflict belongs, the corresponding time range, the basis for the opposite direction, and the reason why it cannot be directly resolved, for subsequent manual verification.

[0137] (4.4) Fusion Decision Execution: In this embodiment, the final noise reduction decision is not based solely on the output of the large language model, but rather on the false alarm probability output by the first-layer lightweight alarm discrimination model in step (2). And the uncertainty measure u, the decision and confidence results of the large language model obtained in step (4.2), and the strength of supporting attack evidence obtained in step (4.3). Support for the strength of evidence of false positives The evidence conflict degree and the minimum supplementary evidence set missing_evidence are fused to output the final noise reduction decision π.

[0138] Therefore, an attack-side fusion score is defined. False alarm fusion score They are respectively:

[0139]

[0140]

[0141] in, It is an indicator function; As a risk escalation indicator, R=1 when the current alarm involves high-risk rules, high-value assets, core business systems, or regulatory sensitive objects; otherwise, R=0. α, β, γ, δ, λ, and η are preset weight parameters used to adjust the impact of small model priors, large language model conclusions, evidence strength, evidence conflict, and uncertainty on the final decision.

[0142] Based on the above fusion score, the system outputs the final noise reduction decision according to the following deterministic rules:

[0143] (a) - When the value is ≥0.4 and conflict <0.6, output π = report for handling;

[0144] (b) - If ≥0.4, R=0, and missing_evidence is empty or contains only non-critical supplementary evidence, output π=filter;

[0145] (c) When | - If |<0.4, or if missing_evidence is not empty but the missing evidence can be automatically filled in within the preset delay window, output π=delayed reporting;

[0146] (d) When conflict ≥ 0.6, or missing_evidence contains key evidence that is missing and cannot be automatically filled in within the preset time, or R = 1, output π = transfer to manual review.

[0147] Among them, reporting and handling refers to treating the alarm as a real attack and entering the subsequent manual handling process; filtering refers to removing the alarm as a false alarm; delayed reporting refers to re-executing step (3) to supplement evidence and step (4) to review the large model within a preset delay window based on the supplementary evidence list; transferring to manual review refers to submitting the current alarm along with the small model output, the large language model output and related evidence for manual review. In this embodiment, the final noise reduction handling decision π output in step (4.4) constitutes the final handling conclusion of the alarm.

[0148] (4.5) Supervision and fine-tuning of evidence consistency constraints: In this embodiment, after outputting the final noise reduction decision and obtaining the final disposal label y* in step (4.4), a supervision and fine-tuning sample is constructed and parameter efficient supervision and fine-tuning is performed.

[0149] Furthermore, the final disposal label y is determined according to the final noise reduction disposal decision π in step (4.4): when π is "report disposal", the "real attack" conclusion confirmed by the subsequent disposal process is used as the final disposal label y*; when π is "filter", the "business false alarm" conclusion confirmed by filtering is used as the final disposal label y*; when π is "delayed reporting", steps (3) and (4) are re-executed within the preset delay window based on missing_evidence, and the disposal conclusion obtained again is used as the final disposal label y*; when π is "transfer to manual review", the conclusion of manual review is used as the final disposal label y*.

[0150] The supervised sample consists of two parts: input and output. The input is... The output is standardized JSON (see (4.1)), where decision is mapped by the final disposal tag y*: real attack → ATTACK, business false alarm → FALSE_POSITIVE.

[0151] To ensure the quality of the supervised samples, this embodiment adopts a deterministic screening and sampling strategy: when evidence collection fails and E is empty, only REVIEW training samples are included; high-risk false positive samples and high uncertainty samples are oversampled by 2 times to enhance the model's ability to correct fusion decision errors and boundary samples. Among them, high-risk false positive samples are those that have delayed reporting or manual review in the final noise reduction decision of step (4.4), but whose final disposal label y* after the alarm is completed is a real attack; high uncertainty samples refer to those that enter the difficult review process in step (2) because the uncertainty metric u reaches the uncertainty threshold, and are still judged as insufficient evidence or conflicting evidence in step (4.3).

[0152] The supervised fine-tuning adopts the LoRA (Low-Rank Adaptation) scheme, with the following fixed configuration:

[0153] LoRA rank r=16, alpha=32, dropout=0.05;

[0154] Injection modules: q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, gate_proj;

[0155] The optimizer is AdamW, with a learning rate of lr=2e-5, weight decay of 0.01, and warmup_ratio=0.05.

[0156] Training plan: Daily incremental epoch = 1 (maximum of 5000 samples) + weekly full epoch = 3;

[0157] Maximum input length: 4096 tokens; Output length: 512 tokens.

[0158] The loss function adopts a combination of "cross-entropy + evidence consistency constraint penalty":

[0159]

[0160] in To output token-level cross-entropy loss; As a penalty item for consistency of evidence, The weight coefficient is used to suppress invalid evidence_id outputs or omission of evidence references when outputting ATTACK / FALSE_POSITIVE, and is defined as:

[0161] )

[0162] in, This is an indicator function; it takes the value 1 when the condition in parentheses is true, and 0 otherwise. id represents a single evidence identifier (evidence_id) referenced in the model output, which is the evidence number parsed from the model output. ŜI is the set of evidence_ids parsed from the model output. E is the evidence set corresponding to the current alarm. This represents the set of all valid evidence_ids in the evidence set E. Through this constraint, the large model forms a stable behavioral pattern of "citing evidence first, then drawing conclusions".

[0163] Implementation example:

[0164] This embodiment verifies the adaptive noise reduction method for false alarms in operation and maintenance scenarios proposed in this invention. The experimental environment is an 8-GPU A800 server, and the large language model verification module uses the Qwen3-30B-A3B-Instruct model. The performance of the method in terms of real attack identification, business false alarm differentiation, and processing efficiency is evaluated by testing actual alarm samples.

[0165] Experimental Objective: This experiment compares the method of ordinary small model, the method of direct judgment of large model, and the method of removing multi-dimensional operation and maintenance evidence with the complete method of this application to verify whether the method of hierarchical judgment and multi-dimensional operation and maintenance evidence adopted in this application can improve the accuracy of alarm analysis while taking into account the efficiency of alarm processing and reduce the risk of real attacks being falsely filtered.

[0166] Experimental data and standardized conditions: Security alarms collected from actual operation and maintenance environments and confirmed by manual final processing were used as experimental samples. These samples were labeled as either real attacks or false alarms. See Table 1. The training set, validation set, and test set should be divided in the same way, and all four methods should use the same test set and final labels.

[0167] Table 1: Composition of the experimental dataset

[0168] The comparison method settings are shown in Table 2:

[0169] Table 2: Explanation of Comparison Methods

[0170] Evaluation indicators: Real attacks are classified as positive, and false positives are classified as negative. The following metrics are used for evaluation: 1. Accuracy: The percentage of correctly identified alarms out of all test alarms; 2. Precision: The percentage of alerts judged as real attacks that are actually real attacks; 3. Recall: The percentage of all real attacks that are correctly identified; 4. F1 score: The harmonic mean of precision and recall; 5. False Positive Rate of Real Attacks: The proportion of real attacks that are incorrectly identified as false positives by the business; the lower the value, the better. 6. Average processing time: The average time required to complete the judgment of a single alarm; Among them, Accuracy, Precision, Recall, and F1 are used to describe the overall judgment effect; the false filtering rate of real attacks is used to describe the security of automatic noise reduction; average processing time and large model call rate are used to describe the efficiency advantages brought by layered processing.

[0171] Table 3: Comparative Experimental Results of Ablation Using Four Methods

[0172] Referring to Table 3, the experimental results show that the method of this invention has good recognition performance in alarm noise reduction tasks. The common small model method M1 performs binary classification directly based solely on the structured features of the alarms, with an average processing time of only 0.03 seconds per alarm. It also does not require calling a large language model, offering advantages such as simple implementation, fast processing speed, and low resource consumption. However, M1 is insufficient in handling boundary samples, out-of-distribution samples, and complex alarms that require business context for judgment. Its Recall and F1 scores are only 57.0% and 65.2%, respectively, and the false filtering rate for real attacks reaches 43.0%.

[0173] The complete method M4 in this application uses a joint gating system of false positive probability and uncertainty to transfer high-uncertainty alarms to evidence-enhanced review, increasing Recall to 82.8%, a 25.8 percentage point improvement over M1; F1 to 90.1%, a 24.9 percentage point improvement over M1; and the false filtering rate of real attacks to 17.2%, a reduction of 25.8 percentage points. Furthermore, M4's Accuracy and Precision are improved by 10.6 and 22.8 percentage points respectively compared to M1. While ordinary small models have good processing efficiency, they tend to directly classify boundary alarms that cannot be reliably judged as business false alarms. This application identifies such difficult alarms through uncertainty diversion and performs secondary review, significantly improving the ability to identify real attacks and reducing the risk of real attacks being falsely filtered. The cost is an increase in average processing time from 0.03 seconds to 0.21 seconds, but this latency remains low, and the security improvement is significant.

[0174] The large-model direct judgment method M2 leverages the semantic understanding capabilities of a large language model, resulting in overall superior performance compared to ordinary small-model methods. However, since all alarms require invoking the large language model, the large model invocation rate reaches 100%, with an average processing time of 1.57 seconds per alarm. This leads to issues such as high inference costs, high resource consumption, and limited scalability in high-concurrency scenarios. Furthermore, because M2 only uses basic alarm information and lacks multi-dimensional operational factual evidence constraints, its accuracy, recall, and F1 score are 89.3%, 69.4%, and 76.4%, respectively, while the false positive rate for actual attacks remains at 30.6%.

[0175] Compared to M2, the complete method M4 in this application improves accuracy by 6.2 percentage points, precision by 14.0 percentage points, recall by 13.4 percentage points, and F1 score by 13.7 percentage points, while reducing the false positive rate of real attacks by 13.4 percentage points. Simultaneously, the average processing time of M4 is reduced from 1.57 seconds / alarm to 0.21 seconds / alarm, a reduction of 1.36 seconds. This demonstrates that directly calling the large language model for all alarms does not guarantee better judgment results; instead, it incurs significant computational resource overhead. This application uses a lightweight model to process easily judged alarms, only initiating large language model verification for difficult alarms. While achieving higher judgment metrics, it significantly reduces processing latency and the proportion of large model calls, achieving a balance between judgment effectiveness and processing efficiency.

[0176] The M3 method for removing multi-dimensional operational evidence retains the uncertainty diversion mechanism of the small model and the verification mechanism of the large language model, with an average processing time of 0.15 seconds per alarm, demonstrating good resource control. However, during the verification of complex alarms, M3 only provides the alarm itself to the large language model, without providing change records, asset baselines, exposure surfaces, similar historical cases, and subsequent behavior verification information. Therefore, when alarms have similar surface characteristics but different actual business backgrounds, the large language model lacks factual evidence to verify the judgment.

[0177] Compared to M3, the complete method M4 in this application improves accuracy from 91.6% to 95.5%, an increase of 3.9 percentage points; precision from 90.4% to 98.9%, an increase of 8.5 percentage points; recall from 73.8% to 82.8%, an increase of 9.0 percentage points; F1 from 81.3% to 90.1%, an increase of 8.8 percentage points; and the false positive rate of real attacks decreases from 26.2% to 17.2%, a decrease of 9.0 percentage points. Due to the addition of multi-source evidence query, structured processing, and behavioral verification steps, the average processing time of M4 is 0.06 seconds longer than that of M3, but the large model call rate is the same for both. Furthermore, M4 achieves significant improvements in accuracy, attack recall capability, and false positive filtering risk. This indicates that the improvement in M4's performance does not come from increasing the number of large model calls, but mainly from the factual supplementation and constraint of multi-dimensional operational evidence on the difficult alarm judgment process. This result directly proves that multi-dimensional evidence generation and subsequent behavioral verification have independent technical contributions.

[0178] These experimental results demonstrate the superiority of the false alarm adaptive noise reduction method proposed in this invention. It performs excellently in terms of processing performance, identification accuracy, and real-time response capability, providing strong support for operation and maintenance personnel and significantly improving the management efficiency and security of alarms.

[0179] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the above-described evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios.

[0180] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios.

[0181] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0183] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0184] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0185] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. An evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios, characterized in that, Includes the following steps: (1) Obtain the original alarm data and preprocess it. Extract the alarm structure features from the preprocessed original alarm data to construct a structured alarm sample dataset. (2) Based on the structured alarm sample data obtained in step (1), construct the first-layer lightweight alarm discrimination model and output the false alarm probability and uncertainty measure value for real-time alarms; according to the false alarm probability and uncertainty measure value, classify the alarms into real attack type, business false alarm type and difficult review type. (3) For difficult review cases, trigger the evidence generation and behavior verification process, automatically associate and aggregate multi-source evidence elements, and standardize the multi-source evidence elements into structured evidence vectors; (4) Input the structured evidence vector and the corresponding alarm structured features into the domain-adapted large language model, perform secondary analysis on difficult-to-review alarms, and construct a supervision sample for supervising fine-tuning and adaptive updating of the large language model to perform evidence consistency constraints.

2. The method according to claim 1, characterized in that, The preprocessing includes decoding alarm data, parsing fields, cleaning outliers, and format normalization.

3. The method according to claim 1, characterized in that, The alarm structured features include basic field features, time series statistical features, contextual features, and alarm text semantic features.

4. The method according to claim 1, characterized in that, Step (2) includes the following sub-steps: (2.1) Lightweight model construction: Select a lightweight model as the first-layer discriminator and train it based on the structured alarm sample dataset constructed in step (1); define a unified output specification, and the model output shall include at least the false alarm probability and the uncertainty measure corresponding to the false alarm probability; (2.2) Uncertainty measurement construction: To characterize the risk of boundary samples and out-of-distribution samples, an uncertainty measurement mechanism is constructed, which includes integrated divergence measurement, random inactivation sampling variance estimation, or probability uncertainty estimation based on confidence intervals; (2.3) Three-stream gating strategy construction: based on the false alarm probability and uncertainty metric, gating and diverting alarms are performed, and the alarms are divided into: a) alarms with a false alarm probability not lower than the false alarm threshold and an uncertainty lower than the uncertainty threshold are classified as business false alarms; b) Alarms with a false alarm probability below the false alarm threshold and an uncertainty below the uncertainty threshold are classified as real attacks; c) Alarms with uncertainty not lower than the uncertainty threshold are classified as difficult to verify and proceed to step (3).

5. The method according to claim 1, characterized in that, The multi-source evidence elements include one or more of the following: operational context evidence, asset baseline and exposure surface evidence, similar event evidence, and subsequent behavior verification evidence.

6. The method according to claim 5, characterized in that, Operation and maintenance context evidence generation: Automatically associate change orders, release windows, maintenance windows, configuration distribution records, scaling records, and policy change records to form change evidence elements, and extract the matching relationship and relevance indicators with alarm time windows to form operation and maintenance context evidence elements; Asset baseline and exposure surface evidence generation: Automatically associate asset list with configuration management data, extract asset roles, environment types, exposure surface information and historical communication baselines; calculate the degree of deviation between the current alarm behavior and the historical baseline, and form asset baseline and exposure surface evidence elements; Similar event evidence generation: Retrieve similar handling samples from the historical handling database and the work order closed-loop database that are similar to the current alarm in terms of rules, assets, communication patterns and semantic summaries, extract their handling results and key evidence, and form similar event evidence elements; Subsequent behavior verification evidence generation: Based on the alarm type and triggering rules, select the corresponding verification strategy, verify and check the observable data within the preset time window after the alarm is triggered, and form subsequent behavior verification evidence elements; the subsequent behavior verification evidence includes at least the abnormal status code sequence, the success sequence after authentication failure, port scanning signs, lateral access signs, and external connection signs.

7. The method according to claim 1, 5, or 6, characterized in that, The evidence elements are standardized and normalized to construct a structured evidence vector; the evidence vector includes evidence content, evidence source identifier, evidence timestamp, and evidence credibility score.

8. The method according to claim 1, characterized in that, Step (4) includes the following sub-steps: (4.1) Construction of evidence constraint input template: Construct a structured input template, organize the structured evidence vector and alarm structured features into the input of the large language model according to a predefined format, so that the model output must explicitly include evidence citation identifier and evidence consistency judgment information; (4.2) Secondary analysis of difficult alarms: The domain-adapted big language model performs secondary analysis on difficult review alarms, and the output includes at least the alarm classification result, confidence level, evidence support list, counter-evidence / conflict list, supplementary evidence list and summary description; (4.3) Determination of insufficient evidence and conflict and downgrade processing: Based on the structured evidence vector obtained in step (3), combined with the evidence type, evidence direction, credibility score and relevance score of each piece of evidence, calculate the strength of evidence supporting attack, the strength of evidence supporting false alarm and the degree of evidence conflict, so as to determine the insufficient evidence and the evidence conflict of the current difficult review alarm. When there is insufficient evidence, the case is downgraded, a supplementary evidence list is generated, and corresponding evidence is added; when there is conflict of evidence, the case is downgraded, and conflicting evidence items are generated. (4.4) Fusion Decision Execution: Combining the gating results of the first-layer small model, the judgment results of the large language model, the confidence level, the strength of evidence supporting attacks, the strength of evidence supporting false alarms, and the degree of evidence conflict, the final noise reduction and handling decision is output. The final noise reduction and handling decision includes reporting or handling, filtering, delayed reporting, and transferring to manual review. (4.5) Supervision and fine-tuning of evidence consistency constraints: Combine the final noise reduction decision in step (4.4) to determine the final disposal label and construct the supervision and fine-tuning sample; Supervised fine-tuning updates are performed on the large language model based on the samples.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios as described in any one of claims 1-8.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the evidence-driven hierarchical adaptive false alarm noise reduction method for operation and maintenance scenarios as described in any one of claims 1-8.

Images