Pre-training language model training method and alert noise reduction method

Abstract

The disclosure provides a pre-training language model training method and an alarm noise reduction method, relates to the technical field of computers, and particularly relates to the technical fields of natural language processing, deep learning and cloud platforms. A specific embodiment of the pre-training language model training method comprises the following steps: obtaining a training sample set, wherein the training sample set comprises positive samples and negative samples, the positive samples are sample structured alarm texts labeled with real event alarm labels, and the negative samples are sample structured alarm texts labeled with non-real event alarm labels; taking the sample structured alarm texts as inputs and corresponding labeled labels as outputs, training a bidirectional encoder representation model based on a converter to obtain a pre-training language model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of natural language processing, deep learning, and cloud platform technology. Background Technology

[0002] With the continuous expansion of data centers, distributed systems, and cloud platforms, various servers, network devices, middleware, and upper-layer applications can easily generate massive amounts of alarm messages in a short period of time when they experience failures or anomalies, creating an alarm storm. These alarm messages are characterized by their large quantity, diverse sources, non-standard text, strong correlation, and high redundancy, hindering the rapid location of the root cause of the failure and becoming a key unresolved issue in operation and maintenance work.

[0003] Currently, the industry mainly uses manual monitoring combined with regional reporting to handle alarm information. At the same time, rule-based methods are also used for alarm processing. Alarm information is filtered and organized through fixed rules, keyword matching, field filtering, and other means. Some solutions also combine simple vectorization or traditional machine learning models to further process and analyze alarm data, providing technical support for the daily operation and maintenance of data centers. Summary of the Invention

[0004] This disclosure presents a pre-trained language model training method and an alarm noise reduction method.

[0005] In a first aspect, embodiments of this disclosure propose a pre-trained language model training method, comprising: acquiring a training sample set, the training sample set including positive samples and negative samples, positive samples being sample structured alarm text labeled with real event alarm tags, and negative samples being sample structured alarm text labeled with non-real event alarm tags; using the sample structured alarm text as input and the corresponding labeled tags as output, training a bidirectional encoder representation model based on a converter to obtain a pre-trained language model.

[0006] Secondly, this disclosure proposes an alarm noise reduction method, including: generating structured alarm text based on multi-source heterogeneous alarm information from a data center; inputting the structured alarm text into a pre-trained language model and outputting the semantic overlap confidence between alarms; selecting alarms with semantic overlap confidence higher than a preset confidence threshold as target alarms; and inputting the target alarms into an agent for clustering to generate alarm event clusters.

[0007] Thirdly, this disclosure proposes a pre-trained language model training device, comprising: an acquisition module configured to acquire a training sample set, the training sample set including positive samples and negative samples, positive samples being sample structured alarm text labeled with real event alarm tags, and negative samples being sample structured alarm text labeled with non-real event alarm tags; and a training module configured to take the sample structured alarm text as input and the corresponding labeled tags as output, and train a bidirectional encoder representation model based on a converter to obtain a pre-trained language model.

[0008] Fourthly, this disclosure proposes an alarm noise reduction device, comprising: a generation module configured to generate structured alarm text based on multi-source heterogeneous alarm information from a data center; an output module configured to input the structured alarm text into a pre-trained language model and output the semantic overlap confidence between alarms; a selection module configured to select alarms with semantic overlap confidence higher than a preset confidence threshold as target alarms; and a clustering module configured to input the target alarms into an agent for clustering to generate alarm event clusters.

[0009] Fifthly, embodiments of this disclosure provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods described in the first or second aspect.

[0010] In a sixth aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in the first or second aspect.

[0011] In a seventh aspect, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method as described in the first or second aspect.

[0012] The key or essential features of the embodiments disclosed herein are not intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. Wherein: Figure 1 This is a flowchart of an embodiment of the pre-trained language model training method according to the present disclosure; Figure 2This is a flowchart of yet another embodiment of the pre-trained language model training method according to the present disclosure; Figure 3 This is a flowchart of an embodiment of the alarm noise reduction method according to the present disclosure; Figure 4 This is an architecture diagram of an alarm storm noise reduction system based on a pre-trained language model; Figure 5 This is a schematic diagram of the structure of an embodiment of a pre-trained language model training device according to the present disclosure; Figure 6 This is a schematic diagram of the structure of an embodiment of the alarm noise reduction device according to the present disclosure; Figure 7 This is a block diagram of an electronic device used to implement the pre-trained language model training method of the embodiments of this disclosure. Detailed Implementation

[0014] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] Figure 1 A flowchart 100 is shown as an embodiment of a pre-trained language model training method according to the present disclosure. The pre-trained language model training method includes the following steps: Step 101: Obtain the training sample set.

[0017] In this embodiment, the execution entity of the pre-trained language model training method can be an electronic device with data processing and model training capabilities, such as a data center operation and maintenance server or a cloud computing device. This execution entity obtains a training sample set through data collection, standardization processing, and label annotation, providing a data foundation for model training.

[0018] The training sample set can include positive and negative samples. Both positive and negative samples are structured alarm texts, obtained by standardizing multi-source heterogeneous alarm information from the sample data center. These texts contain structured content such as device address, alarm level, alarm title, detailed description, and business context information. Positive samples are structured alarm texts labeled with "real event alarm," meaning the alarms corresponding to these samples are real alarms that actually cause data center failures or anomalies. Negative samples are structured alarm texts labeled with "non-real event alarm," meaning the alarms corresponding to these samples are false alarms that are not actually caused by failures and are merely due to system redundancy or false alarms.

[0019] Step 102: Take the sample structured alarm text as input and the corresponding labeled tags as output to train the bidirectional encoder representation model based on the converter to obtain a pre-trained language model.

[0020] In this embodiment, the aforementioned execution entity can use the sample structured alarm text as the model input features and the corresponding real event alarm label or non-real event alarm label as the model output label to perform supervised training of the BERT (Bidirectional Encoder Representations from Transformers) model. The core of this BERT model is its self-attention mechanism, which can capture long-distance dependencies in alarm text, transforming the structured alarm text into high-dimensional semantic vectors with context awareness, thereby accurately characterizing the semantic correlation between alarms. Through continuous parameter adjustment and model optimization, a pre-trained language model with alarm semantic discrimination capabilities is finally obtained.

[0021] The pre-trained language model training method provided in this embodiment, through the annotation and model training of sample structured alarm texts, enables the bidirectional encoder representation model based on the converter to learn the semantic features of alarm texts and the discrimination rules of real event alarms, providing core model support for the semantic overlap confidence calculation in subsequent alarm noise reduction. At the same time, this training method can adapt to the multi-source heterogeneous alarm text features of data centers, solving the problem of weak semantic representation ability of traditional models.

[0022] Figure 2 A flow 200 of another embodiment of the pre-trained language model training method according to this disclosure is shown. This pre-trained language model training method includes the following steps: Step 201: Generate sample structured alarm text based on the sample multi-source heterogeneous alarm information from the sample data center.

[0023] In this embodiment, the execution entity of the pre-trained language model training method can collect and process multi-source heterogeneous alarm information of samples based on the actual operation and maintenance data of the sample data center, generate a unified format of sample structured alarm text, and provide standardized input for label annotation and model training.

[0024] In some embodiments, the steps for collecting and standardizing multi-source heterogeneous alarm information are as follows: First, multi-source heterogeneous alarm information is collected from the carriers that generate alarms within the sample data center. These carriers may include, but are not limited to, at least one of the following: servers, network devices, middleware, application services, etc. Faults, anomalies, and other information generated by these carriers during operation are all considered as objects of the multi-source heterogeneous alarm information collection, achieving comprehensive coverage of multi-source alarm data. Then, the multi-source heterogeneous alarm information is standardized to generate sample structured alarm text. The standardization process may include, but is not limited to, at least one of the following: cleaning, denoising, field mapping, and regular expression masking. Cleaning and denoising are used to remove invalid characters, duplicate data, garbled text, and other interfering content from the alarm information. Field mapping is used to eliminate differences in expression, text structure, and timestamp precision among different devices, uniformly mapping alarm fields of different formats to preset standard fields. Regular expression masking is used to identify dynamic variables (such as voltage values, current values, temperature values, device numbers, etc.) in the alarm information and uniformly replace them with static labels to avoid interference from dynamic variables in model training. Through the above standardization process, the unstructured alarm information with messy formats and diverse sources is transformed into a unified standard sample structured alarm text.

[0025] Step 202: Label the structured alarm text of the samples to indicate whether it is a real event alarm, and generate a training sample set.

[0026] In this embodiment, the aforementioned execution entity can label the sample structured alarm texts through manual verification or semi-automatic annotation. The labeled tags are divided into real event alarm tags and non-real event alarm tags, corresponding to real alarms that actually cause faults and false alarms that do not actually cause faults, respectively. All labeled sample structured alarm texts are integrated to generate a training sample set for model training. The positive samples (labeled with real event alarm tags) and negative samples (labeled with non-real event alarm tags) in this training sample set provide a labeling basis for the supervised training of the model.

[0027] Step 203: In response to the determination that the proportion of positive samples in the training sample set is less than a preset proportion threshold, calculate the distribution frequency of positive and negative samples in the sample set.

[0028] In this embodiment, the execution entity first calculates the proportion of positive samples in the training sample set and compares this proportion with a preset ratio threshold. This preset ratio threshold is set based on the actual imbalance of alarm data in the data center, adapting to the actual ratio of real event alarms to false alarms in the data center (e.g., 1:130 or higher). If the proportion of positive samples in the training sample set is less than the preset ratio threshold, the training sample set is determined to be in an extremely imbalanced state. At this point, the distribution frequencies of positive and negative samples in the sample set are calculated to clarify the specific proportions and distribution characteristics of the two types of samples, providing data for subsequent sample equalization processing, and step 204 is executed. If the proportion of positive samples in the training sample set is not less than the preset ratio threshold, the sample distribution of the training sample set is determined to be relatively balanced, and no sample equalization processing is required; step 205 is executed directly.

[0029] Step 204: Perform lossy enhancement on positive samples based on distribution frequency to construct an equalized sampling space with a balanced number of positive and negative samples.

[0030] In this embodiment, the aforementioned execution entity performs lossy enhancement processing on the sparse positive samples based on the calculated distribution frequencies of positive and negative samples. This constructs an equalized sampling space with a balanced number of positive and negative samples, forcing the model to overcome data bias during fine-tuning and deeply explore the semantic boundaries of sparse positive samples. Specifically, the lossy enhancement involves repeatedly copying positive samples to increase their quantity, while simultaneously adjusting the model's training weights to enhance the model's focus on positive samples, allowing the model to fully learn the semantic features of positive samples during training. For example, if the training sample set contains 300 alarm samples, with only one positive sample and the rest negative, repeatedly copying the positive sample ensures a 1:1 balance between the number of positive and negative samples, ultimately constructing an equalized sampling space and providing balanced sample data for subsequent model training.

[0031] Step 205: Input the sample structured alarm text into the converter-based bidirectional encoder representation model, attenuate the gradient of negative samples through a high-order focus loss function, and adjust the parameters of the converter-based bidirectional encoder representation model by combining a multi-scale threshold decision matrix to obtain a pre-trained language model.

[0032] In this embodiment, if the training sample set has undergone equalization processing, the structured alarm text of the samples in the equalized sampling space is input into the bidirectional encoder representation model based on the converter; if the training sample set has not undergone equalization processing, the structured alarm text of the samples in the training sample set is directly input into the model. During model training, a high-order focus loss function is innovatively introduced. This function significantly attenuates the gradient contribution of massive simple negative samples (i.e., false alarms with no actual faults and obvious features), making the model's weight updates highly focused on semantically very similar and difficult-to-distinguish "suspected overlap" hard-to-classify samples, thereby improving the model's discrimination accuracy for real event alarms. At the same time, the model parameters are adjusted in conjunction with a multi-scale threshold decision matrix. This multi-scale threshold decision matrix presets different levels of threshold standards for determining the confidence level of alarm semantic overlap (e.g., below 0.3 is low confidence, 0.3-0.95 is medium confidence, and above 0.95 is high confidence). During model training, the parameters are optimized based on this matrix to improve the accuracy of confidence level determination. By completing the model training and parameter adjustment in the above manner, a pre-trained language model with accurate alarm semantic discrimination and confidence calculation capabilities is finally obtained.

[0033] The pre-trained language model training method provided in this disclosure achieves unified training data through standardized processing of multi-source heterogeneous alarm information. It solves the problem of extremely low positive sample ratio in data center alarm data by lossy enhancement of imbalanced samples and construction of an equalized sampling space. It optimizes the model training effect by combining a high-order focus loss function and a multi-scale threshold decision matrix, enabling the pre-trained language model to accurately capture the deep semantic features of alarm text, effectively identify real event alarms, and provide high-performance model support for subsequent alarm noise reduction.

[0034] Figure 3 A flow 300 of an embodiment of an alarm noise reduction method according to the present disclosure is shown. The alarm noise reduction method includes the following steps: Step 301: Generate structured alarm text based on multi-source heterogeneous alarm information from the data center.

[0035] In this embodiment, the entity executing the alarm noise reduction method can be an electronic device with real-time data acquisition and processing capabilities, such as a real-time operation and maintenance processing device in the data center or a cloud-based alarm processing platform. This entity collects multi-source heterogeneous alarm information generated during the operation of the data center in real time, performs standardized processing on it, and generates structured alarm text, providing standard input for subsequent semantic analysis and confidence calculation.

[0036] In some embodiments, the steps for collecting and standardizing multi-source heterogeneous alarm information are as follows: First, multi-source heterogeneous alarm information is collected from the carriers that generate alarms within the data center. These carriers can include, but are not limited to, at least one of the following: servers, network devices, middleware, application services, etc., enabling real-time collection of alarm information from all devices and applications within the data center, comprehensively covering the sources of fault alarms.

[0037] Then, the multi-source heterogeneous alarm information is standardized to generate structured alarm text. The standardization process may include, but is not limited to, at least one of the following: cleaning, denoising, field mapping, regular expression masking, etc. The processing method is consistent with the standardization process of the training sample set mentioned above. Invalid data is removed through cleaning and denoising, alarm format is unified through field mapping, and dynamic variables are replaced through regular expression masking. Finally, the original multi-source heterogeneous alarm information is transformed into a unified standard structured alarm text containing device address, alarm level, alarm title, detailed description, etc.

[0038] Step 302: Input the structured alarm text into the pre-trained language model and output the semantic overlap confidence between alarms.

[0039] In this embodiment, the execution entity inputs the standardized structured alarm text into the pre-trained language model trained in the above embodiment. Through the model's deep semantic analysis capabilities, it calculates and outputs the semantic overlap confidence between each alarm. This confidence is a quantitative value used to determine the semantic similarity between any two alarms. The higher the value, the higher the semantic overlap between the two alarms, and the more likely they are redundant alarms caused by the same fault.

[0040] In some embodiments, the process of generating semantic overlap confidence is as follows: a pre-trained language model performs deep semantic encoding on structured alarm text, captures long-distance dependencies in the structured alarm text through the model's self-attention mechanism, extracts deep semantic features of the alarm text, and transforms them into high-dimensional semantic vectors with context awareness; the model calculates the semantic similarity between any two alarms based on the high-dimensional semantic vectors, and finally generates and outputs the semantic overlap confidence between each alarm, thereby achieving accurate judgment of alarm semantics.

[0041] Step 303: Select alarms with semantic overlap confidence higher than the preset confidence threshold as target alarms.

[0042] In this embodiment, the execution entity compares the semantic overlap confidence score output by the model with a preset confidence threshold. This preset confidence threshold is set based on a multi-scale threshold decision matrix, selecting alarms with a semantic overlap confidence score higher than this threshold as target alarms, thus filtering out redundant alarms that are highly semantically overlapping and likely caused by the same fault. Alarms with a semantic overlap confidence score lower than the preset confidence threshold are directly rejected and do not proceed to the subsequent clustering stage, thereby achieving preliminary screening and noise reduction of alarm information.

[0043] Step 304: Input the target alarm into the intelligent agent for clustering to generate alarm event clusters.

[0044] In this embodiment, the aforementioned execution entity inputs the selected target alarms into the agent, which then performs an alarm clustering task to aggregate and merge redundant target alarms, generating alarm event clusters. This achieves deep noise reduction and convergence of alarm storms, facilitating maintenance personnel to quickly locate the root cause of faults.

[0045] In some embodiments, the clustering process of the intelligent agent specifically involves: clustering target alarms based on semantic overlap confidence, combining the physical and logical attributes of the alarms, to generate alarm event clusters. Physical topology attributes may include, but are not limited to, data center location and alarm generation time. The intelligent agent presets a fixed time threshold (e.g., 5 minutes) for alarm generation time. Alarms caused by the same fault, even if their generation times differ slightly, are considered to be alarms within the same time range as long as they are within this time threshold. Logical topology attributes may include, but are not limited to, the fault logic meaning corresponding to the alarm, i.e., the specific fault type pointed to by the alarm (e.g., mains power failure, abnormal temperature, battery fire, server crash, etc.). If multiple alarms have the same logical meaning, even if their physical attributes such as data center location and generation time differ slightly, the intelligent agent classifies them into the same category. Finally, redundant alarms with the same logical meaning and meeting preset conditions such as the time threshold are aggregated into an alarm event cluster. Each alarm event cluster corresponds to an actual fault event in the data center.

[0046] In some embodiments, to achieve continuous model optimization and continuous improvement in alarm denoising effect, the alarm denoising method further includes a model iterative optimization step, specifically: First, obtain user feedback on alarm event clusters. That is, when the operation and maintenance personnel are processing data center fault work orders, they manually verify the accuracy of the alarm event clusters generated by the intelligent agent and provide feedback on whether the clustering of alarm event clusters is accurate, whether it corresponds to the actual fault, and whether there are any missed or false alarms.

[0047] Then, based on the feedback results, the structured alarm text is labeled to determine whether it is a real event alarm, and an incremental sample set is generated. The alarm data that has been manually verified and confirmed is used as a new training sample. The labels are then labeled according to the labeling rules of the training sample set mentioned above, and the incremental sample set is generated after integration.

[0048] Finally, the pre-trained language model is iteratively optimized using the incremental sample set. The incremental sample set is input into the pre-trained language model, and the above model training process is repeated to fine-tune and optimize the model parameters. This enables the model to adapt to new alarm templates generated by newly installed equipment or new services in the data center, continuously improving the model's semantic discrimination and alarm noise reduction capabilities, and realizing the model's self-upgrade and closed-loop iterative optimization.

[0049] The alarm noise reduction method provided in this disclosure achieves alarm data unification through standardized processing of multi-source heterogeneous alarm information, accurately calculates the confidence level of alarm semantic overlap through deep semantic analysis of a pre-trained language model, and efficiently aggregates redundant alarms through agent clustering. Ultimately, it converges a massive alarm storm into a small number of alarm event clusters, significantly reducing alarm output traffic and alleviating the workload of operations and maintenance personnel. Simultaneously, through manual feedback and iterative model optimization, it continuously improves alarm noise reduction capabilities, solving the problems of poor generalization and insufficient scalability of traditional alarm processing methods, and greatly improving the efficiency of data center fault handling and system stability.

[0050] Taking a real-world fault involving battery fires and numerous abnormal temperatures in a data center as an example, the actual handling process and results are as follows: At the moment the fault occurred, the multi-source heterogeneous alarm access module acquired tens of thousands of concurrent alarm messages in real time, including data center environmental monitoring logs, host machine overheating and crash alarms, network node offline, and a large number of timeout calls from upper-layer business systems.

[0051] First, the system performs regular expression masking and feature extraction on these alarm texts with different formats, filtering out dynamic temperature and voltage changes, and generating a standardized text stream.

[0052] Next, the standardized data is input into a pre-trained language model, which extracts its deep semantic features and accurately outputs the confidence level of highly correlated semantic overlap among various alarms.

[0053] Subsequently, the agent receives these associated alarms with high confidence scores, performs clustering operations based on the relationships between nodes in the data center topology, and successfully aggregates and compresses the tens of thousands of concurrent redundant alarms caused by cascading reactions into a single high-level structured fault event. It then directly outputs the root cause diagnosis conclusion to the front-end console: "A battery fire in a certain area of ​​the data center caused a sudden rise in the physical environment temperature, resulting in the server in that area triggering thermal protection physical shutdown and a large-scale avalanche of upper-layer services," thereby greatly accelerating the repair progress of the dispatchers.

[0054] Figure 4 The system architecture diagram of an alarm storm denoising system based on a pre-trained language model is shown. This system integrates the aforementioned pre-trained language model training method and alarm denoising method, and is suitable for alarm storm processing scenarios in data centers, distributed systems, and cloud platforms. The system adopts an open architecture, encompassing six core modules: multi-source heterogeneous alarm access and standardization module 401, deep semantic encoding and feature extraction module 402, imbalanced sample space reconstruction module 403, discriminative inference module for imbalanced sample loss function 404, intelligent agent alarm storm clustering module 405, and model self-upgrading and closed-loop iterative optimization module 406. These modules work collaboratively to achieve a full-process alarm storm denoising process, from real-time alarm information access, standardization processing, semantic encoding, sample optimization, discriminative inference, intelligent clustering to model iterative optimization.

[0055] The multi-source heterogeneous alarm access and standardization module 401 serves as the system's input layer. It is responsible for collecting alarm information from multiple sources, including servers, network devices, middleware, and application services, in real time. Through standardization processes such as cleaning, noise reduction, field mapping, and regular expression masking, it generates unified structured alarm text, providing standard input for subsequent modules.

[0056] The deep semantic encoding and feature extraction module 402 is based on the BERT model. It performs deep semantic encoding on structured alarm text, captures long-distance dependencies in the text through a self-attention mechanism, generates high-dimensional semantic vectors, and characterizes the semantic correlation between alarms.

[0057] The unbalanced sample space reconstruction module 403 is only used during the model training phase. In response to the unbalanced situation where the proportion of positive samples in alarm data is extremely low, it constructs a 1:1 balanced sampling space by dynamically calculating the sample distribution frequency and performing lossy enhancement on positive samples, thereby solving the problem of model data bias.

[0058] The discriminative reasoning module 404 of the imbalanced sample loss function takes a high-dimensional semantic vector as input, focuses on hard-to-classify samples through a high-order focus loss function, and outputs the semantic overlap confidence between alarms by combining a multi-scale threshold decision matrix.

[0059] The alarm storm clustering module 405 of the intelligent agent takes high-confidence target alarms as input, combines the physical topology attributes and logical topology attributes of the alarms, and aggregates redundant alarms into alarm event clusters to achieve alarm storm noise reduction.

[0060] The model self-upgrade and closed-loop iterative optimization module 406 collects manual verification feedback data from maintenance personnel, constructs an incremental learning set, and periodically triggers model fine-tuning to achieve iterative optimization of the pre-trained language model and system self-upgrade.

[0061] The system also incorporates a failure handling mechanism, assigning a unique `module_id` or `agent_id` to each microservice or model module for identification. When the agent's alarm storm clustering module fails to retrieve information via an external topology interface, or when the deep semantic model inference encounters an anomaly, the system will automatically retry retrieving the judgment confidence or topology link. If multiple retries fail, the system will trigger a degradation process, transmitting the underlying raw alarm details to the front-end console to directly display standardized raw alarm information. This ensures that real fault alarms are not missed in extreme cases, guaranteeing system reliability.

[0062] Furthermore, the system's openness is reflected in three aspects: First, the openness of its processing methods, supporting alarm identification from multiple heterogeneous devices and business sources, and enabling rapid integration of alarm formats from new devices and new businesses; second, the openness of its system composition, coupling cutting-edge technologies such as deep semantic representation, high-order focus loss functions, and intelligent agents, allowing for continuous improvement in system performance as these technologies evolve; and third, the openness of its data dimensions, collecting verification and work order processing data from operations and maintenance personnel in a closed loop, and continuously optimizing the model through incremental training, thus solving the dependence of traditional noise reduction models on the timeliness of static rules.

[0063] This system addresses the challenges of high false negative rates and poor generalization of existing models caused by massive alarm storms in cloud data centers, characterized by tens of millions of concurrent alarms, highly overlapping semantics but varied textual expressions, and extremely unbalanced positive and negative sample distribution (with a ratio exceeding 1:130). By employing deep semantic representation and deep learning techniques for difficult-to-classify samples, this system achieves accurate removal of duplicate alarms from massive amounts of noise. Compared to traditional alarm processing systems based on fixed rules, this system can effectively adapt to multi-source heterogeneous hardware and software alarm sources, completely solving the noise reduction problem of synonymous but different expressions and causal chain alarms.

[0064] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a pre-trained language model training device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0065] likeFigure 5 As shown, the pre-trained language model training device 500 of this embodiment may include an acquisition module 501 and a training module 502. The acquisition module 501 is configured to acquire a training sample set, which includes positive samples and negative samples. Positive samples are structured alarm text labeled with real event alarm tags, and negative samples are structured alarm text labeled with non-real event alarm tags. The training module 502 is configured to take the structured alarm text as input and the corresponding labeled tags as output to train a converter-based bidirectional encoder representation model to obtain a pre-trained language model.

[0066] In this embodiment, the specific processing of the acquisition module 501 and the training module 502 in the pre-trained language model training device 500 and the resulting technical effects can be found by referring to [reference needed]. Figure 1 The relevant descriptions of steps 101-102 in the corresponding embodiments will not be repeated here.

[0067] In some optional implementations of this embodiment, the acquisition module 501 is further configured to: generate sample structured alarm text based on sample multi-source heterogeneous alarm information from the sample data center; label the sample structured alarm text with whether it is a real event alarm tag, and generate a training sample set.

[0068] In some optional implementations of this embodiment, the acquisition module 501 is further configured to: collect sample multi-source heterogeneous alarm information from the carrier that generates alarms in the sample data center, the carrier including at least one of the following: server, network device, middleware, application service; perform standardization processing on the multi-source heterogeneous alarm information to generate sample structured alarm text, the standardization processing including at least one of the following: cleaning, denoising, field mapping, regular expression masking.

[0069] In some optional implementations of this embodiment, the pre-trained language model training device 500 further includes: a calculation module configured to calculate the distribution frequencies of positive and negative samples in the sample set in response to determining that the proportion of positive samples in the training sample set is less than a preset proportion threshold; and a construction module configured to perform lossy enhancement on the positive samples based on the distribution frequencies to construct an equalized sampling space with a balanced number of positive and negative samples.

[0070] In some optional implementations of this embodiment, the training module 502 is further configured to: input the sample structured alarm text into the converter-based bidirectional encoder representation model, attenuate the gradient of the negative samples through a high-order focus loss function, and adjust the parameters of the converter-based bidirectional encoder representation model in combination with a multi-scale threshold decision matrix to obtain a pre-trained language model.

[0071] Further reference Figure 6As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an alarm noise reduction device, which is similar to... Figure 3 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0072] like Figure 6 As shown, the alarm noise reduction device 600 of this embodiment may include: a generation module 601, an output module 602, a selection module 603, and a clustering module 604. The generation module 601 is configured to generate structured alarm text based on multi-source heterogeneous alarm information from a data center; the output module 602 is configured to input the structured alarm text into a pre-trained language model and output the semantic overlap confidence between alarms; the selection module 603 is configured to select alarms with a semantic overlap confidence higher than a preset confidence threshold as target alarms; and the clustering module 604 is configured to input the target alarms into an intelligent agent for clustering to generate alarm event clusters.

[0073] In this embodiment, the specific processing of the generation module 601, output module 602, selection module 603, and clustering module 604 in the alarm noise reduction device 600, and the resulting technical effects, can be found in reference to [reference needed]. Figure 3 The relevant descriptions of steps 301-304 in the corresponding embodiments will not be repeated here.

[0074] In some optional implementations of this embodiment, the generation module 601 is further configured to: collect multi-source heterogeneous alarm information from the carrier that generates alarms within the data center, the carrier including at least one of the following: server, network device, middleware, application service; perform standardization processing on the multi-source heterogeneous alarm information to generate structured alarm text, the standardization processing including at least one of the following: cleaning, denoising, field mapping, regular expression masking.

[0075] In some optional implementations of this embodiment, the output module 602 is further configured to: perform deep semantic encoding on the structured alarm text, and capture long-distance dependencies in the structured alarm text through an attention mechanism to generate semantic overlap confidence between alarms.

[0076] In some optional implementations of this embodiment, the clustering module 604 is further configured to: cluster target alarms based on semantic overlap confidence, combining physical attributes and logical attributes, to generate alarm event clusters.

[0077] In some optional implementations of this embodiment, the alarm noise reduction device 600 further includes: an acquisition module configured to acquire user feedback results on alarm event clusters; an annotation module configured to annotate structured alarm text with labels indicating whether they are real event alarms based on the feedback results, generating an incremental sample set; and an optimization module configured to iteratively optimize the pre-trained language model using the sample set.

[0078] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0079] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0080] In some optional implementations of this embodiment, Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0081] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0082] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0083] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as pre-trained language model training methods. For example, in some embodiments, the pre-trained language model training method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the pre-trained language model training method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform pre-trained language model training methods by any other suitable means (e.g., by means of firmware).

[0084] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0087] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0088] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0089] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0090] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0091] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A pre-trained language model training method, comprising: Obtain a training sample set, which includes positive samples and negative samples. The positive samples are structured alarm texts labeled with real event alarm tags, and the negative samples are structured alarm texts labeled with non-real event alarm tags. The sample structured alarm text is used as input, and the corresponding labeled tags are used as output to train the bidirectional encoder representation model based on the converter, thus obtaining a pre-trained language model.

2. The method according to claim 1, wherein, The acquisition of the training sample set includes: Based on the multi-source heterogeneous alarm information of the sample data center, the structured alarm text of the sample is generated; The training sample set is generated by labeling the structured alarm text of the sample with whether it is a real event alarm tag.

3. The method according to claim 2, wherein, The sample structured alarm text generated from the multi-source heterogeneous alarm information based on the sample data center includes: Collect multi-source heterogeneous alarm information of the sample from the carrier that generates alarms within the sample data center, wherein the carrier includes at least one of the following: server, network device, middleware, application service; The multi-source heterogeneous alarm information is standardized to generate the sample structured alarm text. The standardization process includes at least one of the following: cleaning, denoising, field mapping, and regular expression masking.

4. The method according to claim 1, wherein, The method further includes: In response to determining that the proportion of positive samples in the training sample set is less than a preset proportion threshold, the distribution frequencies of positive and negative samples in the sample set are calculated. Based on the distribution frequency, the positive samples are subjected to lossy enhancement to construct an equalized sampling space in which the number of positive samples and negative samples are balanced.

5. The method according to any one of claims 1-4, wherein, The process of using the sample structured alarm text as input and the corresponding labeled tags as output to train a converter-based bidirectional encoder representation model to obtain a pre-trained language model includes: The sample structured alarm text is input into the converter-based bidirectional encoder representation model. The gradient of the negative sample is decayed by a high-order focus loss function, and the parameters of the converter-based bidirectional encoder representation model are adjusted by combining a multi-scale threshold decision matrix to obtain the pre-trained language model.

6. An alarm noise reduction method, comprising: Based on multi-source heterogeneous alarm information from the data center, structured alarm text is generated; The structured alarm text is input into a pre-trained language model, which outputs the semantic overlap confidence between alarms. Select alarms with semantic overlap confidence scores higher than a preset confidence threshold as target alarms; The target alarm is input into the intelligent agent for clustering to generate alarm event clusters.

7. The method according to claim 6, wherein, The multi-source heterogeneous alarm information based on the data center generates structured alarm text, including: The multi-source heterogeneous alarm information is collected from the carrier that generates alarms within the data center, and the carrier includes at least one of the following: server, network device, middleware, application service; The multi-source heterogeneous alarm information is standardized to generate the structured alarm text. The standardization process includes at least one of the following: cleaning, denoising, field mapping, and regular expression masking.

8. The method according to claim 6, wherein, The step of inputting the structured alarm text into a pre-trained language model and outputting the semantic overlap confidence between alarms includes: The structured alarm text is subjected to deep semantic encoding, and long-distance dependencies in the structured alarm text are captured through an attention mechanism to generate semantic overlap confidence between the alarms.

9. The method according to claim 6, wherein, The step of inputting the target alarm into the intelligent agent for clustering to generate alarm event clusters includes: Based on semantic overlap confidence, the target alarms are clustered by combining physical and logical attributes to generate the alarm event clusters.

10. The method according to any one of claims 6-9, wherein, The method further includes: Obtain user feedback on the alarm event cluster; Based on the feedback results, the structured alarm text is labeled to determine whether it is a real event alarm tag, and an incremental sample set is generated; The pre-trained language model is iteratively optimized using the sample set.

11. A pre-trained language model training device, comprising: The acquisition module is configured to acquire a training sample set, which includes positive samples and negative samples. The positive samples are structured alarm texts labeled with real event alarm tags, and the negative samples are structured alarm texts labeled with non-real event alarm tags. The training module is configured to take the sample structured alarm text as input and the corresponding labeled tags as output to train the bidirectional encoder representation model based on the converter, thereby obtaining a pre-trained language model.

12. The apparatus according to claim 11, wherein, The acquisition module is further configured to: Based on the multi-source heterogeneous alarm information of the sample data center, the structured alarm text of the sample is generated; The training sample set is generated by labeling the structured alarm text of the sample with whether it is a real event alarm tag.

13. The apparatus according to claim 12, wherein, The acquisition module is further configured to: Collect multi-source heterogeneous alarm information of the sample from the carrier that generates alarms within the sample data center, wherein the carrier includes at least one of the following: server, network device, middleware, application service; The multi-source heterogeneous alarm information is standardized to generate the sample structured alarm text. The standardization process includes at least one of the following: cleaning, denoising, field mapping, and regular expression masking.

14. The apparatus according to claim 11, wherein, The device further includes: The calculation module is configured to calculate the distribution frequencies of positive and negative samples in the training sample set in response to determining that the proportion of positive samples in the training sample set is less than a preset proportion threshold. The construction module is configured to perform lossy enhancement on the positive samples based on the distribution frequency, and construct an equalized sampling space in which the number of positive samples and negative samples are balanced.

15. The apparatus according to any one of claims 11-14, wherein, The training module is further configured to: The sample structured alarm text is input into the converter-based bidirectional encoder representation model. The gradient of the negative sample is decayed by a high-order focus loss function, and the parameters of the converter-based bidirectional encoder representation model are adjusted by combining a multi-scale threshold decision matrix to obtain the pre-trained language model.

16. An alarm noise reduction device, comprising: The generation module is configured to generate structured alarm text based on multi-source heterogeneous alarm information from the data center. The output module is configured to input the structured alarm text into a pre-trained language model and output the semantic overlap confidence between alarms; The selection module is configured to select alarms with a semantic overlap confidence level higher than a preset confidence threshold as target alarms; The clustering module is configured to input the target alarm into the agent for clustering and generate alarm event clusters.

17. The apparatus according to claim 16, wherein, The generation module is further configured to: The multi-source heterogeneous alarm information is collected from the carrier that generates alarms within the data center, and the carrier includes at least one of the following: server, network device, middleware, application service; The multi-source heterogeneous alarm information is standardized to generate the structured alarm text. The standardization process includes at least one of the following: cleaning, denoising, field mapping, and regular expression masking.

18. The apparatus according to claim 16, wherein, The output module is further configured to: The structured alarm text is subjected to deep semantic encoding, and long-distance dependencies in the structured alarm text are captured through an attention mechanism to generate semantic overlap confidence between the alarms.

19. The apparatus according to claim 16, wherein, The clustering module is further configured to: Based on semantic overlap confidence, the target alarms are clustered by combining physical and logical attributes to generate the alarm event clusters.

20. The apparatus according to any one of claims 16-19, wherein, The device further includes: The acquisition module is configured to acquire user feedback results for the alarm event cluster; The annotation module is configured to annotate the structured alarm text with labels to indicate whether it is a real event alarm based on the feedback results, and generate an incremental sample set; The optimization module is configured to iteratively optimize the pre-trained language model using the sample set.

21. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-5 or 6-10.

22. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-5 or 6-10.

23. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5 or 6-10.

Images