Alarm data processing method and device, equipment and medium

Abstract

The invention relates to the technical field of network security, in particular to an alarm data processing method and device, equipment and a medium. In the embodiment of the invention, a first feature vector is determined based on an alarm name, an alarm description and an effective load carried in target alarm data, and a second feature vector is determined based on other alarm data similar to the target alarm data; therefore, the target risk level of the target alarm data can be predicted based on the first feature vector and the second feature vector, screening of the alarm data is realized, response to all the alarm data is avoided, and the alarm data processing efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to an alarm data processing method, device, equipment and medium. Background Art

[0002] In today's cybersecurity field, with the rapid development of information technology and Internet services, the types and number of security threats are also growing. Enterprises and organizations are facing more and more cyber attacks, such as malware, ransomware attacks, DDoS attacks, etc. These threats may lead to data leakage, business interruption and even serious financial losses. In order to effectively defend against these security threats, security teams rely on the alarm information generated by various security devices and systems to promptly detect and respond to potential security incidents.

[0003] In related technologies, security systems often generate a large amount of alarm data, and the security team needs to process all the alarm data. However, there are a large number of false alarms and duplicate alarm data in the alarm data processed by the security team. These large amounts of false alarms and duplicate alarm data seriously affect the efficiency and response speed to real alarm data. Summary of the invention

[0004] The present application provides an alarm data processing method, device, equipment and medium to solve the problem that a large amount of existing false alarms and repeated alarm data seriously affect the efficiency and response speed of real alarm data.

[0005] In a first aspect, an embodiment of the present application provides a method for processing alarm data, the method comprising:

[0006] Performing text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determining a first feature vector corresponding to the target alarm data;

[0007] Determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; perform one-hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data;

[0008] The first feature vector and the second feature vector are input into a prediction model to obtain a target risk level of the target warning data output by the prediction model, and if the target risk level exceeds a preset threshold, the warning data is responded to.

[0009] In a second aspect, an embodiment of the present application further provides an alarm data processing device, the device comprising:

[0010] A processing module, configured to perform text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determine a first feature vector corresponding to the target alarm data; determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; and perform unique hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data;

[0011] A prediction module is used to input the first feature vector and the second feature vector into a prediction model, obtain a target risk level of the target warning data output by the prediction model, and respond to the warning data if the target risk level exceeds a preset threshold.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device, which includes at least a processor and a memory, and the processor is used to implement the steps of any of the above-mentioned alarm data processing methods when executing a computer program stored in the memory.

[0013] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned alarm data processing methods.

[0014] In an embodiment of the present application, the electronic device performs text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determines the first feature vector corresponding to the target alarm data; determines other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; performs unique hot encoding according to the total number of other alarm data, and determines the second feature vector corresponding to the target alarm data; inputs the first feature vector and the second feature vector into the prediction model to obtain the target risk level of the target alarm data output by the prediction model, and responds to the alarm data if the target risk level exceeds a preset threshold. In an embodiment of the present application, the first feature vector is determined based on the alarm name, alarm description and payload carried in the target alarm data, and the second feature vector is determined based on other alarm data similar to the target alarm data, so that the target risk level of the target alarm data can be predicted based on the first feature vector and the second feature vector, thereby realizing the screening of the alarm data, avoiding responding to all the alarm data, and improving the efficiency and accuracy of the alarm data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A schematic diagram of an alarm data processing process provided in an embodiment of the present application;

[0017] Figure 2 A schematic diagram of processing the payload PayLoad provided in an embodiment of the present application;

[0018] Figure 3 An architectural diagram of the alarm data processing provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of the structure of an alarm data processing device provided in an embodiment of the present application;

[0020] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0022] In order to improve the efficiency and accuracy of alarm data processing, the embodiments of the present application provide an alarm data processing method, apparatus, device and medium.

[0023] In an embodiment of the present application, text embedding processing is performed on the alarm name, alarm description and payload carried in the target alarm data to be processed, and a first feature vector corresponding to the target alarm data is determined; other alarm data similar to the target alarm data is determined based on the destination IP, source IP and tenant ID corresponding to the target alarm data; unique hot encoding is performed based on the total number of other alarm data to determine a second feature vector corresponding to the target alarm data; the first feature vector and the second feature vector are input into a prediction model to obtain a target risk level of the target alarm data output by the prediction model, and if the target risk level exceeds a preset threshold, the alarm data is responded to.

[0024] Figure 1 A schematic diagram of an alarm data processing process provided in an embodiment of the present application, the process includes:

[0025] S101: Perform text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determine a first feature vector corresponding to the target alarm data.

[0026] An alarm data processing method provided in an embodiment of the present application is applied to an electronic device, which may be a PC or a server.

[0027] In related technologies, security systems often generate a large amount of alarm data, including a large number of false positives and duplicate alarm data. These low-quality alarm data not only waste the time and energy of security analysts, but may also cover up real security threats. According to industry reports, security teams need to deal with tens of thousands of alarms every day, most of which are ultimately proven to be harmless or insignificant. This "alarm fatigue" phenomenon seriously affects the efficiency of security teams and the speed of response to real threats.

[0028] In the face of this challenge, there is an urgent need for a technology that can effectively reduce alert noise and improve the quality of alert data. Ideal alert noise reduction technology should be able to distinguish between real security threats and routine, insignificant alerts, so that security analysts can focus on dealing with real security incidents. In addition, the language and text information involved in alert data are becoming increasingly diverse, which puts higher requirements on the automation and accuracy of alert processing.

[0029] Based on this, an embodiment of the present application provides an alarm data processing method for denoising a large amount of alarm data and screening out alarm data with a high risk level, thereby improving the processing quality of alarm data, reducing the workload of security analysts, and enhancing the response speed and accuracy to network security threats.

[0030] Specifically, the electronic device obtains the target alarm data to be processed, including the alarm name, alarm description and payload, and performs text embedding processing on the alarm name, alarm description and payload to obtain the first feature vector corresponding to the target alarm data. The first feature vector carries relevant information of the target alarm data, so that the target alarm data can be better screened according to the first feature vector.

[0031] S102: Determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; perform one-hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data.

[0032] In an embodiment of the present application, the target alarm data also carries the destination IP, source IP and tenant ID. The electronic device can search for other alarm data similar to the target alarm data in the received alarm data based on the destination IP, source IP and tenant ID carried in the target alarm data.

[0033] In practical applications, when an emergency occurs, a large amount of similar alarm data may be sent. Based on this, the electronic device can determine the timing characteristics of the target alarm data according to the total number of other alarm data similar to the target alarm data in the received alarm data.

[0034] Specifically, in the embodiment of the present application, the electronic device performs one-hot encoding according to the total number of other alarm data to determine the second feature vector corresponding to the target alarm data.

[0035] S103: Input the first feature vector and the second feature vector into a prediction model, obtain a target risk level of the target alarm data output by the prediction model, and respond to the alarm data if the target risk level exceeds a preset threshold.

[0036] In an embodiment of the present application, a trained prediction model is configured in the electronic device, and the electronic device can input the first feature vector and the second feature vector into the prediction model, so that the prediction model predicts based on the first feature vector and the second feature vector, determines and outputs the target risk level of the target alarm data. The electronic device obtains the target risk level of the target alarm data output by the prediction model, and responds to the alarm data if the target risk level exceeds a preset threshold.

[0037] Specifically, in an embodiment of the present application, the electronic device concatenates the first feature vector and the second feature vector to obtain an alarm classification and grading vector, and inputs the alarm classification and grading vector into a prediction model to obtain a target risk level.

[0038] The electronic device may concatenate the first feature vector and the second feature vector horizontally to generate an alarm classification and grading vector.

[0039] In addition, in the embodiment of the present application, the prediction model adopts a normalized (softmax) algorithm, and the prediction model has multiple outputs, each of which corresponds to a risk level. For example, the output of the prediction model can be 4, 3, 2, 1, and 0, corresponding to five risk levels of emergency, high, medium, low, and no alarm.

[0040] In the embodiment of the present application, the electronic device uses accuracy, recall rate and F1 value as evaluation indicators. During the training phase of the prediction model, the electronic device can adjust the parameters of the prediction model by the following formula:

[0041]

[0042] Among them, P represents precision, R represents recall, TP represents the number of correct predictions for positive examples, FP represents the number of incorrect predictions for negative examples, TN represents the number of correct predictions for negative examples, and FN represents the number of incorrect predictions for positive examples.

[0043] In an embodiment of the present application, a first feature vector is determined based on the alarm name, alarm description and payload carried in the target alarm data, and a second feature vector is determined based on other alarm data similar to the target alarm data, so that the target risk level of the target alarm data can be predicted based on the first feature vector and the second feature vector, thereby realizing the screening of alarm data, avoiding responding to all alarm data, and improving the efficiency and accuracy of alarm data processing.

[0044] Embodiment 2:

[0045] In order to improve the efficiency and accuracy of alarm data processing, based on the above embodiment, in the embodiment of the present application, the alarm name, alarm description and payload carried in the target alarm data to be processed are subjected to text embedding processing, and the first feature vector corresponding to the target alarm data is determined to include:

[0046] According to a preset text embedding algorithm, text embedding processing is performed on the alarm name, the alarm description and the payload respectively to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload;

[0047] Converting the first sub-feature vector, the second sub-feature vector, and the third sub-feature vector into sub-feature vectors of preset dimensions respectively;

[0048] The converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector are concatenated to obtain the first feature vector.

[0049] In an embodiment of the present application, since the formats of the alarm name, alarm description and payload are different, the information carried is also different. In order to better perform text embedding, in an embodiment of the present application, the electronic device performs text embedding processing on the alarm name, alarm description and payload carried in the target alarm data respectively.

[0050] Specifically, in an embodiment of the present application, the electronic device performs text embedding processing on the alarm name, alarm description and payload according to a preset text embedding algorithm, determines a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload; converts the first sub-feature vector, the second sub-feature vector and the third sub-feature vector into sub-feature vectors of preset dimensions respectively; and concatenates the converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector to obtain a first feature vector.

[0051] Among them, in an embodiment of the present application, the electronic device can use the BERT model for text embedding processing. Generally, the dimension of the sub-feature vector output by the BERT model is 512*512, and the electronic device needs to convert the 512*512 dimensional sub-feature vector to 1*512 dimension. Specifically, the electronic device can perform maximum pooling (MaxPooling) in the horizontal dimension to generate a vector of 1*512 dimension.

[0052] For example, the electronic device performs text embedding processing on the alarm name field. Most of the alarm name fields are English fields. The electronic device can embed the alarm name through the BERT model to generate a first sub-feature vector of 512*512 dimension, and then perform MaxPooling on the horizontal dimension to convert the first sub-feature vector of 512*512 dimension into 1*512 dimension.

[0053] The electronic device alarm description is text embedded. Most of the alarm description fields are English fields. The electronic device can embed the alarm description through the BERT model to generate a second sub-feature vector of 512*512 dimension, and then perform MaxPooling on the horizontal dimension to convert the second sub-feature vector of 512*512 dimension into 1*512 dimension.

[0054] Embodiment 3:

[0055] In order to improve the efficiency and accuracy of alarm data processing, based on the above embodiments, in the embodiment of the present application, the alarm name and the alarm description are in English format, and the payload is in Chinese format;

[0056] The step of performing text embedding processing on the alarm name, the alarm description, and the payload respectively according to a preset text embedding algorithm to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description, and a third sub-feature vector corresponding to the payload includes:

[0057] Using a BERT model trained based on an English-formatted data set to perform text embedding processing on the alarm name and the alarm description, respectively, to obtain a first sub-feature vector corresponding to the alarm name and a second sub-feature vector corresponding to the alarm description;

[0058] The ChineseBERT model trained based on a data set in a Chinese format is used to perform text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload.

[0059] In an embodiment of the present application, the alarm name and alarm description are in English format, and the payload is in Chinese format. If the same model is used to perform text embedding on the alarm name, alarm description and payload, the accuracy of the first feature vector finally obtained is low.

[0060] Based on this, in order to improve the accuracy of the first feature vector, in an embodiment of the present application, the electronic device separates Chinese and English texts so that specially optimized text processing strategies can be used for different languages, thereby effectively improving the accuracy and processing speed of information extraction.

[0061] Specifically, the electronic device pre-trains a data set in English format to obtain a BERT model for text embedding processing of text in English format, and pre-trains a data set in Chinese format to obtain a ChineseBERT model for text embedding processing of text in Chinese format.

[0062] After the electronic device obtains the alarm name, alarm description and payload carried in the target alarm data, if the electronic device determines that the alarm name is in English format, the electronic device inputs the alarm name into the BERT model trained using a data set based on the English format, so that the BERT model performs text embedding on the alarm name to obtain a first sub-feature vector corresponding to the alarm name; if the electronic device determines that the alarm description is in English format, the electronic device inputs the alarm description into the BERT model trained using a data set based on the English format, so that the BERT model performs text embedding on the alarm description to obtain a second sub-feature vector corresponding to the alarm description; if the electronic device determines that the payload is in Chinese format, the electronic device inputs the payload into the ChineseBERT model trained using a data set based on the Chinese format, so that the ChineseBERT model performs text embedding on the payload to obtain a third sub-feature vector corresponding to the payload.

[0063] In addition, in some special scenarios of the embodiments of the present application, the alarm name and alarm description can also be in Chinese format, and the payload can also be in English format. If the alarm name and alarm description are in Chinese format, the ChineseBERT model is used to perform text embedding processing on the alarm name and alarm description respectively; if the payload is in English format, the BERT model is used to perform text embedding processing on the payload, without limitation here.

[0064] That is, in the embodiments of the present application, by achieving the separation of Chinese and English texts, specially optimized text processing strategies can be adopted for different languages, and the ChineseBERT model is used to perform text embedding processing on texts in Chinese format, and the BERT model is used to perform text embedding processing on texts in English format.

[0065] Embodiment 4:

[0066] In order to improve the efficiency and accuracy of alarm data processing, based on the above embodiments, in the embodiment of the present application, the ChineseBERT model trained based on the Chinese format data set performs text embedding processing on the payload, and the third sub-feature vector corresponding to the payload is obtained, including:

[0067] Determining whether the character length corresponding to the payload exceeds a preset character length threshold;

[0068] If not, inputting the payload into the ChineseBERT model, and determining the output vector of the ChineseBERT model as the third sub-feature vector;

[0069] If the character length exceeds the preset character length threshold, the payload is segmented according to the preset character length threshold to obtain at least two sub-payloads; wherein the sub-character length corresponding to the sub-payload does not exceed the preset character length threshold; at least two sub-payloads are respectively input into the ChineseBERT model to obtain each sub-output vector corresponding to each sub-payload; each sub-output vector is subjected to dimensionality reduction processing, and each sub-output vector after dimensionality reduction is spliced ​​according to the position of each sub-payload in the payload to obtain the third sub-feature vector.

[0070] In practical applications, the alarm text contains a lot of key information such as payload, which is crucial for accurate identification and processing of alarms. The payload field often contains important information, and the character length of the payload carried in each alarm data is different. The ChineseBERT model has limited processing capabilities for text, so there may be a situation where the character length of the payload to be processed exceeds the maximum processing capability of the ChineseBERT model.

[0071] Based on this, in an embodiment of the present application, a preset character length threshold is pre-configured in the electronic device, and the preset character length does not exceed the maximum character length of the text that the ChineseBERT model can process. For example, if the ChineseBERT model can process a maximum of 512 characters, then the preset character length threshold does not exceed 512. The electronic device determines whether the ChineseBERT model supports text embedding processing for a payload of this character length based on the character length of the payload to be processed and the preset character length threshold. If the electronic device determines that the ChineseBERT model does not support text embedding processing for a payload of this character length, the electronic device segments the payload and implements text embedding processing for the payload by performing text embedding processing on each sub-payload obtained by the segmentation.

[0072] Specifically, in the embodiment of the present application, the electronic device determines whether the character length corresponding to the payload exceeds a preset character length threshold. If the electronic device determines that the character length does not exceed the preset character length threshold, the payload is input into the ChineseBERT model, and the output vector of the ChineseBERT model is determined as the third sub-feature vector.

[0073] If the electronic device determines that the character length exceeds the preset character length threshold, the electronic device divides the payload according to the preset character length threshold to obtain at least two sub-payloads; wherein the sub-character length corresponding to the sub-payload does not exceed the preset character length threshold; the at least two sub-payloads are respectively input into the ChineseBERT model to obtain each sub-output vector corresponding to each sub-payload; each sub-output vector is subjected to dimensionality reduction processing, and each sub-output vector after dimensionality reduction is spliced ​​according to the position of each sub-payload in the payload to obtain a third sub-feature vector.

[0074] It should be noted that in the embodiment of the present application, the lateral dimension (the number of columns of the matrix) of the third sub-feature vector is consistent with the lateral dimension of the vector output by the ChineseBERT model. In the scenario where the character length of the payload exceeds the preset character length threshold, the lateral dimension of each sub-output vector output by the ChineseBERT model is also consistent with the lateral dimension of the third sub-feature vector to be determined. Therefore, it is necessary to perform dimensionality reduction processing on the sub-output vector, and the dimension of dimensionality reduction is related to the number of sub-payloads and the lateral dimension of the third sub-feature vector to be determined, so that the dimension of the third sub-feature vector finally spliced ​​meets the requirements. For example, the number of sub-payloads is 2, and the dimension of the third sub-feature vector is 512*512, then the sub-output vector after dimensionality reduction is 256*512.

[0075] Figure 2 A schematic diagram of processing the effective load PayLoad provided in the embodiment of the present application, such as the Figure 2 As shown, the preset character length threshold is 512. The electronic device determines the character length of PayLoad. If the character length is less than 512, the electronic device generates a third sub-feature vector of 512*512 dimension through the ChineseBERT model, and converts the third sub-feature vector into a vector of 1*512 dimension through MaxPooling. If the character length is greater than 512, the electronic device obtains two sub-payloads according to the nearest segmentation step, generates two 512*512 matrices (sub-output vectors) for each of the two sub-payloads through the ChineseBERT model, generates two 256*256 matrices (sub-output vectors after dimensionality reduction) through 2*2 MaxPooling, generates a 256*512 matrix through horizontal splicing, and then generates a 512-dimensional third sub-feature vector of the horizontal dimension through another layer of MaxPooling.

[0076] Embodiment 5:

[0077] In order to improve the efficiency and accuracy of alarm data processing, on the basis of the above embodiments, in the embodiment of the present application, the other alarm data similar to the target alarm data is determined according to the destination IP, source IP and tenant ID corresponding to the target alarm data, including:

[0078] Determine that the corresponding generation time is within a first preset time range and carries the first alarm data of the target IP, source IP and tenant ID;

[0079] Determine that the corresponding generation time is within a second preset time range and carries the second alarm data of the target IP and the source IP; wherein the first preset time range and the second preset time range have the same time length, the target generation time of the target alarm data is the end time of the first preset time range, and the target generation time is the middle time of the second preset time range;

[0080] Determine that the corresponding generation time is within the second preset time range and the third alarm data carries the tenant ID;

[0081] The first alarm data, the second alarm data and the third alarm data are determined as the other alarm data.

[0082] In the embodiment of the present application, other alarm data similar to the target alarm data may be screened based on the time series characteristics.

[0083] Specifically, the electronic device determines that the corresponding generation time is within the first preset time range and carries the first alarm data of the target IP, source IP and tenant ID; the electronic device determines that the corresponding generation time is within the second preset time range and carries the second alarm data of the target IP and source IP, and determines that the corresponding generation time is within the second preset time range and carries the third alarm data of the tenant ID. The electronic device determines the first alarm data, the second alarm data and the third alarm data as other alarm data.

[0084] The first preset time range has the same time length as the second preset time range, the target generation time of the target alarm data is the end time of the first preset time range, and the target generation time is the middle time of the second preset time range.

[0085] Generally, the first preset time range is 30 minutes before the target generation time of the target alarm data, and the second preset time range is 15 minutes before and 15 minutes after the target generation time of the target alarm data.

[0086] Embodiment 6:

[0087] In order to improve the efficiency and accuracy of alarm data processing, based on the above embodiments, in the embodiment of the present application, the one-hot encoding according to the total number of the other alarm data includes:

[0088] Determine a first target bucket corresponding to the first alarm data according to a range corresponding to each bucket in the pre-saved bucket classification and a first quantity corresponding to the first alarm data, perform one-hot encoding on a first identifier corresponding to the first target bucket, and determine a fourth sub-feature vector corresponding to the first alarm data;

[0089] Determine a second target bucket corresponding to the second alarm data according to the range corresponding to each bucket and the second quantity corresponding to the second alarm data, perform one-hot encoding on the second identifier corresponding to the second target bucket, and determine a fifth sub-feature vector corresponding to the second alarm data;

[0090] Determine, according to the range corresponding to each bucket and the third quantity corresponding to the third alarm data, a third target bucket corresponding to the third alarm data, perform one-hot encoding on the third identifier corresponding to the third target bucket, and determine a sixth sub-feature vector corresponding to the third alarm data;

[0091] The fourth sub-feature vector, the fifth sub-feature vector and the sixth sub-feature vector are concatenated to obtain the second feature vector.

[0092] In the embodiment of the present application, when the electronic device determines the first feature vector, the electronic device determines the first feature vector by bucketing, grading and one-hot encoding for other alarm data with different similarities.

[0093] The electronic device determines the first target bucket corresponding to the first alarm data based on the range corresponding to each bucket in the pre-saved bucket classification and the first quantity corresponding to the first alarm data, performs one-hot encoding on the first identifier corresponding to the first target bucket, and determines the fourth sub-feature vector corresponding to the first alarm data.

[0094] Specifically, the electronic device buckets and grades the first quantity of the first alarm data generated within 30 minutes before the target generation time of the target alarm data and having the same source IP, destination IP, and tenant ID. Among them, when the first quantity is 0, the first target bucket obtained by grading is 0; when the first quantity is any one of 1-1000, the first target bucket obtained by grading is 1; when the first quantity is any one of 1000-2000, the first target bucket obtained by grading is 2, and so on. After determining the first target bucket, the electronic device performs one-hot encoding to generate a 256-dimensional fourth sub-feature vector.

[0095] The electronic device determines the second target bucket corresponding to the second alarm data according to the range corresponding to each bucket and the second quantity corresponding to the second alarm data, performs one-hot encoding on the second identifier corresponding to the second target bucket, and determines the fifth sub-feature vector corresponding to the second alarm data.

[0096] Specifically, the electronic device buckets and grades the second number of second alarm data generated within 15 minutes before and after the target generation time of the target alarm data and having the same source IP and destination IP. Among them, when the second number is 0, the second target bucket obtained by classification is 0; when the second number is any one of 1-1000, the second target bucket obtained by classification is 1; when the second number is any one of 1000-2000, the second target bucket obtained by classification is 2, and so on. After determining the second target bucket, the electronic device performs one-hot encoding to generate a 256-dimensional fifth sub-feature vector.

[0097] The electronic device determines the third target bucket corresponding to the third alarm data according to the range corresponding to each bucket and the third quantity corresponding to the third alarm data, performs one-hot encoding on the third identifier corresponding to the third target bucket, and determines the sixth sub-feature vector corresponding to the third alarm data.

[0098] Specifically, the electronic device buckets and grades the third quantity of the third alarm data generated within 15 minutes before and after the target generation time of the target alarm data and having the same tenant ID. Among them, when the third quantity is 0, the third target bucket obtained by grading is 0; when the third quantity is any one of 1-1000, the third target bucket obtained by grading is 1; when the third quantity is any one of 1000-2000, the third target bucket obtained by grading is 2, and so on. After determining the third target bucket, the electronic device performs one-hot encoding to generate a 256-dimensional sixth sub-feature vector.

[0099] The electronic device concatenates the fourth sub-feature vector, the fifth sub-feature vector and the sixth sub-feature vector to obtain a second feature vector.

[0100] The main purpose of the embodiments of the present application is to provide a method for reducing the noise of alarm data in the security field, which can significantly improve the processing quality of alarm data, reduce the workload of security analysts, and enhance the response speed and accuracy to network security threats. Specifically, the present invention mainly solves the following problems:

[0101] 1. Ignoring the problem of time correlation: By introducing timing features, the embodiments of the present application can consider the time correlation between alarm events, effectively identify and filter out alarms that occur frequently in a short period of time due to the same reason, thereby reducing the false alarm rate and improving the accuracy of alarm processing.

[0102] Problems with multi-language text processing: Security alerts often involve multiple languages, especially Chinese and English. The embodiments of the present application separate Chinese and English texts, so that specially optimized text processing strategies can be used for different languages, effectively improving the accuracy and processing speed of information extraction.

[0103] Insufficient extraction of key information: Alarm data contains a lot of key information such as payload, which is crucial for accurate identification and processing of alarms. The project has enhanced the information recognition ability of the model by expanding the representation length of this key information, ensuring that it can more accurately capture the details that are crucial to alarm analysis.

[0104] Through the application of these technologies, the embodiments of the present application can not only significantly reduce false positives and irrelevant alarms, but also improve the security team's attention to real threats and processing efficiency. The ultimate goal is to provide enterprises and organizations with a more intelligent, efficient and accurate network security defense mechanism to ensure the security of information assets and maintain business continuity and stability.

[0105] In the security field, the alarm system is a key component used to monitor and warn of potential threats and abnormal behaviors. However, these systems often generate a large amount of alarm data, many of which are false alarms and duplicate alarms, which seriously affect the efficiency of the security operation and maintenance team. Faced with a large amount of alarm data, how to effectively filter out truly valuable information from it has become a problem that needs to be solved urgently. The embodiment of the present application proposes a new alarm denoising method. By introducing time series features, realizing automatic separation of Chinese and English texts, and expanding the length of key information in the alarm text, the embodiment of the present application significantly improves the processing accuracy and efficiency of alarm data.

[0106] First, the embodiments of the present application introduce time series features: Traditional alarm noise reduction techniques often ignore the time series information of alarm occurrence, which limits the effectiveness of the method when processing real-time data. By introducing two time series features, the embodiments of the present application can consider the temporal correlation of alarm events. For example, similar alarms that occur frequently in a short period of time may belong to the same threat or system error. Using time series analysis, alarm events can be predicted and classified more accurately, thereby improving the accuracy of noise reduction.

[0107] Second, the English text separation of the embodiment of the present application: In a globalized network environment, security alerts often contain multiple languages, especially Chinese and English. When the pre-training vectors are generated based on the structural and semantic differences of different languages, the accuracy of text processing and information extraction may be interfered with. The embodiment of the present application improves the accuracy and efficiency of text data processing by implementing the automatic separation of Chinese and English texts and independently processing text data in different languages.

[0108] Thirdly, the embodiment of the present application expands the length of the key alarm information: Alarm data often contains dense key information, such as the payload key field, etc., which is crucial for alarm analysis. The embodiment of the present application enhances the model's ability to identify key information in the alarm text by expanding the representation length of these key information. This not only helps to improve the recognition rate of key information, but also improves the accuracy and reliability of subsequent classification and analysis processes through detailed information features.

[0109] Based on the above embodiments, the alarm data processing provided in the embodiments of the present application can be implemented and used in multiple scenarios, and specific embodiments include but are not limited to the following:

[0110] 1. Intrusion Detection System (IDS):

[0111] In scenarios where network traffic is monitored in real time to detect potential threats and abnormal behaviors, accurate alert levels are determined to increase security analysts’ attention and response speed to real threats by reducing false positives and missed negatives.

[0112] 2. Security Information and Event Management (SIEM) System:

[0113] Centrally manage security warnings within the enterprise. The alarm data processing method provided in the embodiment of the present application can effectively help analysts filter out key events from massive security logs, determine events that require alarms, and optimize the event response process.

[0114] 3. Cloud security and virtualized environment monitoring:

[0115] In cloud computing and virtualization environments, security events of virtual machines and cloud services are monitored and analyzed. The alarm data processing method provided in the embodiment of the present application can help improve the security of the cloud environment and reduce the number of alarms caused by environmental complexity.

[0116] Figure 3 The architecture diagram of the alarm data processing provided in the embodiment of the present application is as follows: Figure 3As shown, the electronic device determines the first quantity (src_ip&dst_ip&tenantld) of the first alarm data that is consistent with the destination IP, source IP and tenant ID of the target alarm data 30 minutes (Before 30min) before the target generation time of the target alarm data, and one-hot encodes the first quantity to obtain the fourth sub-feature vector, the electronic device determines the second quantity (src_ip&dst_ip) of the second alarm data that is consistent with the destination IP and source IP of the target alarm data 15 minutes (In 30min) before and after the target generation time, determines the third quantity (tenant ID) of the third alarm data that is consistent with the tenant ID of the target alarm data 15 minutes (In 30min) before and after the target generation time, and one-hot encodes the second quantity and the third quantity respectively to obtain the fifth sub-feature vector and the sixth sub-feature vector.

[0117] The electronic device uses the BERT model to perform text embedding processing on the alarm name and alarm description respectively, and uses MaxPooling to process the output of the BERT model to obtain the first sub-feature vector and the second sub-feature vector of the preset dimension. The electronic device uses the ChineseBERT model to perform text embedding processing on PayLoad, and uses MaxPooling to process the output of the ChineseBERT model to obtain the third sub-feature vector of the preset dimension.

[0118] The electronic device horizontally splices the fourth sub-feature vector, the fifth sub-feature vector, the sixth sub-feature vector, the first sub-feature vector, the second sub-feature vector and the third sub-feature vector to obtain an alarm classification and grading vector. The electronic device predicts the alarm classification and grading vector based on the trained softmax model to determine the target risk level of the target alarm data.

[0119] Embodiment 7:

[0120] Based on the above embodiments, Figure 4 A schematic diagram of the structure of an alarm data processing device provided in an embodiment of the present application, the device comprising:

[0121] The processing module 401 is used to perform text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determine the first feature vector corresponding to the target alarm data; determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; perform one-hot encoding according to the total number of the other alarm data, and determine the second feature vector corresponding to the target alarm data;

[0122] The prediction module 402 is used to input the first feature vector and the second feature vector into the prediction model, obtain the target risk level of the target alarm data output by the prediction model, and respond to the alarm data if the target risk level exceeds a preset threshold.

[0123] In a possible implementation, the processing module 401 is specifically used to perform text embedding processing on the alarm name, alarm description and payload according to a preset text embedding algorithm, determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload; convert the first sub-feature vector, the second sub-feature vector and the third sub-feature vector into sub-feature vectors of preset dimensions respectively; and concatenate the converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector to obtain the first feature vector.

[0124] In a possible implementation manner, the alarm name and the alarm description are in English format, and the payload is in Chinese format;

[0125] The processing module 401 is specifically used to use the BERT model obtained by training based on the English format data set to perform text embedding processing on the alarm name and the alarm description respectively, so as to obtain a first sub-feature vector corresponding to the alarm name and a second sub-feature vector corresponding to the alarm description; and use the ChineseBERT model obtained by training based on the Chinese format data set to perform text embedding processing on the payload, so as to obtain a third sub-feature vector corresponding to the payload.

[0126] In one possible implementation, the processing module 401 is specifically used to determine whether the character length corresponding to the payload exceeds a preset character length threshold; if not, the payload is input into the ChineseBERT model, and the output vector of the ChineseBERT model is determined as the third sub-feature vector; if the character length exceeds the preset character length threshold, the payload is segmented according to the preset character length threshold to obtain at least two sub-payloads; wherein the sub-character length corresponding to the sub-payload does not exceed the preset character length threshold; at least two sub-payloads are respectively input into the ChineseBERT model to obtain each sub-output vector corresponding to each sub-payload; each sub-output vector is subjected to dimensionality reduction processing, and each sub-output vector after dimensionality reduction is spliced ​​according to the position of each sub-payload in the payload to obtain the third sub-feature vector.

[0127] In a possible implementation, the processing module 401 is specifically used to determine that the corresponding generation time is within a first preset time range, and carries the first alarm data of the target IP, source IP and tenant ID; determine that the corresponding generation time is within a second preset time range, and carries the second alarm data of the target IP and source IP; wherein the first preset time range is the same as the time length of the second preset time range, the target generation time of the target alarm data is the end time of the first preset time range, and the target generation time is the middle time of the second preset time range; determine that the corresponding generation time is within the second preset time range, and carries the third alarm data of the tenant ID; determine the first alarm data, the second alarm data and the third alarm data as the other alarm data.

[0128] In a possible implementation, the processing module 401 is specifically used to determine the first target bucket corresponding to the first alarm data according to the range corresponding to each bucket in the pre-saved bucket classification and the first quantity corresponding to the first alarm data, one-hot encode the first identifier corresponding to the first target bucket, and determine the fourth sub-feature vector corresponding to the first alarm data; determine the second target bucket corresponding to the second alarm data according to the range corresponding to each bucket and the second quantity corresponding to the second alarm data, one-hot encode the second identifier corresponding to the second target bucket, and determine the fifth sub-feature vector corresponding to the second alarm data; determine the third target bucket corresponding to the third alarm data according to the range corresponding to each bucket and the third quantity corresponding to the third alarm data, one-hot encode the third identifier corresponding to the third target bucket, and determine the sixth sub-feature vector corresponding to the third alarm data; concatenate the fourth sub-feature vector, the fifth sub-feature vector and the sixth sub-feature vector to obtain the second feature vector.

[0129] Embodiment 8:

[0130] Based on the above embodiments, the present application also provides an electronic device, Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, it includes: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0131] The memory 503 stores a computer program. When the program is executed by the processor 501, the processor 501 performs the following steps:

[0132] Performing text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determining a first feature vector corresponding to the target alarm data;

[0133] Determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; perform one-hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data;

[0134] The first feature vector and the second feature vector are input into a prediction model to obtain a target risk level of the target warning data output by the prediction model, and if the target risk level exceeds a preset threshold, the warning data is responded to.

[0135] In a possible implementation manner, performing text embedding processing on the alarm name, alarm description, and payload carried in the target alarm data to be processed to determine the first feature vector corresponding to the target alarm data includes:

[0136] According to a preset text embedding algorithm, text embedding processing is performed on the alarm name, the alarm description and the payload respectively to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload;

[0137] Converting the first sub-feature vector, the second sub-feature vector, and the third sub-feature vector into sub-feature vectors of preset dimensions respectively;

[0138] The converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector are concatenated to obtain the first feature vector.

[0139] In a possible implementation manner, the alarm name and the alarm description are in English format, and the payload is in Chinese format;

[0140] The step of performing text embedding processing on the alarm name, the alarm description, and the payload respectively according to a preset text embedding algorithm to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description, and a third sub-feature vector corresponding to the payload includes:

[0141] Using a BERT model trained based on an English-formatted data set to perform text embedding processing on the alarm name and the alarm description, respectively, to obtain a first sub-feature vector corresponding to the alarm name and a second sub-feature vector corresponding to the alarm description;

[0142] The ChineseBERT model trained based on a data set in a Chinese format is used to perform text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload.

[0143] In a possible implementation, the ChineseBERT model trained based on a data set in a Chinese format performs text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload, including:

[0144] Determining whether the character length corresponding to the payload exceeds a preset character length threshold;

[0145] If not, inputting the payload into the ChineseBERT model, and determining the output vector of the ChineseBERT model as the third sub-feature vector;

[0146] If the character length exceeds the preset character length threshold, the payload is segmented according to the preset character length threshold to obtain at least two sub-payloads; wherein the sub-character length corresponding to the sub-payload does not exceed the preset character length threshold; at least two sub-payloads are respectively input into the ChineseBERT model to obtain each sub-output vector corresponding to each sub-payload; each sub-output vector is subjected to dimensionality reduction processing, and each sub-output vector after dimensionality reduction is spliced ​​according to the position of each sub-payload in the payload to obtain the third sub-feature vector.

[0147] In a possible implementation manner, determining other alarm data similar to the target alarm data according to the destination IP, source IP, and tenant ID corresponding to the target alarm data includes:

[0148] Determine that the corresponding generation time is within a first preset time range and carries the first alarm data of the target IP, source IP and tenant ID;

[0149] Determine that the corresponding generation time is within a second preset time range and carries the second alarm data of the target IP and the source IP; wherein the first preset time range and the second preset time range have the same time length, the target generation time of the target alarm data is the end time of the first preset time range, and the target generation time is the middle time of the second preset time range;

[0150] Determine that the corresponding generation time is within the second preset time range and the third alarm data carries the tenant ID;

[0151] The first alarm data, the second alarm data and the third alarm data are determined as the other alarm data.

[0152] In a possible implementation, performing one-hot encoding according to the total number of the other alarm data includes:

[0153] Determine a first target bucket corresponding to the first alarm data according to a range corresponding to each bucket in the pre-saved bucket classification and a first quantity corresponding to the first alarm data, perform one-hot encoding on a first identifier corresponding to the first target bucket, and determine a fourth sub-feature vector corresponding to the first alarm data;

[0154] Determine a second target bucket corresponding to the second alarm data according to the range corresponding to each bucket and the second quantity corresponding to the second alarm data, perform one-hot encoding on the second identifier corresponding to the second target bucket, and determine a fifth sub-feature vector corresponding to the second alarm data;

[0155] Determine, according to the range corresponding to each bucket and the third quantity corresponding to the third alarm data, a third target bucket corresponding to the third alarm data, perform one-hot encoding on the third identifier corresponding to the third target bucket, and determine a sixth sub-feature vector corresponding to the third alarm data;

[0156] The fourth sub-feature vector, the fifth sub-feature vector and the sixth sub-feature vector are concatenated to obtain the second feature vector.

[0157] Since the principle of solving the problem by the above electronic device is similar to the alarm data processing method, the implementation of the above electronic device can refer to the embodiment of the method, and the repeated parts will not be repeated.

[0158] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 502 is used for communication between the above electronic device and other devices. The memory may include a random access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0159] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0160] Embodiment 9:

[0161] On the basis of the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor implements the following steps when executing:

[0162] Performing text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determining a first feature vector corresponding to the target alarm data;

[0163] Determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; perform one-hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data;

[0164] The first feature vector and the second feature vector are input into a prediction model to obtain a target risk level of the target warning data output by the prediction model, and if the target risk level exceeds a preset threshold, the warning data is responded to.

[0165] In a possible implementation manner, performing text embedding processing on the alarm name, alarm description, and payload carried in the target alarm data to be processed to determine the first feature vector corresponding to the target alarm data includes:

[0166] According to a preset text embedding algorithm, text embedding processing is performed on the alarm name, the alarm description and the payload respectively to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload;

[0167] Converting the first sub-feature vector, the second sub-feature vector, and the third sub-feature vector into sub-feature vectors of preset dimensions respectively;

[0168] The converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector are concatenated to obtain the first feature vector.

[0169] In a possible implementation manner, the alarm name and the alarm description are in English format, and the payload is in Chinese format;

[0170] The step of performing text embedding processing on the alarm name, the alarm description, and the payload respectively according to a preset text embedding algorithm to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description, and a third sub-feature vector corresponding to the payload includes:

[0171] Using a BERT model trained based on an English-formatted data set to perform text embedding processing on the alarm name and the alarm description, respectively, to obtain a first sub-feature vector corresponding to the alarm name and a second sub-feature vector corresponding to the alarm description;

[0172] The ChineseBERT model trained based on a data set in a Chinese format is used to perform text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload.

[0173] In a possible implementation, the ChineseBERT model trained based on a data set in a Chinese format performs text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload, including:

[0174] Determining whether the character length corresponding to the payload exceeds a preset character length threshold;

[0175] If not, inputting the payload into the ChineseBERT model, and determining the output vector of the ChineseBERT model as the third sub-feature vector;

[0176] If the character length exceeds the preset character length threshold, the payload is segmented according to the preset character length threshold to obtain at least two sub-payloads; wherein the sub-character length corresponding to the sub-payload does not exceed the preset character length threshold; at least two sub-payloads are respectively input into the ChineseBERT model to obtain each sub-output vector corresponding to each sub-payload; each sub-output vector is subjected to dimensionality reduction processing, and each sub-output vector after dimensionality reduction is spliced ​​according to the position of each sub-payload in the payload to obtain the third sub-feature vector.

[0177] In a possible implementation manner, determining other alarm data similar to the target alarm data according to the destination IP, source IP, and tenant ID corresponding to the target alarm data includes:

[0178] Determine that the corresponding generation time is within a first preset time range and carries the first alarm data of the target IP, source IP and tenant ID;

[0179] Determine that the corresponding generation time is within a second preset time range and carries the second alarm data of the target IP and the source IP; wherein the first preset time range and the second preset time range have the same time length, the target generation time of the target alarm data is the end time of the first preset time range, and the target generation time is the middle time of the second preset time range;

[0180] Determine that the corresponding generation time is within the second preset time range and the third alarm data carries the tenant ID;

[0181] The first alarm data, the second alarm data and the third alarm data are determined as the other alarm data.

[0182] In a possible implementation, performing one-hot encoding according to the total number of the other alarm data includes:

[0183] Determine a first target bucket corresponding to the first alarm data according to a range corresponding to each bucket in the pre-saved bucket classification and a first quantity corresponding to the first alarm data, perform one-hot encoding on a first identifier corresponding to the first target bucket, and determine a fourth sub-feature vector corresponding to the first alarm data;

[0184] Determine a second target bucket corresponding to the second alarm data according to the range corresponding to each bucket and the second quantity corresponding to the second alarm data, perform one-hot encoding on the second identifier corresponding to the second target bucket, and determine a fifth sub-feature vector corresponding to the second alarm data;

[0185] Determine, according to the range corresponding to each bucket and the third quantity corresponding to the third alarm data, a third target bucket corresponding to the third alarm data, perform one-hot encoding on the third identifier corresponding to the third target bucket, and determine a sixth sub-feature vector corresponding to the third alarm data;

[0186] The fourth sub-feature vector, the fifth sub-feature vector and the sixth sub-feature vector are concatenated to obtain the second feature vector.

[0187] Since the principle of solving the problem by the above-mentioned computer program product is similar to that of the alarm data processing method, the implementation of the above-mentioned computer program product can refer to the implementation of the method, and the repeated parts will not be repeated.

[0188] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0189] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0192] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for processing alarm data, characterized in that: The method comprises: Performing text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determining a first feature vector corresponding to the target alarm data; Determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; perform one-hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data; The first feature vector and the second feature vector are input into a prediction model to obtain a target risk level of the target warning data output by the prediction model, and if the target risk level exceeds a preset threshold, the warning data is responded to.

2. The method according to claim 1, characterized in that The performing text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed to determine the first feature vector corresponding to the target alarm data includes: According to a preset text embedding algorithm, text embedding processing is performed on the alarm name, the alarm description and the payload respectively to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload; Converting the first sub-feature vector, the second sub-feature vector, and the third sub-feature vector into sub-feature vectors of preset dimensions respectively; The converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector are concatenated to obtain the first feature vector.

3. The method according to claim 2, characterized in that: The alarm name and the alarm description are in English format, and the payload is in Chinese format; The step of performing text embedding processing on the alarm name, the alarm description, and the payload respectively according to a preset text embedding algorithm to determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description, and a third sub-feature vector corresponding to the payload includes: Using a BERT model trained based on a data set in an English format to perform text embedding processing on the alarm name and the alarm description, respectively, to obtain a first sub-feature vector corresponding to the alarm name and a second sub-feature vector corresponding to the alarm description; The ChineseBERT model trained based on a data set in a Chinese format is used to perform text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload.

4. The method according to claim 3, characterized in that The ChineseBERT model trained based on the Chinese format data set performs text embedding processing on the payload to obtain a third sub-feature vector corresponding to the payload, including: Determining whether the character length corresponding to the payload exceeds a preset character length threshold; If not, inputting the payload into the ChineseBERT model, and determining the output vector of the ChineseBERT model as the third sub-feature vector; If the character length exceeds the preset character length threshold, the payload is segmented according to the preset character length threshold to obtain at least two sub-payloads; wherein the sub-character length corresponding to the sub-payload does not exceed the preset character length threshold; at least two sub-payloads are respectively input into the ChineseBERT model to obtain each sub-output vector corresponding to each sub-payload; each sub-output vector is subjected to dimensionality reduction processing, and each sub-output vector after dimensionality reduction is spliced ​​according to the position of each sub-payload in the payload to obtain the third sub-feature vector.

5. The method according to claim 1, characterized in that The determining, according to the destination IP, source IP and tenant ID corresponding to the target alarm data, other alarm data similar to the target alarm data includes: Determine that the corresponding generation time is within a first preset time range and carries the first alarm data of the target IP, source IP and tenant ID; Determine that the corresponding generation time is within a second preset time range and carries the second alarm data of the target IP and the source IP; wherein the first preset time range and the second preset time range have the same time length, the target generation time of the target alarm data is the end time of the first preset time range, and the target generation time is the middle time of the second preset time range; Determine that the corresponding generation time is within the second preset time range and the third alarm data carries the tenant ID; The first alarm data, the second alarm data and the third alarm data are determined as the other alarm data.

6. The method according to claim 5, characterized in that The performing one-hot encoding according to the total number of the other alarm data includes: Determine a first target bucket corresponding to the first alarm data according to a range corresponding to each bucket in the pre-saved bucket classification and a first quantity corresponding to the first alarm data, perform one-hot encoding on a first identifier corresponding to the first target bucket, and determine a fourth sub-feature vector corresponding to the first alarm data; Determine a second target bucket corresponding to the second alarm data according to the range corresponding to each bucket and the second quantity corresponding to the second alarm data, perform one-hot encoding on the second identifier corresponding to the second target bucket, and determine a fifth sub-feature vector corresponding to the second alarm data; Determine, according to the range corresponding to each bucket and the third quantity corresponding to the third alarm data, a third target bucket corresponding to the third alarm data, perform one-hot encoding on the third identifier corresponding to the third target bucket, and determine a sixth sub-feature vector corresponding to the third alarm data; The fourth sub-feature vector, the fifth sub-feature vector and the sixth sub-feature vector are concatenated to obtain the second feature vector.

7. An alarm data processing device, characterized in that: The device comprises: A processing module, configured to perform text embedding processing on the alarm name, alarm description and payload carried in the target alarm data to be processed, and determine a first feature vector corresponding to the target alarm data; determine other alarm data similar to the target alarm data according to the destination IP, source IP and tenant ID corresponding to the target alarm data; and perform unique hot encoding according to the total number of the other alarm data to determine a second feature vector corresponding to the target alarm data; A prediction module is used to input the first feature vector and the second feature vector into a prediction model, obtain a target risk level of the target warning data output by the prediction model, and respond to the warning data if the target risk level exceeds a preset threshold.

8. The device according to claim 7, characterized in that The processing module is specifically used to perform text embedding processing on the alarm name, alarm description and payload according to a preset text embedding algorithm, determine a first sub-feature vector corresponding to the alarm name, a second sub-feature vector corresponding to the alarm description and a third sub-feature vector corresponding to the payload; convert the first sub-feature vector, the second sub-feature vector and the third sub-feature vector into sub-feature vectors of preset dimensions respectively; and concatenate the converted first sub-feature vector, the converted second sub-feature vector and the converted third sub-feature vector to obtain the first feature vector.

9. An electronic device, characterized in that: The electronic device comprises at least a processor and a memory, and the processor is used to implement the steps of the alarm data processing method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the steps of the alarm data processing method as described in any one of claims 1 to 6.

Images