Transform-based network security alarm noise reduction method

Abstract

The embodiment of the invention provides a Transform-based network security alarm noise reduction method, and the method comprises the steps: obtaining alarm data which is generated by a security device in real time; the alarm data are input into a target real alarm recognition model, whether the alarm data belong to real alarms or not is determined through the target real alarm recognition model, and the target real alarm recognition model comprises an input layer, a feature quantification layer, a target model processing layer and an output layer; the feature quantization layer is at least configured to adopt a Transform encoder to encode the dependency relationship and the interaction information obtained from the HTTP request and the response content. According to the characterization processing provided by the embodiment of the invention, the understanding and expression capability of the model on the alarm data is greatly improved, so that the model can more accurately distinguish the real alarm from the false alarm when classifying the alarm, and the false alarm rate is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network security. Specifically, the embodiments of the present application relate to a network security alarm noise reduction method based on Transformer. Background Art

[0002] Most current network security devices generate network security alerts based on preset rules, but this approach often generates a large number of false alarms. When the number of alerts accumulates to a certain level, it is difficult for security personnel to quickly and accurately identify real attack alerts from the massive amount of alerts. This situation is called "alert fatigue." Alert fatigue can cause security personnel to lose their attention, greatly increasing the risk of missing key threats, thereby reducing the overall security protection efficiency.

[0003] In order to protect the security of internal networks, major enterprises have deployed various types of security devices, which are designed to provide comprehensive protection for internal networks. Security devices monitor network traffic in real time, identify and warn potential attacks, and security operators rely on these warnings to respond to and handle security threats in a timely manner to ensure the security and stability of the network environment.

[0004] However, existing security devices mainly rely on rule matching models to screen and identify attack traffic. Although this method can identify attack behaviors to a certain extent, it often leads to a large number of false positives due to the limitations and fixedness of the rules themselves. Most of these false positives come from normal business requests being mistakenly matched as attack behaviors. With the complexity of the network environment and the diversification of business needs, the boundary between normal traffic and malicious traffic has become increasingly blurred, which further exacerbates the problem of false positives.

[0005] In real operational scenarios, security operators need to face thousands of alarm messages every day. These alarms include both real attack behaviors and a large number of false alarms. The high frequency of false alarms not only consumes a lot of manpower and time, but also seriously distracts the attention of operators and increases the risk of missing important attack events. The existence of false alarms makes it impossible for operators to efficiently identify and deal with real security threats, which in turn poses a challenge to the overall security protection capabilities of the enterprise. Therefore, how to effectively reduce false alarms and improve the accuracy of alarms has become an urgent problem for enterprises in network security operations. Summary of the invention

[0006] The purpose of the embodiment of the present application is to provide a Transformer-based network security alarm denoising method. The embodiment of the present application uses the Transformer encoder to process the most complex and critical parts of the alarm data (such as HTTP request and response content), and can capture long-distance dependencies and complex interactive information in the data on a global scale. This characterization process greatly improves the model's ability to understand and represent alarm data, allowing the model to more accurately distinguish between real alarms and false alarms when classifying alarms, reducing the false alarm rate.

[0007] In a first aspect, an embodiment of the present application provides a Transformer-based network security alarm noise reduction method, the method comprising: obtaining alarm data, wherein the alarm data is generated in real time by a security device; inputting the alarm data into a target real alarm recognition model, and confirming whether the alarm data is a real alarm through the target real alarm recognition model, wherein the target real alarm recognition model comprises an input layer, a feature quantization layer, a target model processing layer and an output layer, and the feature quantization layer is at least configured to encode dependencies and interaction information obtained from HTTP request and response content.

[0008] Some embodiments of the present application improve the accuracy of the authenticity assessment results of the alarm information by introducing a Transformer encoder to encode the relevant requests and responses in the alarm information.

[0009] In some embodiments of the present application, confirming whether the alarm data is a real alarm through the target real alarm recognition model includes: preprocessing the alarm data through the input layer; quantizing the preprocessed data through the feature quantization layer to obtain a feature vector; calculating a authenticity estimation value for the alarm data through the model processing layer and the feature vector; and outputting the authenticity estimation value through the output layer.

[0010] Some embodiments of the present application process the alarm information through various layers and obtain an evaluation result on the authenticity of the alarm information.

[0011] In some embodiments of the present application, the alarm data carries an HTTP request, an HTTP response, or an HTTP URL, wherein the confirming whether the alarm data is a real alarm through the target real alarm recognition model includes: initially encoding the HTTP request, HTTP response, or HTTP URL carried by the alarm data through a Word2Vec model to obtain a word sequence; generating unique corresponding position coding information for each word in the word sequence; adding the position coding information to the word sequence to obtain a target sequence; dividing the target sequence into block vectors of the same length; and inputting the block vectors into the Transformer encoder for encoding.

[0012] Some embodiments of the present application encode and extract features of HTTP requests, HTTP responses, or HTTP URLs through the Word2Vec model and Transformer encoder, thereby improving the accuracy of feature extraction of such information.

[0013] In some embodiments, the input layer includes: a first type of information input module, configured to input the first type of data carried by the alarm data into a corresponding encoding module; a second type of information input module, configured to input the second type of data carried by the alarm data into a corresponding encoding module; a third type of information input module, configured to input the third type of data carried by the alarm data into a corresponding encoding module; a fourth type of information input module, configured to input the fourth type of data carried by the alarm data into a corresponding encoding module; the feature quantization layer includes: a first encoding module, configured to be connected to the output end of the first type of information input module, and The Word2Vec model is used to encode the first type of data to obtain a first feature code; the second encoding module is configured to be connected to the output end of the second type of information input module, and the one-hot encoding model is used to encode the second type of data to obtain a second feature code; the third encoding module is configured to be connected to the output end of the third type of information input module, and the label embedding model is used to encode the third type of data to obtain a third feature code; the fourth encoding module is configured to be connected to the output end of the fourth type of information input module, and the statistical features of the fourth type of information are extracted to encode the fourth type of data. The target model processing layer comprises: a first processing unit, which is configured to process the input first concatenated vector in sequence through a first fully connected layer, a first normalization processing layer, an activation function, and an anti-overfitting algorithm to obtain a first vector to be concatenated, wherein the first concatenated vector is obtained by concatenating the first feature code, the third feature code, and the fourth feature code. The fifth feature code is obtained by a first processing unit, a second processing unit, and a second processing unit. The first processing unit is configured to process the fifth feature code through a second fully connected layer, a second normalization processing layer, an activation function, and an anti-overfitting algorithm in sequence to obtain a second vector to be spliced; the third processing unit is configured to process the input target splicing vector through a third fully connected layer, a third normalization processing layer, an activation function, and an anti-overfitting algorithm in sequence to obtain the alarm authenticity evaluation result, wherein the target splicing vector is obtained by splicing the first vector to be spliced ​​and the second vector to be spliced.

[0014] Some embodiments of the present application classify, quantize, encode and extract features of various types of data carried in alarm information, which significantly improves the adequacy and accuracy of feature extraction for alarm information, thereby improving the accuracy of the authenticity assessment results of the acquired alarm information.

[0015] In some embodiments, before inputting the alarm data into the real alarm recognition model, the method also includes: training the model processing layer to obtain the target model processing layer; wherein, the training of the model processing layer to obtain the target model processing layer includes: collecting historical alarm data, wherein the historical alarm data includes attack names, HTTP requests and response contents; marking the historical alarm data into two types, false alarms and non-false alarms, as label data; using the feature quantization layer to quantize the historical alarm data to obtain a feature vector; using the model processing layer and the feature vector to obtain an authenticity evaluation result for the historical alarm data; using a loss function to determine the difference between the authenticity evaluation result and the corresponding label data to obtain a loss value, and determining whether to continue training based on the loss value.

[0016] Some embodiments of the present application train the model through historical alarm data.

[0017] In some embodiments, the historical alarm data includes a first category of data, a second category of data, a third category of data and a fourth category of data; wherein, the use of the feature quantization layer to quantize the historical alarm data to obtain a feature vector includes: converting the text information included in the first category of data into a numerical feature vector to obtain a first feature vector; converting the second category of data into a sparse vector composed of 0 and 1 to obtain a second feature vector; mapping the third category of data into continuous numerical data to obtain a third feature vector; extracting the statistical features of the characters in the fourth category of data to obtain a fourth feature vector; and performing the following processing on the fourth category of data to obtain a fifth feature vector: encoding using a second Word2Vec model, filling with word position information and encoding with a Transformer encoder; the use of the model processing layer and the feature vector to obtain an authenticity assessment result for the historical alarm data includes: processing the second feature vector one by one through one-dimensional convolution, normalization and ReLU activation function to obtain an initial second feature vector; The first merged vector is processed one by one by a fully connected layer, normalization, RLU activation function and dropout (as an anti-overfitting algorithm) to obtain a first to-be-merged vector; the fifth eigenvector is sequentially subjected to the following operations to obtain a second to-be-merged vector: average pooling, full connection processing, normalization processing, ReLU activation and Dropout operation; the first to-be-merged vector and the second to-be-merged vector are merged, and the merged result is subjected to multiple full connection processing to obtain a one-dimensional vector as the historical alarm data; wherein the first type of data is used to carry threat information, source address or destination address, the second type of data is unordered classified data, and the unordered classified data includes HTTP request method, the number of hours of occurrence time or numerical data after binning processing, the third type of data is ordered classified data, and the ordered classified data includes alarm level or confidence, and the fourth type of data includes multi-character type data, and the multi-character type data includes HTTP request, HTTP response or HTTP URL.

[0018] Some embodiments of the present application improve the accuracy of the evaluation results output by the trained model by fully exploiting the information carried by historical alarm information.

[0019] In some embodiments, the extracting of statistical features of characters in the fourth category of data includes: counting the character length of HTTP requests and HTTP responses, counting the length of invisible characters, counting the number of occurrences of characters in HTTP request headers and HTTP response headers (for example, counting the number of occurrences of “,;:. / =”), and counting the number of occurrences of characters in HTTP request bodies and HTTP response bodies (for example, counting the number of occurrences of “=. / &”).

[0020] In a second aspect, some embodiments of the present application provide a Transformer-based network security alarm noise reduction device, the device comprising: an alarm information acquisition module, configured to obtain alarm data, wherein the alarm data is generated in real time by a security device; an alarm result authenticity prediction module, configured to input the alarm data into a target real alarm recognition model, and confirm whether the alarm data is a real alarm through the target real alarm recognition model, wherein the target real alarm recognition model comprises an input layer, a feature quantization layer, a target model processing layer and an output layer, and the feature quantization layer is at least configured to use a Transformer encoder to encode dependencies and interaction information obtained from HTTP request and response content.

[0021] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method for detecting false alarm information as described in any one of the embodiments of the first aspect.

[0022] In a fourth aspect, some embodiments of the present application provide a computer program product, comprising computer program instructions, which, when read and executed by a processor, execute the method for detecting false alarm information as described in any one of the embodiments of the first aspect.

[0023] In a fifth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the method for detecting false alarm information as described in any one of the embodiments included in the first aspect can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1One of the flow charts of the method for detecting false alarm information provided in an embodiment of the present application;

[0026] Figure 2 A second flowchart of a method for detecting false alarm information provided in an embodiment of the present application;

[0027] Figure 3 An architecture diagram of a target real alarm identification model or a real alarm identification model provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of the structure of a Transformer encoder provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the training process provided in an embodiment of the present application;

[0030] Figure 6 A block diagram of the composition of the device for detecting false alarm information provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0034] At present, the industry generally adopts a rule-based approach to detect false alarms. This approach relies on security experts to manually analyze the characteristics of false alarms and extract characteristic rules. Subsequently, when the security device generates an alarm, if the alarm matches these rules, it will be marked as a false alarm, thereby preventing it from being displayed on the user interface, thereby reducing the number of false alarms.

[0035] The rule-based noise reduction method provided by the relevant technology has the following disadvantages: 1) Strong rule dependence and unstable results: The effect of false alarm detection depends entirely on the quality of the rules. These rules are usually written manually, which means that the detection effect depends to a large extent on the security capabilities and experience of the rule writer. If the rules are not written properly, real security alerts may be misjudged as false alarms, resulting in missed alarms and endangering network security. 2) Rule lag and inability to cope with dynamic changes: The formulation of rules is often carried out after false alarms have been generated, which means that false alarm problems can only be remedied afterwards. In the actual operating environment, false alarms are usually caused by the behavior of normal business systems. As business systems are constantly updated and changed, the characteristics of false alarms will also change accordingly, resulting in the inability of old rules to effectively cope with new false alarm situations. 3) High maintenance cost and high resource consumption: The rule-based false alarm detection method needs to continuously update the rules to adapt to the ever-changing network environment and business needs. This continuous rule update not only requires a lot of manpower and time, but also increases operating costs and reduces the overall efficiency of the system.

[0036] The reduction of false alarms provided by the relevant technology can be achieved in two ways: one is to improve the detection capability of security equipment and reduce the number of false alarms from the source; the other is to add a filtering mechanism outside the security equipment to perform secondary screening on the generated alarms and filter out false alarms. Due to the wide variety of security equipment within the enterprise, most equipment can only improve the quality of alarms by optimizing alarm rules, and cannot effectively reduce the generation of false alarms. Therefore, the embodiment of the present application chooses the second method: that is, adding a filtering layer outside the existing security equipment. When the security device generates an alarm, the method proposed in the embodiment of the present application is used to detect false alarms for the alarm. Only after it is confirmed to be a real alarm will it be displayed to the security operation personnel. This method not only improves the accuracy of the alarm, but also reduces the burden on the security team, allowing them to focus more on dealing with real security threats.

[0037] The inventors of the present application have found in their research that in the scenario of network security alarm noise reduction, the alarm data is usually extremely complex, especially the alarms involving Web-type attacks. For these alarms, HTTP request and HTTP response data are key factors in determining the authenticity of the alarm. In addition, information such as URL fields, attack source IP and destination IP also plays an important role in alarm judgment. However, although HTTP request and response data exist in plain text format, due to their lack of language structure, traditional encoding methods are difficult to directly convert them into valid feature vectors.

[0038] In an embodiment of the present application, the encoder part in the Transformer model is used to extract features for HTTP requests and HTTP responses in the alarm data. In this way, the embodiment of the present application can convert complex text data into high-dimensional feature vectors, and combine them with other key alarm information such as URL, source IP and destination IP. Ultimately, these combined feature vectors will be used as input to the model processing layer (or target model processing layer) to drive the model training of the machine learning algorithm or to perform alarm authenticity assessment, thereby improving the accuracy and effectiveness of alarm noise reduction.

[0039] The Transformer model is a deep learning model used in natural language processing (NLP) and other sequence data processing fields. It consists of two main parts: the encoder and the decoder. The Transformer encoder can convert the input sequence into a continuous representation and extract high-quality feature vectors.

[0040] Therefore, some embodiments of the present application aim to solve the problem of how to effectively reduce false alarms in alarms and improve the quality and accuracy of alarms through alarm noise reduction technology. The strength of this application is to design a deep learning model by combining Transformer technology. The model uses multiple methods to process features and can intelligently identify and filter false alarms. In this way, security operators can focus on real security threats, thereby greatly alleviating alarm fatigue and improving the overall effectiveness of network security protection.

[0041] Please see Figure 1 , Figure 1 A Transformer-based network security alarm noise reduction method provided in some embodiments of the present application includes:

[0042] S101, obtaining alarm data, wherein the alarm data is generated in real time by a security device.

[0043] S102, inputting the alarm data into a target real alarm identification model, and confirming whether the alarm data is a real alarm through the target real alarm identification model, wherein the target real alarm identification model includes an input layer, a feature quantization layer, a target model processing layer and an output layer, and the feature quantization layer is at least configured to use a Transformer encoder to encode the dependency and interaction information obtained from the HTTP request and response content.

[0044] It is understandable that in order to evaluate the authenticity of the alarm data, some embodiments of the present application need to train the real alarm recognition model to obtain the target real alarm recognition model. It is understandable that the network architecture of these two models is the same. Figure 2 As shown, the method of the embodiment of the present application includes a training phase and an application phase. In the training phase, the following steps need to be performed: collecting historical alarm data; marking alarm data; feature engineering (i.e., using Figure 3 The application stage does not include: obtaining real-time alarm data; feature engineering; alarm denoising model (or processing using the model processing layer); real alarm (i.e. outputting a decision on whether it is a real alarm).

[0045] For example, in some embodiments of the present application, before the alarm data is input into the real alarm recognition model, the method also includes: training the model processing layer to obtain the target model processing layer; wherein, the training of the model processing layer to obtain the target model processing layer includes: collecting historical alarm data, wherein the historical alarm data includes attack names, HTTP requests and response contents; marking the historical alarm data into two types, false alarms and non-false alarms, as label data; using the feature quantization layer to quantize the historical alarm data to obtain a feature vector; using the model processing layer and the feature vector to obtain an authenticity evaluation result for the historical alarm data; using a loss function to determine the difference between the authenticity evaluation result and the corresponding label data to obtain a loss value, and determining whether to continue training based on the loss value.

[0046] That is to say, the exemplary training phase of the embodiment of the present application includes:

[0047] 1) Collection of historical alarm data: This is the basis for model training. In this step, it is necessary to focus on collecting key fields for model training, such as attack name, HTTP request and response content, etc.

[0048] 2) Alarm data labeling: Each alarm needs to be classified and labeled according to whether it is a false alarm. If it is a false alarm, it is marked as "1"; if it is not a false alarm, it is marked as "0". These marks will be used as label data for model training.

[0049] 3) Feature Engineering: Processing is performed according to the data type. For ordinary text data (as an example of the first type of data, such as threat information, source address, and destination address), the Word2Vec model is used for encoding; for unordered categorical data (as an example of the second type of data, such as HTTP request method, the hour of occurrence time, etc.), One-Hot encoding is used; for ordered categorical data (as an example of the third type of data, such as alarm level, confidence), label encoding is used for encoding; for numerical data (such as port information, number of alarms), the data is first converted into categorical data after binning, and then One-Hot encoding is performed; for complex character data (as an example of the fourth type of data, such as HTTP request, HTTP response, and HTTP URL), statistical features such as the number of occurrences of special characters are first extracted in combination with prior knowledge. In addition, Word2Vec is used to encode it, and then the Transformer encoder is used for processing. It should be noted that statistical features and data encoding are used for training at the same time and belong to the same level of data. Statistical features include: counting the character length of HTTP requests and HTTP responses, counting the length of invisible characters, counting the number of occurrences of characters such as ",;:. / =" in HTTP request headers and HTTP response headers, and counting the number of occurrences of characters such as "=. / &" in HTTP request bodies and HTTP response bodies. Data encoding includes: 1. Using word2Vec to encode such data to obtain intermediate data; 2. Using a Transformer encoder to process the intermediate data to obtain the final feature data. 4) Model training: After feature engineering is completed, the alarm data is converted into feature vectors that can be used for model training, and trained through back propagation of the neural network to ultimately build an effective alarm denoising model.

[0050] In the application stage of the embodiment of the present application, the real-time alarm data is first processed by the same feature engineering as the training stage to generate feature vectors. Then, these feature vectors are input into the trained alarm denoising model for processing. After the analysis of the model, the system will be able to judge the authenticity of the alarm, filter out false alarms, and finally retain only the real alarm information.

[0051] The following first illustrates the process of training the real alarm recognition model to obtain the target real alarm recognition model.

[0052] Before introducing the training process, first combine Figure 3 The architecture of the real alarm recognition model is explained by way of example.

[0053] In some embodiments of the present application, the input layer of the real alarm recognition model includes:

[0054] The first type of information input module is configured to input the first type of data carried by the alarm data into the corresponding encoding module. For example, the first type of data is used to carry threat information, source address or destination address, and the corresponding first type of information input module exemplarily includes: Figure 3 As shown: a threat_info module, which is used to input the threat information carried by the first type of data into the corresponding encoding module; a src_address module, which is used to input the source address carried by the first type of data into the corresponding encoding module; and a dst_address module, which is used to input the destination address carried by the first data into the corresponding encoding module.

[0055] The second type of information input module is configured to input the second type of data carried by the alarm data into the corresponding encoding module. For example, the second type of data is unordered classified data, and the unordered classified data includes HTTP request method, the number of hours of occurrence time, or numerical data after binning. The corresponding second type of information input module exemplarily includes the following: Figure 3 As shown: the http_method module is used to input the HTTP request method carried by the second type of data into the corresponding encoding module; the time_h module is used to input the hour at which the HTTP request event carried by the second type of data occurs into the corresponding encoding module; it can be understood that Figure 3 Not all submodules included in the second type of information input module are shown, and those skilled in the art can add corresponding input modules according to actual needs.

[0056] The third type of information input module is configured to input the third type of data carried by the alarm data into the corresponding encoding module. For example, the third type of data is ordered classification data, and the ordered classification data includes the alarm level or confidence level. The corresponding third type of information input module exemplarily includes the following: Figure 3 The alarm_level module shown is used to input the alarm level carried by the third type of data into the corresponding encoding module; and the confidence module is used to input the confidence data carried by the third type of data into the corresponding encoding module.

[0057] The fourth type of information input module is configured to input the fourth type of data carried by the alarm data into a corresponding encoding module. For example, the fourth type of data includes multi-character type data, and the multi-character type data includes HTTP request, HTTP response or HTTP URL, and the corresponding fourth type of information input module exemplarily includes the following Figure 3Shown are: an http_url_path module, used to input information related to the HTTP URL carried by the fourth category of data into the corresponding encoding module, an http_req module, used to input data related to the HTTP request carried by the fourth category of data into the corresponding encoding module, and an http_res module, used to input data related to the HTTP response carried by the fourth category of data into the corresponding encoding module.

[0058] The feature quantization layer includes:

[0059] The first encoding module (such as Figure 3 The Word2Vec module shown in the figure) is configured to be connected to the output end of the first type of information input module, and uses the Word2Vec model to encode the first type of data to obtain a first feature code.

[0060] The second encoding module (such as Figure 3 The One Hot module shown in the figure is configured to be connected to the output end of the second type of information input module, and uses the one hot encoding model to encode the second type of data to obtain a second feature code.

[0061] The third encoding module (such as Figure 3 The Label Encoding module shown in the figure is configured to be connected to the output end of the third type of information input module, and uses the label embedding model to encode the third type of data to obtain a third feature code.

[0062] The fourth encoding module (such as Figure 3 The statistical feature extraction module shown in the figure is configured to be connected to the output end of the fourth type of information input module, and to encode the fourth type of data by extracting the statistical features of the fourth type of information to obtain a fourth feature code;

[0063] The fifth encoding module (such as Figure 3 The modules Word2Vec module and Transformer Encode) shown are configured to be connected to the output end of the fourth type of information input module, and encode the fourth type of data at least through the Word2Vec model and the Transformer encoder to obtain the fifth feature code.

[0064] The target model processing layer includes:

[0065] The first processing unit (such as Figure 3 The model processing layer shown includes an FC&Norm&Relu&Dropout), which is configured to sequentially pass through the first fully connected layer, the first normalization processing layer, the activation function, and the anti-overfitting algorithm to the first concatenated vector (such as Figure 3X1, X2, X3...X P ) is processed to obtain the first vector to be spliced ​​(such as Figure 3 Shown b l,1 ,b l,2 ,b l,3 ……b l,P ), wherein the first splicing vector is obtained by splicing the first feature code, the third feature code, the fourth feature code and the preprocessed second feature code, and the preprocessing includes one-dimensional convolution and normalization.

[0066] The second processing unit (such as Figure 3 The model processing layer shown includes an FC&Norm&Relu&Dropout), which is configured to sequentially process the fifth feature code (such as Figure 3 X1, X2, X3...X m ) to obtain the second vector to be spliced ​​(such as Figure 3 Shown b 2,1 ,b 2,2 ,b 2,3 ……b 2,t ).

[0067] The third processing unit (such as Figure 3 The model processing layer shown includes the last FC&Norm&Relu&Dropout), which is configured to sequentially pass the third fully connected layer, the third normalization processing layer, the activation function, and the anti-overfitting algorithm to the input target concatenation vector (such as Figure 3 Shown b l,1 ,b l,2 ,b l,3 ……b 2,t ) to obtain the alarm authenticity assessment result (such as Figure 3 Shown b 4,1 ), wherein the target splicing vector is obtained by splicing the first vector to be spliced ​​and the second vector to be spliced.

[0068] Figure 3 The example also includes the Sigmod module and the output of the Real Alem real alarm and the False Alem false alarm exemplary discrimination results, that is, Figure 3 The output layer is used to output whether the alarm data is a real alarm or a false alarm.

[0069] In some embodiments of the present application, in the input layer of the model, the alarm data is first preprocessed and divided into four categories according to data characteristics: the first category data and ordinary text data (such as threat information, source address and destination address), the second category data, namely unordered categorical data (such as HTTP request method, hour of occurrence time, numerical data after binning, etc.), the third category data, namely ordered categorical data (such as alarm level, confidence level) and the fourth category data, namely complex character data (such as HTTP request, HTTP response and HTTP URL).

[0070] In the feature vectorization layer, each type of data in the embodiment of the present application is processed using a corresponding encoding method. The unordered categorical data is encoded by One-Hot, and the categorical data is converted into a sparse vector composed of 0 and 1, so as to be input into the neural network for learning; the ordinary text data is encoded by the Word2Vec model, and the text is converted into a numerical feature vector. The model captures the contextual relationship between words and learns the embedding vector of each word, thereby expressing the semantic information of the vocabulary; for ordered categorical data, label encoding is used to map it to continuous numerical data; and for complex character data, in view of its wide distribution of characters, the statistical features of the characters are first extracted (the character length of HTTP requests and responses, the length of non-visible characters, the number of occurrences of characters such as ,;:. / = in HTTP request headers and HTTP response headers, and the number of occurrences of =. / & in HTTP request bodies and HTTP response bodies), and in addition, it is encoded using Word2Vec and input into the Transformer encoder to generate a high-dimensional feature vector and deeply capture the complex features and long-distance dependencies in the data.

[0071] In some embodiments of the present application, visible characters are defined as: uppercase and lowercase letters, English punctuation marks, numbers, and spaces, that is, the ASCII character range 32 to 126. Invisible characters are characters that are not within the visible character range. The method for determining its length is: traverse all single characters in the character string to be counted, and count the number of invisible characters.

[0072] Combination Figure 3 In the embodiment of the present application, the model processing layer is used to perform the following operations on the input feature vector:

[0073] 1) Use a one-dimensional convolutional network (1D Convolution) to process One-Hot encoded data. One-Hot encoded data is usually high-dimensional and sparse, and one-dimensional convolution can help extract important features and reduce the dimension of the data. After processing, it is normalized and the ReLU activation function is used to enhance nonlinear features.

[0074] 2) The Word2Vec encoded data, the label encoded data, and the extracted statistical features are normalized respectively, and then connected and merged with the output vector of step 1) to form a new feature vector.

[0075] 3) The feature vector merged in step 2) is further processed through a fully connected layer to enable it to learn more complex relationships. It is then normalized and ReLU activated. In order to prevent the model from overfitting and improve the generalization ability of the model, dropout is added for processing.

[0076] 4) The HTTP request and response data encoded and filled by Word2Vec are passed into the Transformer encoder to obtain a new feature vector, and then the generalization ability of the model is improved through mean pooling.

[0077] 5) The feature vector obtained in step 4) is fully connected, normalized, ReLU activated and operated.

[0078] 6) Merge the output vectors of step 3) and step 5) and perform N times of full connection processing (including normalization, ReLU activation, Dropout and other operations).

[0079] 7) Perform the final full connection processing on the output vector of step 6) and output a vector with a dimension of 1. The result is converted to the interval of 0-1 in the Sigmod function.

[0080] At the output layer, according to the output of the Sigmod function, if the result is greater than 0.5, the alarm is considered to be a false alarm, and if the result is less than 0.5, the alarm is considered to be a true alarm.

[0081] Combine the following Figure 4 The Transformer encoder structure used is exemplarily described, and the encoder includes: an input module Input Vec, an embedding module Input Embeding, a positional encoding module Positonal Encoding, a segment embedding module Segment Embeding and an Nx module. The Nx module exemplarily includes: a multi-head attention module Muti-headAttention, a normalization module Add&Norm, a feedforward module Feed Forward, another normalization module Add&Norm and an output module Outputs Vec.

[0082] The input of the entire encoder of the embodiment of the present application is a series of word vectors, which are first mapped to feature vectors in a high-dimensional space through an input embedding module. Since the Transformer model itself does not have the ability to process the order of sequence data, it is necessary to introduce positional encoding to provide information about the position of words in the sequence.

[0083] In some embodiments of the present application, the position coding is calculated using sine and cosine functions to generate a unique code for each position. This coding method introduces different frequency combinations of sine and cosine, so that the model can obtain effective position information according to the position of the input sequence. Specifically, the formula for position coding is as follows:

[0084]

[0085] Among them, pos represents the position in the sequence, i represents the dimension index of the embedding vector, and d model is the total dimension of the embedding vector. Through this encoding method, the model can incorporate the order information in the sequence into each feature vector, thereby better understanding and handling temporal dependencies.

[0086] After adding the position encoding, the Transformer encoder of some embodiments of the present application further processes the feature vector through the Segment Embedding module, dividing the input vector into block vectors of the same length to ensure the consistency of batch processing. Then, the encoder processes it through N layers of multi-head attention mechanism, residual connection, normalization, and feed forward neural network.

[0087] Transformer's multi-head attention mechanism can simultaneously focus on multiple different positions in the input sequence, effectively capturing the long-distance dependencies between elements in the sequence. This mechanism performs a global analysis of the input data by computing multiple attention heads in parallel, thereby generating richer contextual information. Combined with residual connections and normalization operations, it ensures stable gradient propagation and prevents model degradation or overfitting during deep learning.

[0088] After multiple layers of processing, the feature vectors are further integrated to form the final output of the model. Through this structure, the Transformer encoder can provide deep feature representation for complex data patterns, significantly enhancing the model's ability to understand and process diverse input data.

[0089] It is not difficult to understand that S102 confirms whether the alarm data is a real alarm through the target real alarm recognition model, including: preprocessing the alarm data through the input layer; quantizing the preprocessed data through the feature quantization layer to obtain a feature vector; calculating the authenticity estimation value for the alarm data through the model processing layer and the feature vector; and outputting the authenticity estimation value through the output layer.

[0090] In some embodiments of the present application, the alarm data carries an HTTP request, an HTTP response or an HTTP URL; wherein, S102 confirms whether the alarm data is a real alarm through the target real alarm recognition model, including: performing initial encoding on the HTTP request, HTTP response or HTTP URL carried by the alarm data through the Word2Vec model to obtain a word sequence; generating unique corresponding position coding information for each word in the word sequence; adding the position coding information to the word sequence to obtain a target sequence; dividing the target sequence into block vectors of the same length; and inputting the block vectors into the Transformer encoder for encoding.

[0091] In some embodiments of the present application, the alarm data includes a first category of data, a second category of data, a third category of data, and a fourth category of data (for the specific information carried by each of these four categories of data, please refer to the above description); wherein, the use of the feature quantization layer to quantize the historical alarm data to obtain a feature vector includes: converting the text information included in the first category of data into a numerical feature vector to obtain a first feature vector; converting the second category of data into a sparse vector composed of 0 and 1 to obtain a second feature vector; mapping the third category of data to continuous numerical data to obtain a third feature vector; extracting the statistical features of the characters in the fourth category of data to obtain a fourth feature vector; and performing the following processing on the fourth category of data to obtain a fifth feature vector: encoding using the second Word2Vec model, filling with word position information and encoding with a Transformer encoder; the use of the model to process The method comprises the following steps: the first eigenvector is processed one by one through a one-dimensional convolution, a normalization process and a ReLU activation function to obtain an initial second eigenvector; the first eigenvector, the third eigenvector and the fourth eigenvector are normalized and the data is merged with the initial second eigenvector to obtain a first merged vector; the first merged vector is processed one by one through a fully connected layer, normalization, an RLU activation function and a dropout process to obtain a first vector to be merged; the fifth eigenvector is sequentially subjected to the following operations to obtain a second vector to be merged: average pooling, full connection process, normalization process, ReLU activation and Dropout operation; the first vector to be merged and the second vector to be merged are merged, and the merged result is subjected to multiple full connection processes to obtain a one-dimensional vector as the authenticity evaluation result of the training data alarm.

[0092] In one implementation, in order to improve the computational efficiency of the model, some embodiments of the present application use a computer equipped with at least one GPU to accelerate the training and reasoning process of the model. The parallel computing capability of the GPU significantly shortens the training time of the deep learning model and ensures rapid iteration and optimization on large-scale data sets.

[0093] The training process of the model in some embodiments of the present application includes:

[0094] First, the preprocessed alarm data is input into the model for feature vectorization, which converts the raw data into a numerical representation suitable for model learning. Next, the feature vector is passed into the model for forward propagation calculation, which is processed layer by layer through the model's multi-layer network structure to generate a predicted output result. This process is called forward propagation, and its main purpose is to calculate the output of the model under the current parameter settings.

[0095] After the prediction result is obtained through forward propagation, it is compared with the real label in the alarm data, and the difference between the prediction result and the real label is calculated using Binary Cross-Entropy (BCE) as the loss function. The smaller the value of the loss function, the closer the model's prediction is to the real situation.

[0096] Next, based on the value of the loss function, the model parameters are updated. Specifically, the Adam Optimizer is used to adjust the values ​​of the model parameters by calculating the gradient of the loss function with respect to each parameter. This process is called backpropagation, and its purpose is to improve the accuracy of the model by continuously adjusting the parameters to minimize the loss function.

[0097] After completing one forward propagation and one backward propagation, verify the effect of the model. If the value of the loss function meets the pre-set stop condition or reaches the expected model performance standard, the model training ends and the final model parameters are saved; if it does not meet the requirements, the updated parameters are used to perform forward propagation calculations again and enter the next round of training. This cycle will be repeated until the model effect reaches the expected level.

[0098] Through the above training process, the model can continuously optimize its own parameters, thereby improving the classification and recognition capabilities of alarm data, and ultimately achieving effective distinction between real alarms and false alarms.

[0099] According to the training process described in the above example, the training process of some embodiments of the present application includes: first, historical alarm data needs to be collected. In order to ensure the accuracy and effectiveness of the data, each piece of collected alarm data needs to be manually marked. Specifically, for alarms triggered by real attack behaviors, they are marked as "0"; for false alarms caused by normal business activities, they are marked as "1". In this process, manual labeling is a key step, and a variety of methods can be used to improve efficiency, such as through rule matching (such as using pre-defined alarm rules) or batch search (such as automated scripts to quickly filter out alarms with similar features) and other means to ensure that data labeling is completed quickly and accurately.

[0100]

[0101] Table 1 Alarm data example

[0102] After labeling, the original data set needs to be further processed for model training. Usually, the data set is divided into a training set and a validation set in a certain ratio (for example, 3:1). The training set is used to learn the model, and the validation set is used to evaluate the performance of the model during the training process. If the amount of data is sufficient, an independent test set can be additionally divided to comprehensively evaluate and verify the final effect of the model after training is completed, so as to avoid overfitting and improve the generalization ability of the model. In this implementation, the sample of the alarm data of the data set is shown in Table 1, which contains various types of alarm information and their corresponding labels.

[0103] After completing the data set segmentation, it is necessary to reasonably set the various hyperparameters of the model to ensure the effectiveness and stability of the training process. For the model proposed in the present invention, the hyperparameters involved include the configuration of the Transformer encoder, Word2Vec model parameters, neural network structure, and other related training parameters. In this example, the specific hyperparameter settings are as follows:

[0104] 1) Transformer encoder hyperparameters:

[0105] The embedding dimension of the Transformer encoder is set to 256, and two attention heads are used. The overall network depth is 2 layers. In order to prevent the sequence input to the Transformer encoder from being too long at one time, resulting in high computational complexity or memory overflow, a segmented processing method is used to divide the input sequence into multiple segments of 512 bytes in length. Each segment retains the context information of the first 240 bytes to maintain the continuity and semantic integrity of the sequence. This segmented processing method can improve computational efficiency while ensuring model performance.

[0106] 2) Hyperparameters of the Word2Vec model:

[0107] For different feature fields, the Word2Vec model uses different word vector dimensions: the word vector dimension of the threat information field is 16, the dimension of the source address and destination address fields is 8, the word vector dimension of the HTTP request field is 256, and the word vector dimension of the HTTP response field is 256.

[0108] 3) Hyperparameters of the neural network structure:

[0109] When processing different types of encoded data, the structure of the neural network is also designed accordingly. For the processing of One-Hot encoded features, a 64-dimensional fully connected neural network is used; the feature vector encoding of HTTP request and response information is 256-dimensional, and the encoding of other features is also set to 256-dimensional. After all feature vectors are merged, they are input into a deeper neural network for further feature extraction and learning. The network structure is a 128-dimensional and 256-dimensional fully connected layer to ensure that the model has enough capacity to capture complex feature patterns.

[0110] 4) Other hyperparameter settings:

[0111] The initial value of the learning rate is set to 0.008. In order to make the model converge better, a step-wise learning rate adjuster (StepLR) is used. After each training iteration, the learning rate is adjusted to 80% of the original value. This dynamic adjustment strategy helps to maintain a high learning rate in the early stage of training to accelerate model convergence; and gradually reduce the learning rate in the later stage to avoid oscillation of the model when approaching the global optimal solution.

[0112] After completing the setting of the hyperparameters, the model training process begins. In some examples of the present application, the verification criterion for the model effect is based on the binary cross-entropy loss value on the verification set. When the loss value no longer continues to decrease with the increase in the number of training times, it means that the model has reached a convergence state and the training process can end.

[0113] In order to comprehensively evaluate the performance of the model, in some examples of this application, 30% of the total data set is divided into a test set for independent performance testing. The data in the test set does not participate in the learning of the model during the training process to ensure that it can reflect the performance of the model in real scenarios. After the model training is completed, the test set will go through the same preprocessing steps as the training data and then be input into the model for prediction evaluation.

[0114] The test results show that the overall accuracy of the model on the test set is 0.903, indicating that the overall accuracy of the model in identifying all alarm samples (including real alarms and false alarms) is 90.3%. In addition, in terms of detecting false alarms, the accuracy of the model reached 0.863, which means that 86.3% of the samples judged as false alarms by the model are correct. At the same time, the recall rate of the model for false alarm samples is 0.808, indicating that the model can identify 80.8% of false alarms. In terms of the comprehensive evaluation index F0.5 value of false alarm detection, the model achieved a score of 0.862, which shows that the model pays attention to precision while taking into account a certain recall rate.

[0115] These test results indicate that the model provided by the embodiments of the present application has good performance in alarm noise reduction, especially in the detection and processing of false alarm samples, showing high accuracy and reasonable recall rate, effectively reducing the interference of false alarms on safety operations.

[0116] It is not difficult to understand that the technical advantages brought by the above technical solutions provided by the embodiments of the present application include:

[0117] 1) It can significantly improve the accuracy and precision of alarm classification. The embodiments of the present application can capture long-distance dependencies and complex interactive information in the data on a global scale by using the Transformer encoder to process the most complex and critical parts of the alarm data (such as HTTP request and response content). This characterization process greatly improves the model's ability to understand and represent alarm data, allowing the model to more accurately distinguish between real alarms and false alarms when classifying alarms, reducing the false alarm rate.

[0118] 2) The method proposed in the embodiment of the present application can improve the efficiency of alarm denoising. The overall structural design of the alarm denoising model proposed in the embodiment of the present application is reasonable. Through the combination of multiple deep learning technologies such as feature vectorization, fully connected neural network processing, pooling layer, normalization, activation function and Dropout, a hierarchical and multi-modular processing flow is formed. This structural design enables the model to efficiently process large-scale, high-dimensional alarm data, effectively reduce false alarms, and maintain good generalization capabilities, and can cope with diverse attack types and ever-changing network environments.

[0119] 3) The use of the embodiments of the present application can reduce the workload of security operators. By significantly reducing the number of false alarms, the embodiments of the present application greatly reduce the workload of security operators. Operators no longer need to waste a lot of time and energy on screening and handling false alarms, and can focus more on analyzing and responding to real security threats, thereby improving the efficiency and response speed of overall network security operations.

[0120] 4) The model of the embodiment of the present application has the ability of continuous learning and self-optimization. The model of the embodiment of the present application has the ability of self-learning. By continuously inputting new alarm data for training and optimization, the model can gradually improve its detection ability for new and unknown attacks. This continuous learning and self-optimization feature enables the model to maintain efficient detection and response capabilities when facing the ever-changing network environment and attack methods.

[0121] In summary, the improvements of the embodiments of the present application include: first, the Transformer encoder is used to characterize the complex alarm data. One of the core innovations of the present invention is to use the Transformer encoder to characterize the most critical and complex parts of the alarm data, such as HTTP request content and HTTP response content. HTTP request and response data are usually highly unstructured, and their content often contains a lot of noise and redundant information, which makes it difficult for traditional feature extraction methods to effectively capture the deep patterns therein. By introducing the Transformer encoder, the present invention can make full use of its powerful self-attention mechanism to capture long-distance dependencies and complex interactive information in the data globally, thereby generating high-quality feature vectors. This method significantly improves the model's understanding and processing capabilities of alarm data, and effectively improves the accuracy and robustness of alarm classification. Secondly, the overall structural design of the alarm denoising model. Another innovation of the present invention lies in the overall structural design of the alarm denoising model, especially the structure of the processing layer in the model. This model uses a multi-layer deep learning network to process multiple features in the alarm data in a hierarchical manner. The model first performs corresponding feature encoding on different types of data (such as classified data, text data, ordered data, etc.), and then processes and fuses these features layer by layer through a multi-layer fully connected neural network. The internal structure of the model also includes normalization, activation function, pooling layer and Dropout and other deep learning technologies to ensure that when processing large-scale, high-dimensional alarm data, overfitting can be effectively prevented and the generalization ability of the model can be maintained. In addition, the present invention pays special attention to modularization and scalability in structural design, so that the model can flexibly adjust the configuration of each processing module according to different application scenarios and alarm data types to achieve the best noise reduction effect.

[0122] like Figure 5 As shown, the training method provided by some embodiments of the present application includes: inputting alarm data; feature vectorization; model calculation forward propagation; calculating loss (BCE Second Hospital cross entropy); directional propagation updating parameters (Adam optimizer); verifying the model effect, if it is confirmed that it does not meet the standard, returning to the model calculation forward propagation, if it is confirmed that it meets the standard, ending the training to obtain the target real alarm recognition model. The specific process can be referred to the description above, and no more details will be given here to avoid repetition.

[0123] Please refer to Figure 6 , Figure 6 The device for network security alarm noise reduction based on Transformer provided in the embodiment of the present application is shown. It should be understood that the device is similar to the above-mentioned Figure 1Corresponding to the method embodiment, each step involved in the above method embodiment can be executed. The specific functions of the device can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in the memory in the form of software or firmware or solidified in the operating system of the device. The Transformer-based network security alarm denoising device includes: an alarm information acquisition module 610 and an alarm result authenticity prediction module 620.

[0124] The alarm information acquisition module 610 is configured to acquire alarm data, wherein the alarm data is generated in real time by the security device.

[0125] The alarm result authenticity prediction module 620 is configured to input the alarm data into a target true alarm recognition model, and confirm whether the alarm data is a true alarm through the target true alarm recognition model, wherein the target true alarm recognition model includes an input layer, a feature quantization layer, a target model processing layer and an output layer, and the feature quantization layer is at least configured to use a Transformer encoder to encode the dependency and interaction information obtained from the HTTP request and response content.

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0127] Some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for detecting false alarm information as described in the above embodiments can be implemented.

[0128] Some embodiments of the present application provide a computer program product, including computer program instructions, which, when read and executed by a processor, execute the method for detecting false alarm information as described in the above embodiments.

[0129] like Figure 7 As shown, some embodiments of the present application provide an electronic device 700, including a memory 710, a processor 720, and a computer program stored in the memory 710 and executable on the processor 720, wherein the processor 720 can implement the method for detecting false alarm information as described in any of the above embodiments when reading the program from the memory 710 through a bus 730 and executing the program.

[0130] Processor 720 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 720 can be a microprocessor.

[0131] The memory 710 may be used to store instructions executed by the processor 720 or data related to the execution of instructions. These instructions and / or data may include code to implement some or all functions of one or more modules described in the embodiments of the present application. The processor 720 of the present embodiment may be used to execute the instructions in the memory 710 to implement Figure 1 The memory 710 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory or other memory known to those skilled in the art.

[0132] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0134] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0135] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0137] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A network security alarm noise reduction method based on Transformer, characterized in that: The method comprises: Acquiring alarm data, wherein the alarm data is generated in real time by a security device; The alarm data is input into a target real alarm identification model, and the target real alarm identification model is used to confirm whether the alarm data is a real alarm, wherein the target real alarm identification model includes an input layer, a feature quantization layer, a target model processing layer and an output layer, and the feature quantization layer is at least configured to use a Transformer encoder to encode the dependency and interaction information obtained from the HTTP request and response content.

2. The method according to claim 1, characterized in that The confirming whether the alarm data is a real alarm by using the target real alarm identification model includes: Preprocessing the warning data through the input layer; The preprocessed data is quantized by the feature quantization layer to obtain a feature vector; Calculate a truthfulness estimation value for the warning data through the model processing layer and the feature vector; The authenticity estimate is outputted through the output layer.

3. The method according to claim 2, characterized in that The alarm data carries an HTTP request, an HTTP response or an HTTP URL; wherein, The confirming whether the alarm data is a real alarm by using the target real alarm identification model includes: Initially encoding the HTTP request, HTTP response or HTTP URL carried by the warning data through the Word2Vec model to obtain a word sequence; Generating unique corresponding position encoding information for each word in the word sequence; Adding the position encoding information to the word sequence to obtain a target sequence; Dividing the target sequence into block vectors of the same length; The block vector is input into the Transformer encoder for encoding.

4. The method according to claim 1, characterized in that The input layer includes: A first type of information input module is configured to input the first type of data carried by the alarm data into a corresponding encoding module; A second type of information input module is configured to input the second type of data carried by the warning data into a corresponding encoding module; A third type of information input module is configured to input the third type of data carried by the warning data into a corresponding encoding module; A fourth type of information input module, configured to input the fourth type of data carried by the warning data into a corresponding encoding module; The feature quantization layer includes: A first encoding module is configured to be connected to the output end of the first type of information input module, and to encode the first type of data using a Word2Vec model to obtain a first feature code; A second encoding module is configured to be connected to the output end of the second type of information input module, and to encode the second type of data using a one-hot encoding model to obtain a second feature code; A third encoding module is configured to be connected to the output end of the third type of information input module, and to encode the third type of data using a label embedding model to obtain a third feature code; a fourth encoding module, configured to be connected to the output end of the fourth type of information input module, and to encode the fourth type of data by extracting statistical features of the fourth type of information to obtain a fourth feature code; a fifth encoding module, configured to be connected to the output end of the fourth type of information input module, and to encode the fourth type of data at least through a Word2Vec model and a Transformer encoder to obtain a fifth feature code; The target model processing layer includes: A first processing unit is configured to process the input first concatenated vector in sequence through a first fully connected layer, a first normalization processing layer, an activation function, and an anti-overfitting algorithm to obtain a first vector to be concatenated, wherein the first concatenated vector is obtained by concatenating the first feature code, the third feature code, the fourth feature code, and the preprocessed second feature code, and the preprocessing includes one-dimensional convolution and normalization processing; A second processing unit is configured to process the fifth feature code through a second fully connected layer, a second normalization processing layer, an activation function, and an anti-overfitting algorithm in sequence to obtain a second vector to be concatenated; The third processing unit is configured to process the input target splicing vector in sequence through a third fully connected layer, a third normalization processing layer, an activation function, and an anti-overfitting algorithm to obtain the alarm authenticity assessment result, wherein the target splicing vector is obtained by splicing the first vector to be spliced ​​and the second vector to be spliced.

5. The method according to claim 1, characterized in that Before inputting the alarm data into the real alarm recognition model, the method further comprises: training the model processing layer to obtain the target model processing layer; The step of training the model processing layer to obtain the target model processing layer includes: Collect historical alarm data, wherein the historical alarm data includes attack name, HTTP request and response content; Marking the historical alarm data into two types: false alarm and non-false alarm as label data; The historical alarm data is quantized using the feature quantization layer to obtain a feature vector; Using the model processing layer and the feature vector to obtain an authenticity assessment result for the historical alarm data; A loss function is used to determine the difference between the authenticity evaluation result and the corresponding label data to obtain a loss value, and whether to continue training is determined according to the loss value.

6. The method according to claim 5, characterized in that The historical alarm data includes first-category data, second-category data, third-category data and fourth-category data; wherein, The adopting the feature quantization layer to quantize the historical alarm data to obtain a feature vector includes: Converting the text information included in the first category of data into a numerical feature vector to obtain a first feature vector; Convert the second type of data into a sparse vector consisting of 0 and 1 to obtain a second eigenvector; Mapping the third type of data into continuous numerical data to obtain a third eigenvector; Extracting statistical features of characters in the fourth category of data to obtain a fourth feature vector; and The fourth type of data is processed as follows to obtain a fifth feature vector: encoded by the second Word2Vec model, filled with word position information, and encoded by the Transformer encoder; The using the model processing layer and the feature vector to obtain the authenticity evaluation result for the historical alarm data includes: Processing the second eigenvectors one by one through one-dimensional convolution, normalization, and ReLU activation function to obtain an initial second eigenvector; Merging the data obtained by normalizing the first eigenvector, the third eigenvector, and the fourth eigenvector with the initial second eigenvector to obtain a first merged vector; Processing the first merged vectors one by one through a fully connected layer, normalization, an RLU activation function, and dropout to obtain a first vector to be merged; Performing the following operations on the fifth eigenvector in sequence to obtain a second vector to be merged: average pooling, full connection processing, normalization processing, ReLU activation and Dropout operation; Merging the first vector to be merged and the second vector to be merged, and performing multiple full connection processes on the merged result to obtain a one-dimensional vector as the historical alarm data; Among them, the first type of data is used to carry threat information, source address or destination address, the second type of data is unordered classified data, and the unordered classified data includes HTTP request method, the hour of occurrence time or numerical data after binning processing; the third type of data is ordered classified data, and the ordered classified data includes alarm level or confidence level; the fourth type of data includes multi-character type data, and the multi-character type data includes HTTP request, HTTP response or HTTP URL.

7. A device for detecting false alarm information, characterized in that: The device comprises: An alarm information acquisition module is configured to acquire alarm data, wherein the alarm data is generated in real time by a security device; The alarm result authenticity prediction module is configured to input the alarm data into a target true alarm recognition model, and confirm whether the alarm data is a true alarm through the target true alarm recognition model, wherein the target true alarm recognition model includes an input layer, a feature quantization layer, a target model processing layer and an output layer, and the feature quantization layer is at least configured to use a Transformer encoder to encode the dependency and interaction information obtained from the HTTP request and response content.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method for detecting false alarm information as described in any one of claims 1 to 6 can be implemented.

9. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method for detecting false alarm information according to any one of claims 1 to 6 is executed.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting false alarm information as described in any one of claims 1 to 6 can be implemented.

Images