Network traffic warning training and learning system based on Transformer multi-output layer

Abstract

The present invention proposes a network traffic early warning training and learning system based on a Transformer multi-output layer, comprising: a Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, a reinforcement learning agent submodule at the output end of the Transformer submodule, and a submodule at the output end of the reinforcement learning agent submodule for determining whether to output an early warning result. The system is characterized in that the output layer of the Transformer submodule is a modified output layer, which transforms one output layer of the original Transformer into a group of n output layers, and the length of each output layer is adapted to the length of the adaptive time series input submodule at each time point. The present invention proposes a network traffic early warning mechanism based on a Transformer multi-output layer, which can achieve faster and more effective time series security status judgment by modifying the output layer of the Transformer so that the output layer changes from a fixed length to an adaptive length and from a single-layer output to a multi-layer output.

Description

Technical Field

[0001] The present invention belongs to the technical field of network traffic security early warning, and in particular relates to a network traffic early warning training and learning system based on Transformer multi-output layers. Background Art

[0002] With the growing demand for network security, real-time analysis of network traffic for anomaly detection and intrusion prevention has become crucial. Transformer neural networks are widely used for real-time network traffic analysis.

[0003] The difficulty in using Transformer for network traffic early warning lies in the limitations of the existing Transformer neural network structure, making it inadequate for real-time scenarios. This is because the Transformer's classification layer is fixed-length, so its input and output data must also be fixed-length data. If the query is calculated based on a 24-hour daily schedule, the input data must be the length of the accumulated data for 24 hours. If the query is calculated based on an hourly schedule, the input data must be the length of the accumulated data collected over 60 minutes. If the query is set to be 60 minutes or 24 hours, the query must be made at the end of the 60th minute or 24 hours. If the query is set to be made in advance, for example, every 60 minutes, there will be data for the first 15 minutes, but not for the last 45 minutes. This period must be padded with zeros because the data volume is insufficient and can only be padded with zeros. Similarly, if you set the query to be done once every 24 hours, you must query at the last moment of the 24th hour. If you query in advance, for example, 12 hours in advance, there will be data for the first 12 hours, but no data for the last 12 hours. The period without data must be filled with 0, because the data length is calculated based on the length of 24 hours. If the length is insufficient, it can only be filled with 0.

[0004] In short, Transformer input data must rely on a complete data sequence for the entire time period for analysis. If data is queried in advance, missing data points must be padded with "0". This results in excessive invalid information when the Transformer processes the input data, resulting in poor classification results (such as warning or no warning). With this fixed-length input data format, the Transformer is unable to meet the requirements of real-time scenarios, especially in emergency situations that require a quick response. Delayed fault diagnosis may result in potential network threats being missed or unable to be addressed in a timely manner. Summary of the Invention

[0005] In response to the problems existing in the prior art, the present invention proposes a network traffic warning training and learning system based on Transformer multi-output layers. The purpose is to solve the problem that the input data of Transformer must rely on the complete data sequence of the entire time period for analysis. If data is queried in advance in the middle, the places where there is no data need to be supplemented with "0", which makes the Transformer appear to be unable to cope with the real-time scenario. Especially in emergency situations that require a quick response, delayed fault judgment may cause potential network threats to be missed or unable to be responded to in time.

[0006] The present invention proposes the following technical solutions to solve the problems existing in the prior art:

[0007] A network traffic warning training and learning system based on a Transformer multi-output layer includes: a Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, and a reinforcement learning agent submodule at the output end of the Transformer submodule; the output end and input end of the Transformer submodule are further externally connected to a classification loss function submodule; the output end and input end of the reinforcement learning agent submodule are further externally connected to an optimization classification loss function submodule; the Transformer submodule includes a feature extractor and a length adaptive output layer, and the input end of the reinforcement learning agent submodule is respectively connected to the Transformer submodule feature extractor, the Transformer submodule output layer, and the adaptive time series input submodule; the reinforcement learning agent submodule is used to determine the accuracy of the classification result of the length adaptive output layer of the Transformer submodule, and determine whether to output a warning message based on the accuracy of the classification result;

[0008] Its characteristics are: the output layer of the Transformer submodule is a modified set of output layers, which transforms one output layer of the original Transformer into a set of output layers consisting of n output layers, and the length of the input end of each output layer is adapted to the length of the adaptive time series input submodule at each time point; the length-adaptive output layer simultaneously outputs the classification result of the current output layer to the classification loss function submodule and the reinforcement learning agent submodule.

[0009] Furthermore, the set of output layers is the output layer allocated according to the time point: the transformed output layer is provided with a time series length judgment module, which determines which time point the time series input belongs to based on the time series length of the current time point, and the time series input belonging to the time point is allocated to the output layer of that time point.

[0010] Furthermore, the group of output layers is n output layers with adaptive lengths, and the length of each output layer is the same as the input length at the time point to which the output layer belongs.

[0011] Furthermore, the reinforcement learning agent submodule cooperates with the n output layers of the Transformer submodule in each training and learning, determines the accuracy of the classification results of the Transformer submodule based on the current output layer, and determines whether to output warning information based on the accuracy of the classification results, until all n output layers are traversed.

[0012] Furthermore, the classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning, updates the model parameters based on the classification results of the current output layer, and returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters until all n output layers are traversed.

[0013] Furthermore, the optimized classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning process, outputs sampling actions and rewards to the optimized classification loss function submodule based on the classification results of the current output layer, and the optimized classification loss function submodule updates the model parameters and returns the updated model parameters to the reinforcement learning agent submodule until all n output layers are traversed.

[0014] Furthermore, the classification loss function sub-module receives the real warning result at one end of its input and the classification result learned by the output layer of the Transformer sub-module at the other end. Its output end returns the updated model parameters to the Transformer sub-module. The Transformer sub-module updates the feature extractor and the length adaptive output layer according to the updated model parameters, and the updated classification result of the length adaptive output layer is used as the output result of the next round of training data.

[0015] Furthermore, the optimized classification loss function submodule has one input end that shares the real warning result of the classification loss function submodule, and one input end that receives the sampling actions and rewards output by the reinforcement learning submodule. The output end returns the updated model parameters to the reinforcement learning submodule, and the reinforcement learning submodule outputs the updated sampling actions and rewards, which serve as the output results of the next round of training data.

[0016] Furthermore, the reinforcement learning submodule outputs the classification result based on the probability analysis of the accuracy of the classification result of the current output layer of the Transformer submodule in each training and learning, until all n output layers are traversed.

[0017] Advantages and effects of the present invention

[0018] The present invention proposes a network traffic early warning mechanism based on Transformer multi-output layers. The early warning mechanism transforms the output layer of the Transformer, changing the output layer from a fixed length to an adaptive length, and from a single-layer output to a multi-layer output. In addition, a reinforcement learning policy network is used to further judge the accuracy of the Transformer output layer. Compared with traditional classification methods, the model of the present invention can achieve faster and more effective time series security status judgment without the need for complete data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the single output layer structure of the existing Transformer;

[0020] Figure 2 This is a schematic diagram of the network traffic early warning training and learning system based on Transformer multi-output layers of the present invention. DETAILED DESCRIPTION

[0021] Design principle of the present invention

[0022] 1. Innovation of the Present Invention: The innovation lies in the transformation of the Transformer's output layer, transforming it from a single output layer into a group of output layers, and further transforming this group of output layers into a group of adaptive-length output layers. The adaptive-length output layers mean that the length of each output layer changes according to the length of the input data, resulting in more accurate classification results for the output layer. This more accurate classification is achieved because the length of the output layer's inlet is no longer fixed. In other words, the input data stored in the inlet is no longer filled with zeros, but rather valid data with an adaptive length. The Transformer's output layer is structured as follows: an outlet and an inlet. The outlet stores the classification results, and the length of the outlet is independent of the input data, while the length of the inlet must be equal to the length of the input data. This is because when the output layer processes input data, it essentially performs a matrix multiplication involving both the input data and the corresponding parameters of the output layer. Due to the nature of matrix multiplication, the dimensions of both sides of the operation must be the same, meaning that the length of the input data is based on the length of the output layer's inlet.

[0023] 2. Principle of the adaptive length design of the output layer: During the training and learning phase, assuming 24 hours is the longest time period, the total length of the 24-hour time period is calculated and then divided into a set number of parts according to the query interval. For example, if the total length of the 24-hour input data is 240 and it is queried once every hour, the length of each hour is 10. If the length of the time series input submodule received by the Transformer output layer is 10, it is considered to be data from the first hour of the 24-hour period and is assigned to the entry of output layer 1. If the data is queried in the second hour, the length should be the length of the first hour + the length of the second hour, a total of 20. If the length of the time series input submodule received by the Transformer output layer is 20, it is considered to be the time series input at the second time point and is assigned to the entry of output layer 2. And so on. The difference from the existing technology is that the existing technology sets an output layer with a fixed total length. When the first hour of time series input is received, the actual length is only 10. Since the input length needs to correspond to the output layer, the following 230 grids need to be filled with "0". In contrast, the input data length of the present invention corresponds one-to-one with the length of the output layer, and there is no need to fill it with 0. In this way, the data given to the entrance of the Transformer output layer is real data without 0, so the Transformer output layer can perform classification based on the real data, and the classification results are more accurate.

[0024] According to the above principles, the present invention designs a network traffic early warning training and learning system based on Transformer multi-output layer. Figure 2 As shown, it includes: a Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, and a reinforcement learning agent submodule at the output end of the Transformer submodule; the output end and input end of the Transformer submodule are also externally connected to the classification loss function submodule; the output end and input end of the reinforcement learning agent submodule are also externally connected to the optimization classification loss function submodule; the Transformer submodule includes a feature extractor and a length adaptive output layer, and the input end of the reinforcement learning agent submodule is respectively connected to the Transformer submodule feature extractor, the Transformer submodule output layer, and the adaptive time series input submodule; the reinforcement learning agent submodule is used to determine the accuracy of the classification result of the length adaptive output layer of the Transformer submodule, and determine whether to output warning information according to the accuracy of the classification result;

[0025] Its characteristics are: the output layer of the Transformer submodule is a modified set of output layers, which transforms one output layer of the original Transformer into a set of output layers consisting of n output layers, and the length of the input end of each output layer is adapted to the length of the adaptive time series input submodule at each time point; the length-adaptive output layer simultaneously outputs the classification result of the current output layer to the classification loss function submodule and the reinforcement learning agent submodule.

[0026] Furthermore, the set of output layers is the output layer allocated according to the time point: the transformed output layer is provided with a time series length judgment module, which determines which time point the time series input belongs to based on the time series length of the current time point, and the time series input belonging to the time point is allocated to the output layer of that time point.

[0027] Furthermore, the group of output layers is n output layers with adaptive lengths, and the length of each output layer is the same as the input length at the time point to which the output layer belongs.

[0028] Furthermore, the reinforcement learning agent submodule cooperates with the n output layers of the Transformer submodule in each training and learning, determines the accuracy of the classification results of the Transformer submodule based on the current output layer, and determines whether to output warning information based on the accuracy of the classification results, until all n output layers are traversed.

[0029] Furthermore, the classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning, updates the model parameters based on the classification results of the current output layer, and returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters until all n output layers are traversed.

[0030] Furthermore, the optimized classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning process, outputs sampling actions and rewards to the optimized classification loss function submodule based on the classification results of the current output layer, and the optimized classification loss function submodule updates the model parameters and returns the updated model parameters to the reinforcement learning agent submodule until all n output layers are traversed.

[0031] Furthermore, the classification loss function sub-module receives the real warning result at one end of its input and the classification result learned by the output layer of the Transformer sub-module at the other end. Its output end returns the updated model parameters to the Transformer sub-module. The Transformer sub-module updates the feature extractor and the length adaptive output layer according to the updated model parameters, and the updated classification result of the length adaptive output layer is used as the output result of the next round of training data.

[0032] Furthermore, the optimized classification loss function submodule has one input end that shares the real warning result of the classification loss function submodule, and one input end that receives the sampling actions and rewards output by the reinforcement learning submodule. The output end returns the updated model parameters to the reinforcement learning submodule, and the reinforcement learning submodule outputs the updated sampling actions and rewards, which serve as the output results of the next round of training data.

[0033] Furthermore, the reinforcement learning submodule outputs the classification result based on the probability analysis of the accuracy of the classification result of the current output layer of the Transformer submodule in each training and learning, until all n output layers are traversed.

[0034] It should be emphasized that the above specific embodiments are merely explanations of the present invention and are not limitations of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the above embodiments as needed, but as long as they are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. A network traffic early warning training and learning system based on Transformer multi-output layers, including: A Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, and a reinforcement learning agent submodule at the output end of the Transformer submodule; the output and input ends of the Transformer submodule are also externally connected to the classification loss function submodule; the output and input ends of the reinforcement learning agent submodule are also externally connected to the optimization classification loss function submodule; the Transformer submodule includes a feature extractor and a length adaptive output layer, and the input end of the reinforcement learning agent submodule is respectively connected to the Transformer submodule feature extractor, the Transformer submodule output layer, and the adaptive time series input submodule; the reinforcement learning agent submodule is used to determine the accuracy of the classification result of the length adaptive output layer of the Transformer submodule, and determine whether to output a warning message based on the accuracy of the classification result; The invention is characterized in that the output layer of the Transformer submodule is a modified set of output layers, which transforms one output layer of the original Transformer into a set of output layers consisting of n output layers, and the length of the input end of each output layer is adapted to the length of the adaptive time series input submodule at each time point; the length-adaptive output layer simultaneously outputs the classification result of the current output layer to the classification loss function submodule and the reinforcement learning agent submodule.

2. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The set of output layers is the output layer allocated according to the time point: the transformed output layer is provided with a time series length judgment module, which determines which time point the time series input belongs to based on the time series length of the current time point, and the time series input belonging to the time point is allocated to the output layer of that time point.

3. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The set of output layers is n output layers with adaptive lengths, and the length of each output layer is the same as the input length at the time point to which the output layer belongs.

4. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The reinforcement learning agent submodule cooperates with the n output layers of the Transformer submodule in each training and learning process, determines the accuracy of the Transformer submodule classification results based on the current output layer, and determines whether to output warning information based on the accuracy of the classification results, until all n output layers are traversed.

5. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning process, updates the model parameters based on the classification results of the current output layer, and returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and length adaptive output layer according to the updated model parameters until all n output layers are traversed.

6. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The optimized classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning process, outputs sampling actions and rewards to the optimized classification loss function submodule based on the classification results of the current output layer, and the optimized classification loss function submodule updates the model parameters and returns the updated model parameters to the reinforcement learning agent submodule until all n output layers are traversed.

7. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The classification loss function submodule receives the real warning result at one end of its input and the classification result of the training and learning of the output layer of the Transformer submodule at the other end. Its output end returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters, and the updated classification result of the length adaptive output layer is used as the output result of the next round of training data.

8. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The optimized classification loss function submodule has one input end that shares the real warning result of the classification loss function submodule, and one input end that receives the sampling action and reward output by the reinforcement learning submodule. The output end returns the updated model parameters to the reinforcement learning submodule, and the reinforcement learning submodule outputs the updated sampling action and reward, which serve as the output result of the next round of training data.

9. The network traffic early warning training and learning system based on Transformer multi-output layers according to claim 1 is characterized by: The reinforcement learning submodule outputs the classification result based on the probability analysis of the accuracy of the classification result of the current output layer of the Transformer submodule in each training learning, until all n output layers are traversed.

Images