Network traffic early warning training learning system based on Transform multiple output layers
By transforming the output layer of Transformer to adaptive length and combining with the reinforcement learning agent submodule, the problem of fixed length limitation in real-time network traffic warning is solved, and faster and more accurate warning judgment is achieved.
Patent Information
- Application Number
- CN202411936635.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-04
- Estimated Expiration
- 2044-12-26
AI Technical Summary
In network traffic warning, the existing Transformer neural network seems to be powerless in scenarios that pursue real-time performance due to the fixed-length input and output structure. Especially in emergency situations, potential network threats may be missed or unable to be dealt with in a timely manner.
A network traffic early warning training learning system based on Transformer multi-output layer is designed. By transforming the output layer of Transformer, it changes from a fixed length to an adaptive length, and introducing a reinforcement learning agent submodule to determine the accuracy of the output layer classification results and decide whether to output early warning information.
It realizes rapid and efficient time series security status judgment without complete data, and improves network traffic warning accuracy and response speed in real-time and emergency situations.
Smart Images
Figure CN119766540A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of network traffic security early warning, and in particular relates to a network traffic early warning training learning system based on Transformer multi-output layers. Background Art
[0002] With the growing demand for network security, real-time analysis of network traffic for anomaly detection and intrusion prevention has become critical. Transformer neural networks are widely used in the analysis of real-time network traffic.
[0003] The difficulty of using Transformer for network traffic warning is that it is limited by the existing Transformer neural network structure and is incapable of real-time scenarios. The reason is that the classification layer of Transformer is of fixed length, so its input and output data must be fixed length data. If the query is calculated once a day for 24 hours, the input data must be the length of the accumulated data for 24 hours; if the query is calculated once an hour, the length of the input data must be the length of the accumulated data collected within 60 minutes. If the query is set to be once every 60 minutes or once every 24 hours, it must be queried at the last moment of the 60th minute or 24 hours. If the query is made in advance, for example, the query is set to be once every 60 minutes, there is data in the first 15 minutes, but no data in the last 45 minutes. The period without data must be filled with 0 because the length of the data volume is not enough and can only be filled with 0. Similarly, if you set the query to be done once every 24 hours, you must query at the last moment of the 24th hour. If you query in advance, for example, 12 hours in advance, there will be data in the first 12 hours, but no data in the last 12 hours. The period without data must be filled with 0, because the data length is calculated according to the length of 24 hours. If the length is not enough, it can only be filled with 0.
[0004] In short, the input data of Transformer must rely on the complete data sequence of the entire time period for analysis. If the data is queried in advance, the places where there is no data need to be supplemented with "0", which causes too much invalid information when Transformer processes the input data, resulting in mediocre classification (classification such as: warning or no warning). Under this fixed-length input data format of Transformer, it seems powerless in the pursuit of real-time scenarios, especially in emergency situations that require rapid response. Delayed fault judgment may cause potential network threats to be missed or unable to be responded to in time. Summary of the invention
[0005] In view of the problems existing in the prior art, the present invention proposes a network traffic warning training and learning system based on Transformer multi-output layers, with the purpose of solving the problem that the input data of Transformer must rely on the complete data sequence of the entire time period for analysis. If the data is queried in advance in the middle, the places where there is no data need to be supplemented with "0", resulting in the Transformer being unable to cope with the situation in which real-time performance is pursued. Especially in emergency situations that require a quick response, delayed fault judgment may cause potential network threats to be missed or unable to be responded to in a timely manner.
[0006] The present invention proposes the following technical solutions to solve the problems existing in the prior art:
[0007] A network traffic warning training and learning system based on Transformer multi-output layers comprises: a Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, and a reinforcement learning agent submodule at the output end of the Transformer submodule; the output end and the input end of the Transformer submodule are also externally connected to a classification loss function submodule; the output end and the input end of the reinforcement learning agent submodule are also externally connected to an optimization classification loss function submodule; the Transformer submodule comprises a feature extractor and a length adaptive output layer, and the input end of the reinforcement learning agent submodule is respectively connected to the Transformer submodule feature extractor, the Transformer submodule output layer, and the adaptive time series input submodule; the reinforcement learning agent submodule is used to determine the accuracy of the classification result of the length adaptive output layer of the Transformer submodule, and determine whether to output warning information according to the accuracy of the classification result;
[0008] Its characteristics are: the output layer of the Transformer submodule is a modified group of output layers, which transforms an output layer of the original Transformer into a group of output layers composed of n output layers, and the length of the entry end of each output layer is adapted to the length of the adaptive time series input submodule at each time point; the length adaptive output layer outputs the classification result of the current output layer to the classification loss function submodule and the reinforcement learning agent submodule at the same time.
[0009] Furthermore, the group of output layers is the output layer allocated according to the time point: the transformed output layer is provided with a time series length judgment module, which judges which time point the time series input belongs to according to the time series length of the current time point, and the time series input belongs to which time point it is allocated to the output layer of that time point.
[0010] Furthermore, the group of output layers is n output layers with adaptive lengths, and the length of each output layer is the same as the input length at the time point to which the output layer belongs.
[0011] Furthermore, the reinforcement learning agent submodule cooperates with the n output layers of the Transformer submodule in each training and learning, determines the accuracy of the classification result of the Transformer submodule according to the current output layer, and determines whether to output warning information according to the accuracy of the classification result, until all n output layers are traversed.
[0012] Furthermore, in each training and learning, the classification loss function submodule cooperates with the n output layers of the Transformer submodule, updates the model parameters according to the classification result of the current output layer, and returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters until all n output layers are traversed.
[0013] Furthermore, the optimized classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning, outputs sampling actions and rewards to the optimized classification loss function submodule for the classification results of the current output layer, and the optimized classification loss function submodule updates the model parameters and returns the updated model parameters to the reinforcement learning agent submodule until all n output layers are traversed.
[0014] Furthermore, the classification loss function submodule receives the real warning result at one end of its input and receives the classification result of the training and learning of the output layer of the Transformer submodule at the other end, and its output end returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters, and the updated classification result of the length adaptive output layer is used as the output result of the next round of training data.
[0015] Furthermore, the optimized classification loss function submodule has one input end that shares the real warning result of the classification loss function submodule, and one input end that receives the sampling actions and rewards output by the reinforcement learning submodule, and an output end that returns the updated model parameters to the reinforcement learning submodule, and the reinforcement learning submodule outputs the updated sampling actions and rewards, which serve as the output results of the next round of training data.
[0016] Furthermore, the reinforcement learning submodule outputs the classification result according to the probability analysis of the accuracy of the classification result of the current output layer of the Transformer submodule in each training and learning, until all n output layers are traversed.
[0017] Advantages and effects of the present invention
[0018] The present invention proposes a network traffic early warning mechanism based on Transformer multi-output layers. The early warning mechanism transforms the output layer of Transformer so that the output layer changes from a fixed length to an adaptive length, and from a single-layer output to a multi-layer output. A reinforcement learning strategy network is used to further judge the accuracy of the Transformer output layer. Compared with traditional classification methods, the model of the present invention can achieve faster and more effective time series security status judgment without the need for complete data. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 This is a schematic diagram of the single output layer structure of the existing Transformer;
[0020] Figure 2 This is a schematic diagram of the network traffic early warning training and learning system based on Transformer multi-output layers of the present invention. DETAILED DESCRIPTION
[0021] Design principle of the present invention
[0022] 1. The innovation of the present invention: The innovation lies in the transformation of the output layer of the Transformer, from one output layer to a group of output layers, and this group of output layers becomes a group of adaptive length output layers, wherein the length of each output layer changes according to the length of the input data, so that the classification result of the output layer is more accurate. The reason for the more accurate classification is that the length of the output layer entrance is no longer fixed, that is, the input data stored in the entrance is no longer filled with zero data, but is valid data with adaptive length. The structure of the output layer of the Transformer is: there is an exit and an entrance, the exit is used to store the classification result, the exit length is independent of the input data, but the entrance length must be equal to the length of the input data. Because when the output layer processes the input data, it is essentially performing a matrix multiplication operation involving both the input data and the corresponding parameters of the output layer, and due to the characteristics of matrix multiplication, the dimensions corresponding to the two sides of the operation are required to be the same, that is, the length of the input data is based on the entrance length of the output layer.
[0023] 2. Principle of the design of the adaptive length of the output layer: During the training and learning phase, assuming that 24 hours is the longest time period, the total length of the 24 hours is counted, and then divided into the set number of copies according to the query interval. For example, the total length of the 24-hour input data is 240, and the query is made once every hour, and the length of each hour is 10. If the length of the time series input submodule received by the Transformer output layer is 10, it is judged to be the data of the first hour of 24 hours, and this data is assigned to the entrance of output layer 1. If the data is queried in the second hour, the length should be the length of the first hour + the length of the second hour, which is 20 in total. If the length of the time series input submodule received by the Transformer output layer is 20, it is considered to be the time series input at the second time point, and the data is assigned to the entrance of output layer 2. And so on. The difference from the prior art is that the prior art sets an output layer with a fixed total length. When the first hour time series input is received, the actual length is only 10. Since the input length needs to correspond to the output layer, the following 230 grids need to be filled with "0". The input data length of the present invention corresponds one-to-one to the length of the output layer, and there is no need to fill in 0. In this way, the data given to the entrance of the Transformer output layer is real data without 0, so the Transformer output layer can be classified according to the real data, and the classification result is more accurate.
[0024] According to the above principles, the present invention designs a network traffic early warning training and learning system based on Transformer multi-output layer. Figure 2 As shown, it includes: a Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, and a reinforcement learning agent submodule at the output end of the Transformer submodule; the output end and input end of the Transformer submodule are also externally connected to a classification loss function submodule; the output end and input end of the reinforcement learning agent submodule are also externally connected to an optimization classification loss function submodule; the Transformer submodule includes a feature extractor and a length adaptive output layer, and the input end of the reinforcement learning agent submodule is respectively connected to the Transformer submodule feature extractor, the Transformer submodule output layer, and the adaptive time series input submodule; the reinforcement learning agent submodule is used to determine the accuracy of the classification result of the length adaptive output layer of the Transformer submodule, and determine whether to output warning information according to the accuracy of the classification result;
[0025] Its characteristics are: the output layer of the Transformer submodule is a modified group of output layers, which transforms an output layer of the original Transformer into a group of output layers composed of n output layers, and the length of the entry end of each output layer is adapted to the length of the adaptive time series input submodule at each time point; the length adaptive output layer outputs the classification result of the current output layer to the classification loss function submodule and the reinforcement learning agent submodule at the same time.
[0026] Furthermore, the group of output layers is the output layer allocated according to the time point: the transformed output layer is provided with a time series length judgment module, which judges which time point the time series input belongs to according to the time series length of the current time point, and the time series input belongs to which time point it is allocated to the output layer of that time point.
[0027] Furthermore, the group of output layers is n output layers with adaptive lengths, and the length of each output layer is the same as the input length at the time point to which the output layer belongs.
[0028] Furthermore, the reinforcement learning agent submodule cooperates with the n output layers of the Transformer submodule in each training and learning, determines the accuracy of the classification result of the Transformer submodule according to the current output layer, and determines whether to output warning information according to the accuracy of the classification result, until all n output layers are traversed.
[0029] Furthermore, in each training and learning, the classification loss function submodule cooperates with the n output layers of the Transformer submodule, updates the model parameters according to the classification result of the current output layer, and returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters until all n output layers are traversed.
[0030] Furthermore, the optimized classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning, outputs sampling actions and rewards to the optimized classification loss function submodule for the classification results of the current output layer, and the optimized classification loss function submodule updates the model parameters and returns the updated model parameters to the reinforcement learning agent submodule until all n output layers are traversed.
[0031] Furthermore, the classification loss function submodule receives the real warning result at one end of its input and receives the classification result of the training and learning of the output layer of the Transformer submodule at the other end, and its output end returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters, and the updated classification result of the length adaptive output layer is used as the output result of the next round of training data.
[0032] Furthermore, the optimized classification loss function submodule has one input end that shares the real warning result of the classification loss function submodule, and one input end that receives the sampling actions and rewards output by the reinforcement learning submodule, and an output end that returns the updated model parameters to the reinforcement learning submodule, and the reinforcement learning submodule outputs the updated sampling actions and rewards, which serve as the output results of the next round of training data.
[0033] Furthermore, the reinforcement learning submodule outputs the classification result according to the probability analysis of the accuracy of the classification result of the current output layer of the Transformer submodule in each training and learning, until all n output layers are traversed.
[0034] It should be emphasized that the above specific embodiments are merely explanations of the present invention, and they are not limitations of the present invention. After reading this specification, those skilled in the art can make modifications to the above embodiments without any creative contribution as needed, but as long as they are within the scope of the claims of the present invention, they are protected by patent law.
Claims
1. A network traffic warning training and learning system based on Transformer multi-output layer, comprising: A Transformer submodule, an adaptive time series input submodule at the input end of the Transformer submodule, and a reinforcement learning agent submodule at the output end of the Transformer submodule; the output end and input end of the Transformer submodule are also externally connected to a classification loss function submodule; the output end and input end of the reinforcement learning agent submodule are also externally connected to an optimization classification loss function submodule; the Transformer submodule includes a feature extractor and a length adaptive output layer, and the input end of the reinforcement learning agent submodule is respectively connected to the Transformer submodule feature extractor, the Transformer submodule output layer, and the adaptive time series input submodule; the reinforcement learning agent submodule is used to determine the accuracy of the classification result of the length adaptive output layer of the Transformer submodule, and determine whether to output warning information according to the accuracy of the classification result; The invention is characterized in that the output layer of the Transformer submodule is a modified group of output layers, which transforms an output layer of the original Transformer into a group of output layers composed of n output layers, and the length of the entry end of each output layer is adapted to the length of the adaptive time series input submodule at each time point; the length adaptive output layer outputs the classification result of the current output layer to the classification loss function submodule and the reinforcement learning agent submodule at the same time.
2. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The group of output layers is the output layer allocated according to the time point: the transformed output layer is provided with a time series length judgment module, which judges which time point the time series input belongs to according to the time series length of the current time point, and the time series input belonging to the time point is allocated to the output layer of that time point.
3. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The group of output layers is n output layers with adaptive lengths, and the length of each output layer is the same as the input length at the time point to which the output layer belongs.
4. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The reinforcement learning agent submodule cooperates with the n output layers of the Transformer submodule in each training and learning, determines the accuracy of the classification result of the Transformer submodule according to the current output layer, and determines whether to output warning information according to the accuracy of the classification result, until all n output layers are traversed.
5. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning, updates the model parameters according to the classification result of the current output layer, and returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters until all the n output layers are traversed.
6. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The optimized classification loss function submodule cooperates with the n output layers of the Transformer submodule in each training and learning, outputs sampling actions and rewards to the optimized classification loss function submodule for the classification results of the current output layer, and the optimized classification loss function submodule updates the model parameters and returns the updated model parameters to the reinforcement learning agent submodule until all n output layers are traversed.
7. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The classification loss function submodule receives the real warning result at one input end and receives the classification result of the training and learning of the output layer of the Transformer submodule at the other end. The output end returns the updated model parameters to the Transformer submodule. The Transformer submodule updates the feature extractor and the length adaptive output layer according to the updated model parameters, and the updated classification result of the length adaptive output layer is used as the output result of the next round of training data.
8. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The optimized classification loss function submodule has one input end that shares the real warning result of the classification loss function submodule, and one input end that receives the sampling actions and rewards output by the reinforcement learning submodule. The output end returns the updated model parameters to the reinforcement learning submodule, and the reinforcement learning submodule outputs the updated sampling actions and rewards, which serve as the output results of the next round of training data.
9. According to claim 1, a network traffic early warning training and learning system based on Transformer multi-output layer is characterized by: The reinforcement learning submodule outputs the classification result according to the probability analysis of the accuracy of the classification result of the current output layer of the Transformer submodule in each training and learning, until all n output layers are traversed.