Unmanned aerial vehicle network intrusion detection method based on lightweight Transform

Abstract

The invention discloses an unmanned aerial vehicle network intrusion detection method based on a lightweight Transform, belongs to the technical field of network technology security, and can improve the detection performance of unmanned aerial vehicle network traffic. Comprising the following steps: capturing network traffic data in real time through traffic monitoring equipment or software deployed on a network key node to obtain a network traffic sequence; performing standardization processing on the network flow sequence; increasing position information for the standardized network flow sequence by adopting rotation position coding; performing local feature aggregation and global feature enhancement on the encoded network traffic sequence by using a local aggregation attention unit; and extracting the enhanced features by using a lightweight feedforward neural network, processing the extracted features by using a stacked multi-layer lightweight Transformer encoder, outputting a detection result by using a SoftMax classifier, and detecting an attack type.

Description

Technical Field

[0001] The present invention relates to the field of network technology security technology, and in particular to a drone network intrusion detection method based on a lightweight Transformer. Background Art

[0002] As an emerging technology, drones have been widely used in aerial photography, logistics, agriculture, and military fields in recent years. Although drone technology has been widely used, the security of drone networks has not been fully guaranteed. With the development of information theft technology and the increase in the types of network attacks, ground station control of drones, communication between drones, and the safety of drones themselves are facing huge threats.

[0003] Due to its mobility and openness, drone networks are vulnerable to a variety of attacks, including denial of service (DoS) attacks, information theft, identity forgery, GPS (Global Positioning System) signal spoofing, etc. Attacks may reduce the overall performance of the network, threaten the security of drone communications, and steal user confidential information. In serious cases, drones may crash directly, causing huge losses to civil and military applications.

[0004] The research on drone network traffic intrusion detection technology aims to timely detect and respond to potential malicious attacks by monitoring and analyzing drone network traffic. This technology can detect abnormal behavior, malicious traffic and potential security threats by real-time capture, analysis and identification of drone network traffic, thereby ensuring the normal operation and data security of drone systems and providing strong support for drones to carry out various tasks.

[0005] The disclosure of the above background technology content is only used to assist in understanding the concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0006] The present application provides a drone network intrusion detection method based on a lightweight Transformer, which can improve the detection performance of drone network traffic.

[0007] To achieve the above objectives, the present application discloses the following technical solutions:

[0008] A UAV network intrusion detection method based on lightweight Transformer includes the following steps:

[0009] By deploying traffic monitoring equipment or software on key network nodes, network traffic data can be captured in real time to obtain network traffic sequences;

[0010] Standardize network traffic sequences;

[0011] Rotational position coding is used to add position information to the standardized network traffic sequence;

[0012] The local aggregation attention unit is used to perform local feature aggregation and global feature enhancement on the encoded network traffic sequence;

[0013] A lightweight feedforward neural network is used to extract the enhanced features, and after being processed by a stacked multi-layer lightweight Transformer encoder, a SoftMax classifier is used to output the detection results and detect the attack type.

[0014] In some possible implementations, the step of normalizing the network traffic sequence includes:

[0015] Normalize the numerical features so that they are distributed on the same scale;

[0016] Label encoding is performed on categorical features to convert them into numerical features.

[0017] In some possible implementations, the rotational position encoding is calculated as follows:

[0018]

[0019] Among them, the rotation matrix

[0020]

[0021]

[0022] X is the network traffic sequence data;

[0023] m is the position of the data in the sequence;

[0024] W is a trainable parameter matrix.

[0025] In some possible implementations, the local aggregation attention unit includes a grouped linear transformation, a nonlinear activation function, a layer normalization, a self-attention mechanism, and a residual connection; the local aggregation attention unit is used to perform dimensionality reduction processing on the input network traffic sequence, perform feature aggregation on the local information through a grouped linear transformation, and then use the self-attention mechanism to perform global perception on the aggregated low-dimensional features, perform dimensionality increase processing on the vector after processing by the self-attention mechanism, and finally add a residual connection to obtain an enhanced feature F.

[0026] In some possible implementations, in the local feature aggregation stage, it is assumed that there are L layers of grouped linear transformations;

[0027] The first [L / 2] layers are used for dimensionality increase, and the remaining L-[L / 2] layers are used for dimensionality reduction. The specific process of calculating the number of groups of the linear transformation of each layer is as follows:

[0028]

[0029] Where:

[0030] n l The number of groups for the first layer grouping linear transformation;

[0031] n max The maximum number of groups for grouped linear transformation sets the upper limit of the number of groups.

[0032] In some possible implementations, a residual concatenation operation is used in each layer of grouped linear transformation, and the calculation formula of each layer of grouped linear transformation is:

[0033]

[0034] In the formula, G(·) represents the grouped linear transformation;

[0035] M(·) represents residual concatenation, nonlinear activation function GELU and layer normalization operations;

[0036] W l Represents the learnable parameter matrix of the first layer of the grouped linear transformation;

[0037] b l represents the bias vector;

[0038] g l The number of groups to group the linear transformation for layer 1.

[0039] In some possible implementations, the self-attention mechanism calculates the correlation of sequence data through the query vector and the key vector to obtain an attention matrix, and then calculates the globally enhanced features with the value matrix as follows:

[0040]

[0041] In the formula, the query vector Q = YW Q , key vector K = YW K Sum vector V = YW V , where W Q , W K , W V is the learnable parameter matrix; d k is the embedding dimension, softmax is the activation function;

[0042] Enhanced feature F = I + W s A

[0043] Where: W s It is the parameter matrix for dimensionality-upgrading the output of the self-attention mechanism; I is the residual feature of the input.

[0044] In some possible implementations, the lightweight feedforward neural network includes a fully connected layer, a nonlinear activation function, and a residual connection; assuming that the dimension of the input feature F is d m , the lightweight feedforward neural network first reduces the dimension to Then upgrade to d m ;

[0045] The calculation process of lightweight feedforward neural network for feature extraction is:

[0046] O m =Feedforward(F)=F+((Relu(FW1+b1))W2+b2)

[0047] Where W1, b1 are the parameter matrix and bias vector during dimensionality reduction, W2, b2 are the parameter matrix and bias vector during dimensionality reduction, and Relu(·) is a nonlinear activation function.

[0048] In some possible implementations, the SoftMax classifier outputs the detection result as follows:

[0049]

[0050] In the formula, O M is the output of the Mth layer lightweight Transformer encoder, exp(·) represents the exponential function of e, and K represents the total number of categories.

[0051] In some possible implementations, the lightweight Transformer encoder uses an adaptive model scaling mechanism.

[0052] Compared with the prior art, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0053] 1. Rotational position encoding is used to represent the position of the network traffic sequence with relative position information, avoiding the overfitting problem caused by absolute position sensitivity, enhancing the ability to represent the position information of the network traffic sequence, and helping to better capture data features, thereby improving the detection performance of drone network traffic.

[0054] 2. The local aggregation attention unit is used to perform feature aggregation on local information through grouped linear transformation, and then the self-attention mechanism is used to perceive and enhance the global information. It can effectively aggregate and extract local features, improve the self-attention mechanism's ability to focus on global information, thereby enhancing the ability to extract complex features in network traffic data, and thus improving the detection performance of drone network traffic.

[0055] 3. The lightweight Transformer encoder uses an adaptive model scaling mechanism, which can dynamically adjust the model depth, making the model depth more adaptable to different feature extraction stages and effectively controlling the number of model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0057] Figure 1 Schematic diagram of a process of drone network intrusion detection method based on lightweight Transformer provided in some embodiments of the present application Figure 1 ;

[0058] Figure 2 Schematic diagram of a process of drone network intrusion detection method based on lightweight Transformer provided in some embodiments of the present application Figure 2 ;

[0059] Figure 3 This is a schematic diagram of the lightweight Transformer network structure;

[0060] Figure 4 This is a graph showing the comparative experimental results of the UNSWNB15 dataset;

[0061] Figure 5 This is a comparison experiment result chart of CICIDS2017 dataset;

[0062] Figure 6 This is the experimental result of comparing the generalization performance of the CICIDS2017 noise dataset;

[0063] Figure 7 This is a graph showing the generalization performance comparison experimental results of the UNSWNB15 noise dataset. DETAILED DESCRIPTION

[0064] Specific embodiments of the present invention will now be mentioned in detail. Although the present invention is described in conjunction with these specific embodiments, it should be appreciated that it is not intended to limit the present invention to these specific embodiments. On the contrary, these embodiments are intended to cover substitutions, changes or equivalent embodiments that may be included in the spirit and scope of the invention defined by the claims. In the following description, a large number of specific details are set forth in order to provide a comprehensive understanding of the present invention. The present invention may be implemented without some or all of these specific details.

[0065] When used in conjunction with "including," "methods comprising," or similar language in this specification and the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0066] Application Overview: As an emerging technology, drones have been widely used in aerial photography, logistics, agriculture, and military fields in recent years. Although drone technology has been widely used, the security of drone networks has not been fully guaranteed. With the development of information theft technology and the increase in the types of network attacks, ground station control of drones, communication between drones, and the safety of drones themselves are facing huge threats.

[0067] Due to its mobility and openness, drone networks are vulnerable to a variety of attacks, including denial of service (DoS) attacks, information theft, identity forgery, GPS (Global Positioning System) signal spoofing, etc. Attacks may reduce the overall performance of the network, threaten the security of drone communications, and steal user confidential information. In serious cases, drones may crash directly, causing huge losses to civil and military applications.

[0068] In response to the above technical problems, the overall idea of ​​the technical solution provided by this application is as follows: a drone network intrusion detection method based on a lightweight Transformer is provided, comprising the following steps: through traffic monitoring equipment or software deployed on key network nodes, network traffic data is captured in real time to obtain a network traffic sequence; the network traffic sequence is standardized; rotational position encoding is used to add position information to the standardized network traffic sequence; local aggregation attention units are used to perform local feature aggregation and global feature enhancement on the encoded network traffic sequence; a lightweight feedforward neural network is used to extract the enhanced features, and after processing by stacked multi-layer lightweight Transformer encoders, a SoftMax classifier is used to output the detection results to detect the attack type.

[0069] This method uses rotational position encoding to represent the position of the network traffic sequence with relative position information, avoiding the overfitting problem caused by absolute position sensitivity, enhancing the ability to represent the position information of the network traffic sequence, and helping to better capture data features.

[0070] The local aggregation attention unit is adopted to perform feature aggregation on local information through grouped linear transformation, and then the self-attention mechanism is used to realize the perception and enhancement of global information. It can effectively aggregate and extract local features, improve the ability of the self-attention mechanism to pay attention to global information, and thus enhance the ability to extract complex features in network traffic data.

[0071] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically introduced in conjunction with the accompanying drawings. Figure 1 Combined with Figure 2 and Figure 3 , the embodiment of the present application provides a drone network intrusion detection method based on a lightweight Transformer, comprising the following steps:

[0072] S101: Capture network traffic data in real time by deploying traffic monitoring equipment or software on key network nodes to obtain a network traffic sequence;

[0073] Specifically, the network traffic sequence includes, but is not limited to: source IP address, destination IP address, port number, protocol type, data packet size, and transmission time.

[0074] S102: Standardizing the network traffic sequence;

[0075] Specifically, in some embodiments, the network traffic sequence may be standardized by the following steps:

[0076] The first step is to normalize the numerical features so that they are distributed on the same scale;

[0077] The second step is to perform label encoding on the categorical features and convert them into numerical features.

[0078] S103: adding position information to the standardized network traffic sequence by using rotational position coding;

[0079] Specifically, in some embodiments, the rotational position encoding is calculated as follows:

[0080]

[0081] Among them, the rotation matrix

[0082]

[0083]

[0084] X is the network traffic sequence data;

[0085] m is the position of the data in the sequence;

[0086] W is a trainable parameter matrix.

[0087] S104: using a local aggregation attention unit to perform local feature aggregation and global feature enhancement on the encoded network traffic sequence;

[0088] Specifically, in some embodiments, the local aggregation attention unit includes a grouped linear transformation, a nonlinear activation function, a layer normalization, a self-attention mechanism and a residual connection; the local aggregation attention unit is used to reduce the dimension of the input network traffic sequence, perform feature aggregation on the local information through a grouped linear transformation, and then use the self-attention mechanism to perform global perception on the aggregated low-dimensional features, perform dimensionality increase on the vector after processing by the self-attention mechanism, and finally add a residual connection to obtain an enhanced feature F.

[0089] In the local feature aggregation stage, it is assumed that there are L layers of grouped linear transformations;

[0090] The first [L / 2] layers are used for dimensionality increase, and the remaining L-[L / 2] layers are used for dimensionality reduction. The specific process of calculating the number of groups of the linear transformation of each layer is as follows:

[0091]

[0092] Where:

[0093] n l The number of groups for the first layer grouping linear transformation;

[0094] n max The maximum number of groups for grouped linear transformation sets the upper limit of the number of groups.

[0095] Preferably, in some embodiments, a residual concatenation operation is used in each layer of grouped linear transformation, and the calculation formula of each layer of grouped linear transformation is:

[0096]

[0097] In the formula, G(·) represents the grouped linear transformation;

[0098] M(·) represents residual concatenation, nonlinear activation function GELU and layer normalization operations;

[0099] W l Represents the learnable parameter matrix of the first layer of the grouped linear transformation;

[0100] b l represents the bias vector;

[0101] g l The number of groups to group the linear transformation for layer 1.

[0102] Preferably, in some embodiments, the self-attention mechanism calculates the correlation of the sequence data through the query vector and the key vector to obtain the attention matrix, and then calculates the globally enhanced features with the value matrix as follows:

[0103]

[0104] In the formula, the query vector Q = YW Q , key vector K = YW K Sum vector V = YW V , where W Q , W K , W V is the learnable parameter matrix; d k is the embedding dimension, softmax is the activation function;

[0105] Enhanced feature F = I + W s A

[0106] Where: W s It is the parameter matrix for dimensionality-upgrading the output of the self-attention mechanism; I is the residual feature of the input.

[0107] S105: A lightweight feedforward neural network is used to extract the enhanced features, and after being processed by a stacked multi-layer lightweight Transformer encoder, a SoftMax classifier is used to output the detection results to detect the attack type.

[0108] Specifically, in some embodiments, the lightweight feedforward neural network includes a fully connected layer, a nonlinear activation function and a residual connection; assuming that the dimension of the input feature F is d m , the lightweight feedforward neural network first reduces the dimension to Then upgrade to d m ;

[0109] The calculation process of lightweight feedforward neural network for feature extraction is:

[0110] O m =Feedforward(F)=F+((Relu(FW1+b1))W2+b2)

[0111] Where W1, b1 are the parameter matrix and bias vector during dimensionality reduction, W2, b2 are the parameter matrix and bias vector during dimensionality reduction, and Relu(·) is a nonlinear activation function.

[0112] The SoftMax classifier outputs the detection results as follows:

[0113]

[0114] In the formula, O M is the output of the Mth layer lightweight Transformer encoder, exp(·) represents the exponential function of e, and K represents the total number of categories.

[0115] In order to evaluate the performance of the proposed lightweight Transformer-based drone traffic intrusion detection method, this paper adopts a series of evaluation indicators, including accuracy, precision, recall and F1 score. The hyperparameters of the model used in the training phase are shown in Table 1.

[0116] Table 1 Model hyperparameter combinations

[0117]

[0118] In order to verify the advantages of the lightweight Transformer model, this paper conducted comparative experiments and compared the Transformer model with other advanced machine learning models, including DNN

[20] , convolutional neural network (CNN), long short-term memory network (LSTM) and CNN-GRU. The detection performance of each method was measured using accuracy, precision, recall and F1 score. The results are shown in Figure 2. Figure 4-Figure 7 shown. Figure 4 The test results of five methods in the comparative experiment on the CICIDS2017 dataset are shown. The proposed method based on lightweight Transformer has achieved better performance in various indicators, reflecting the advantages of the proposed method in extracting network sequence data. Compared with DNN, the accuracy of the proposed method has increased by 4.52%, compared with CNN and LSTM, the accuracy has increased by 6.03% and 7.98% respectively, and compared with the CNN-GRU fusion model, it has still increased by 1.92%. The results show that the proposed method has advantages in extracting local and global temporal relationships and can better extract complex features in traffic data.

[0119] Figure 5 The test results of five methods in the comparative experiment on the UNSWNB15 dataset are shown. The results show the advantages of the proposed method in extracting sequence data. Compared with DNN, the accuracy of the proposed method is improved by 2.96%, compared with CNN and LSTM, the accuracy is improved by 5.57% and 7.52% respectively, and compared with the CNN-GRU fusion model, it is still improved by 1.46%. The results show that the proposed method has advantages in extracting global time series and can better extract complex features in traffic data.

[0120] The results of comparative experiments show that the Transformer-based model outperforms other comparative models in terms of accuracy, precision, recall and F1 score. The performance of DNN, CNN and LSTM on the CICIDS2017 dataset and UNSWNB15 dataset is not stable, while the methods proposed in this paper achieve better detection results. In order to verify the network traffic detection effect under complex conditions, the generalization performance of the proposed method is verified under noisy conditions.

[0121] Figure 6 The detection results of five methods in the comparative experiment on the CICIDS2017 noise dataset are shown. The proposed method based on lightweight Transformer has achieved better detection performance in various indicators, which reflects the advantages of the proposed method in extracting sequence data, especially when the noise is more complex. The method proposed in this paper can maintain the detection performance more robustly. At -5dB, it still maintains a detection accuracy of 91.25%. Figure 6 The generalization performance of the comparative experiment under different signal-to-noise ratio conditions is demonstrated. Although the detection performance is better as the signal-to-noise ratio increases, the lightweight Transformer-based method is more stable. The proposed lightweight Transformer-based method has achieved superior detection performance in various indicators, and still maintains a detection accuracy of 85.32% at -5dB, which is more than 5.11% higher than other methods.

[0122] Figure 7 The detection results of the UNSWNB15 dataset comparative experiment under different signal-to-noise ratio conditions are shown. Although the detection performance is better as the signal-to-noise ratio increases, the Transformer-based method is more stable.

[0123] In order to evaluate the importance of local aggregate attention units and lightweight feedforward neural networks (FFN) in the Transformer-based drone traffic intrusion detection model, this paper conducted an ablation experiment. In the experiment, this paper built a complete Transformer model as the baseline model, and then gradually replaced the multi-head attention layer with local aggregate attention units, replaced the traditional FFN with lightweight FFN, and observed the performance changes of the model on the test set. The experimental results of replacing 4 modules are shown in Table 2.

[0124] Table 2 Ablation experiment results (%)

[0125]

[0126] As shown in the ablation experiment results in Table 2, after using the local aggregation attention unit, the accuracy of the model increased from 97.84% to 98.44%, and from 97.34% to 99.34%, respectively. This shows that the local aggregation attention unit plays a key role in improving the Transformer model. Deepening the network structure through group linear transformation helps the model capture important information in the input sequence. Further analysis shows that using the lightweight FFN layer to improve the original FFN layer has different effects on the performance of the model. When using the multi-head attention mechanism, the accuracy of the model decreases from 97.84% to 97.34%, and when using the local aggregation attention unit, it increases from 98.44% to 99.34%. This shows that the collaboration between the lightweight FFN and the local aggregation attention unit helps to extract highly resolvable information and reduce the adverse effects of redundant features. The lightweight FFN can further enhance the generalization performance of the model by improving the activation function and random inactivation layer, and can effectively extract features despite the reduced dimension. However, the synergy between the multi-head attention mechanism and the lightweight FFN is poor, which further demonstrates that the local aggregated attention unit has more advantages than the multi-head attention mechanism. The ablation experiment provides an important basis for optimizing the model structure.

[0127] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A UAV network intrusion detection method based on lightweight Transformer, characterized in that: The following steps are involved: By deploying traffic monitoring equipment or software on key network nodes, network traffic data can be captured in real time to obtain network traffic sequences; Performing standardization processing on the network traffic sequence; Using rotational position coding to add position information to the standardized network traffic sequence; Using a local aggregation attention unit to perform local feature aggregation and global feature enhancement on the encoded network traffic sequence; A lightweight feedforward neural network is used to extract the enhanced features, and after being processed by a stacked multi-layer lightweight Transformer encoder, a SoftMax classifier is used to output the detection results and detect the attack type.

2. According to claim 1, the UAV network intrusion detection method based on lightweight Transformer is characterized in that: The step of standardizing the network traffic sequence includes: Normalize the numerical features so that they are distributed on the same scale; Label encoding is performed on categorical features to convert them into numerical features.

3. The UAV network intrusion detection method based on lightweight Transformer according to claim 2 is characterized in that: The calculation method of the rotational position encoding is: Among them, the rotation matrix X is the network traffic sequence data; m is the position of the data in the sequence; W is a trainable parameter matrix.

4. The UAV network intrusion detection method based on lightweight Transformer according to claim 3 is characterized in that: The local aggregation attention unit includes grouped linear transformation, nonlinear activation function, layer normalization, self-attention mechanism and residual connection; the local aggregation attention unit is used to reduce the dimension of the input network traffic sequence, perform feature aggregation on local information through grouped linear transformation, and then use the self-attention mechanism to perform global perception on the aggregated low-dimensional features, increase the dimension of the vector after processing by the self-attention mechanism, and finally add residual connection to obtain the enhanced feature F.

5. The UAV network intrusion detection method based on lightweight Transformer according to claim 4 is characterized in that: In the local feature aggregation stage, it is assumed that there are L layers of grouped linear transformations; The first [L / 2] layers are used for dimensionality increase, and the remaining L-[L / 2] layers are used for dimensionality reduction. The specific process of calculating the number of groups of the linear transformation of each layer is as follows: Where: n l The number of groups for the first layer grouping linear transformation; n max The maximum number of groups for grouped linear transformation sets the upper limit of the number of groups.

6. The UAV network intrusion detection method based on lightweight Transformer according to claim 5 is characterized in that: The residual concatenation operation is used in each layer of grouped linear transformation. The calculation formula of each layer of grouped linear transformation is: In the formula, G(·) represents grouped linear transformation; M(·) represents residual concatenation, nonlinear activation function GELU and layer normalization operations; W l Represents the learnable parameter matrix of the first layer of the grouped linear transformation; b l represents the bias vector; g l The number of groups to group the linear transformation for layer 1.

7. The UAV network intrusion detection method based on lightweight Transformer according to claim 5 is characterized in that: The self-attention mechanism calculates the correlation of sequence data through query vector and key vector to obtain the attention matrix, and then calculates the globally enhanced features with the value matrix as follows: In the formula, the query vector Q = YW Q , key vector K = YW K Sum vector V = YW V , where W Q , W K , W V is the learnable parameter matrix; d k is the embedding dimension, softmax is the activation function; Enhanced feature F = I + W s A Where: W s It is the parameter matrix for increasing the dimension of the output of the self-attention mechanism; I is the residual feature of the input.

8. The UAV network intrusion detection method based on lightweight Transformer according to claim 7 is characterized in that: The lightweight feedforward neural network includes a fully connected layer, a nonlinear activation function, and a residual connection; assuming that the dimension of the input feature F is d m , the lightweight feedforward neural network first reduces the dimension to Then upgrade to d m ; The calculation process of lightweight feedforward neural network for feature extraction is: Oh m =Feedforward(F)=F+((Relu(FW1+b1))W2+b2) Where W1, b1 are the parameter matrix and bias vector during dimensionality reduction, W2, b2 are the parameter matrix and bias vector during dimensionality reduction, and Relu(·) is a nonlinear activation function.

9. The UAV network intrusion detection method based on lightweight Transformer according to claim 8 is characterized in that: The SoftMax classifier outputs the detection results as follows: In the formula, O M is the output of the Mth layer lightweight Transformer encoder, exp(·) represents the exponential function of e, and K represents the total number of categories.

10. The UAV network intrusion detection method based on lightweight Transformer according to claim 1 is characterized in that: The lightweight Transformer encoder uses an adaptive model scaling mechanism.

Images