Power grid confrontation and defense method and system based on robust-lightweight Transform model optimization

Abstract

The invention discloses a power grid confrontation and defense method and system based on robust-lightweight Transform model optimization, and the method comprises the steps: taking hyper-parameters, such as the number of attention, the type of an activation function, the number of layers, the learning rate and the discard rate, in Transform-Ender as decision variables; a model with good multi-classification detection benign sample F1 score performance in the optimization process is used as an adversarial attack replacement model, an adversarial sample is generated based on a projection gradient descent attack, the number of model parameters after adversarial training is introduced, and minimization of multi-classification detection benign sample 1-F1 scores and an adversarial attack success rate is used as optimization targets. And designing a multi-target discrete optimization technology based on an elite individual external archiving mechanism to carry out offline optimization iteration, and carrying out online deployment on the obtained robust-lightweight Transform model so as to realize confrontation and defense of the intelligent power grid intrusion detection system.

Description

Technical Field

[0001] The present invention relates to the technical field of intrusion detection and deep learning model anti-attack robustness improvement in the field of smart grid information security, and in particular to a power grid anti-attack defense method and system based on robust-lightweight Transformer model optimization. Background Art

[0002] Modern power grids, incorporating advanced control, sensor measurement, information technology, and computer science, effectively meet the needs of real-time state estimation, flow analysis and control, and two-way information exchange. However, their heavy reliance on communication technologies makes them vulnerable to network intrusion attacks. Consequently, many researchers have turned their attention to the field of power grid intrusion detection, particularly those based on deep learning (DL). As data-driven, high-ceiling intelligent methods, DL approaches offer excellent data understanding and feature extraction capabilities, even when faced with the massive amounts of data generated by large-scale systems. However, existing research on intrusion detection methods has largely focused on optimizing detection performance in specific application scenarios, while neglecting the model's inherent vulnerability to malicious adversarial attacks. Systematic research is urgently needed to improve the model's robustness against adversarial attacks while maintaining a stable detection performance. Beyond the model's normal detection performance and robustness to adversarial attacks, further consideration is needed for its lightweight design. How to automatically design a multi-target robust multi-classification intrusion detection model to achieve a compromise optimization of multiple performance indicators such as smart grid intrusion detection performance indicators, model lightweight and robust performance is one of the key technical problems that need to be solved urgently in the current smart grid information security field.

[0003] Although researchers have been able to manually design intrusion detection systems based on deep learning to defend against power grid attacks, the accuracy of these systems in intrusion detection tasks still needs to be further improved. Furthermore, the inherent vulnerabilities of DL models are easily exploited by attackers. By injecting small amounts of specific perturbations into the original data, attackers can construct adversarial attack data, which can dramatically degrade the performance of previously excellent intrusion detection models. Therefore, when faced with adversarial attack data designed to exploit inherent flaws in intrusion detection systems, traditional intrusion detection systems struggle to distinguish subtle differences between adversarial attack data and normal traffic data. They are prone to misclassifying adversarial attack data as normal traffic data, leading to intrusion detection failures, model crashes, and other serious security issues. Therefore, the development of a robust and lightweight Transformer model for power grids is urgently needed to address the challenges of intrusion detection systems' insufficient ability to identify adversarial attack data, low accuracy in distinguishing between normal and malicious data, and slow updating of knowledge to defend against adversarial attack data. Summary of the Invention

[0004] The purpose of the present invention is to address the deficiencies of the existing technology and provide a power grid countermeasure defense method and system based on robust-lightweight Transformer model optimization.

[0005] The objective of the present invention is achieved through the following technical solutions: In a first aspect, an embodiment of the present invention provides a power grid countermeasure defense method based on robust-lightweight Transformer model optimization, comprising the following steps:

[0006] S1: Building a training dataset D for robust-lightweight Transformer model optimization based on smart grid cyber-physical fusion system data tr With the test data set D te ;

[0007] S2: Set multiple first parameters, construct the optimization target of the robust-lightweight Transformer model, and integer encode the model architecture and hyperparameters to be optimized according to the preset first parameters to generate the initial population P gen ={p i}, i=1,2,…,NP, where p i represents the i-th individual, NP represents the population size;

[0008] S3: Evaluate population P genThe model parameter quantity and F1 score of all individuals in the test are evaluated, and whether the F1 score is greater than the adversarial training threshold is determined. If not, the individual adversarial training is interrupted and the pre-evaluated model parameter quantity and F1 score are saved, and the adversarial attack success rate is set to 100%. If so, step S4 is entered for adversarial training and testing;

[0009] S4: Individual p i The characterized architecture and hyperparameters are decoded into the Transformer model as a replacement model for adversarial attack, and the adversarial attack training dataset D is generated based on the replacement model and projected gradient descent attack. adv_tr And the adversarial attack test dataset D adv_te , then based on the training data set D tr With adversarial attack training dataset D adv_tr For individual p i After the training is completed, the adversarial training is performed based on the test data set D te Evaluate the F1 score of individual benign sample detection based on the adversarial attack test dataset D adv_te Evaluate the success rate of adversarial attacks;

[0010] S5: P gen Perform non-dominated sorting and crowding calculation on all individuals in to generate the offspring population Q gen , and record the number of iterations gen;

[0011] S6: For population P gen With Q gen Perform non-dominated sorting and crowding calculation on all individuals in to obtain the new parent population P. gen , save all individuals in the current Pareto front to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update the archive using the update method;

[0012] S7: Determine whether the current number of iterations has reached the maximum number of iterations Gen. If so, save the external archive solution set Ach gen If not, set the current iteration number gen = gen + 1 and return to step S3 to continue searching and evaluating the robust-lightweight Transformer model;

[0013] S8: Screen a robust-lightweight Transformer model that takes into account the F1 score of benign samples in multi-classification detection, the success rate of adversarial attacks, and the minimum number of model parameters, and use it for online deployment of power grid intrusion detection.

[0014] Furthermore, the step S1 specifically includes:

[0015] The smart grid cyber-physical fusion system data is divided into a training dataset D for robust-lightweight Transformer model search. tr With the test data set D te ; and according to the training data set D tr The data distribution of the training data set D tr With the test data set D te Perform Z-Score standardization processing, where the calculation process of Z-Score standardization is shown as follows:

[0016]

[0017] Where x′ α,β represents the value of the βth feature of the αth data sample after normalization, x α,β represents the value of the βth feature of the αth data sample before standardization, μ represents the mean of the βth feature, and σ represents the standard deviation of the βth feature; then the training data set D after Z-Score standardization is completed tr And the test dataset D te Optimal architecture search and hyperparameter optimization for robust-lightweight Transformer models.

[0018] Furthermore, the first parameters include the population size NP, the maximum number of iterations Gen, the maximum number of external archived individuals A max , cross factor Cr, mutation factor F, adversarial training threshold Gt, PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ and PGD adversarial attack iteration number ζ; Among them, the cross factor Cr includes the maximum and minimum cross factors Cr max With Cr min , the variation factor F includes the maximum and minimum variation factors F max With F min ;

[0019] The optimization goal of the robust-lightweight Transformer model is:

[0020]

[0021] Where p represents the encoding individual of the robust-lightweight Transformer model to be optimized, the optimization target obj1(p) is the number of parameters of the robust-lightweight Transformer model Para-num, the optimization target obj2(p) is the multi-classification benign sample intrusion detection performance index 1-F1 score F1-Score, and the optimization target obj3(p) is the PGD adversarial attack success rate AASR; among them, the optimization target obj1(p) is used to measure the number of parameters of the searched robust-lightweight Transformer model; the optimization target obj2(p) refers to the test dataset D constructed using the smart grid cyber-physical fusion data during the robust-lightweight Transformer model search process. te , the 1-F1 score indicator for multi-classification intrusion detection performance evaluation is calculated as shown in formula (3):

[0022]

[0023] Where TP represents the number of samples predicted as true normal in benign samples, TN represents the number of samples predicted as true abnormal in benign samples, FP represents the number of samples predicted as false normal in benign samples, and FN represents the number of samples predicted as false abnormal in benign samples. Benign samples refer to samples that have not been generated by adversarial samples. The optimization target obj3(p) refers to the F1 score of the robust-lightweight Transformer model for benign sample classification detection that is higher than the adversarial training threshold Gt, based on the training dataset D tr With the test data set D te Some benign samples in the robust-lightweight Transformer replacement model generate the PGD adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te Then, based on the training data set D tr With adversarial attack training dataset D adv_tr Train a robust-lightweight Transformer model and use the adversarial attack test dataset D adv_te The PGD adversarial attack success rate obtained when the robust-lightweight Transformer model is subjected to adversarial attacks. The parameters of the robust-lightweight Transformer replacement model are exactly the same as those of the robust-lightweight Transformer model. The specific calculation formula of the PGD adversarial attack success rate AASR is shown in formula (4):

[0024]

[0025] In the formula, the total number of samples refers to the adversarial attack test dataset D adv_teThe total number of samples in the attack dataset D is the number of successful attack samples. adv_te The total number of samples misclassified by the adversarial attack samples in ;

[0026] The model architecture to be optimized and the hyperparameters are integer-encoded according to the preset first parameter to generate the initial population P gen , specifically including:

[0027] Randomly generate a population P containing NP individuals gen , that is, P gen ={p1,p2,…,p NP}, 0≤gen≤Gen, where gen represents the current number of iterations, and the specific expression of each individual in the population is: p i ={p i,1 ,p i,2 ,…,p i,j ,…,p i,D}, where 1≤i≤NP, 1≤j≤D; p i,j represents the jth dimension of the i-th individual; D represents the dimension of the individual encoding; individual p i The specific code is p i ={Ops,Bts,Lrt,Nhd,Fnet,Dpt,Ativ,Lynm,Numl}, where Ops represents the optimizer during Transformer model training; Bts represents the data batch size during Transformer model training; Lrt represents the learning rate during Transformer model training; Nhd represents the number of attention heads in the robust-lightweight Transformer model encoder; Fnet represents the dimension of the feedforward neural network in the robust-lightweight Transformer model encoder; Dpt represents the dropout rate during Transformer model training; Ativ represents the activation function type in the robust-lightweight Transformer model encoder; Lynm represents the layer normalization eps value in the robust-lightweight Transformer model encoder, where the eps value is a smoothing term to prevent the denominator from being zero; Numl represents the number of sub-encoders in the robust-lightweight Transformer model encoder;

[0028] In the above real number discrete coding, the encoding method of each variable is as follows:

[0029] Ops = Randint(1,6), where Randint represents a random integer selection function. The specific values ​​of Ops are: Ops = 1 represents an adaptive moment estimation optimizer, Ops = 2 represents a stochastic gradient descent optimizer, Ops = 3 represents a root mean square propagation optimizer, Ops = 4 represents an adaptive gradient-based optimizer, Ops = 5 represents an asynchronous stochastic gradient descent-based optimizer, and Ops = 6 represents a corrected adaptive moment estimation optimizer; Bts = Rand(Bts1, Bts2, …, Bts k ), where Rand is a random selection function, and the specific value of Bts is a preset k number of Bts1, Bts2,…, Bts k The data batch size encoding during the training process of a Transformer model in , where Bts1 represents the encoding of the preset first batch size, Bts2 represents the encoding of the preset second batch size, and Bts k Indicates the encoding of the preset k-th batch size; Lrt=Rand(Lrt1, Lrt2,…, Lrt l ), where the specific value of Lrt is a preset l Lrt1, Lrt2, ..., Lrt l The learning rate encoding during the training process of a Transformer model in , where Lrt1 represents the encoding of the preset first learning rate, Lrt2 represents the encoding of the preset second learning rate, and Lrt l Indicates the encoding of the preset lth learning rate; Nhd=Rand(Nhd1,Nhd2,…,Nhd t ), where the specific value of Nhd is a preset t number of Nhd1, Nhd2, ..., Nhd t The number of attention heads of a robust-lightweight Transformer model encoder in , where Nhd1 represents the encoding of the preset first number of attention heads of the encoder, Nhd2 represents the encoding of the preset second number of attention heads of the encoder, and Nhd t Represents the encoding of the preset number of attention heads of the t-th encoder; Fnet = Randint(Fnet min ,Fnet max ), where Fnet min With Fnet max Indicates the minimum and maximum dimensions of the robust-lightweight Transformer model encoder feedforward neural network. The specific value of Fnet is Fnet min With Fnet max An integer between; Dpt=Rand(Dpt1,Dpt2,…,Dpt e), where the specific value of Dpt is the preset e Dpt1, Dpt2, ..., Dpt e A certain drop rate in, where Dpt1 represents the encoding of the preset first drop rate, Dpt2 represents the encoding of the preset second drop rate, and Dpt e Indicates the encoding of the preset e-th drop rate; Ativ = Randint(1,2), where the specific value of Ativ is: Ativ = 1 indicates that the type of activation function in the robust-lightweight Transformer model encoder is a linear rectification function, and Ativ = 2 indicates that the type of activation function in the robust-lightweight Transformer model encoder is a Gaussian error linear unit; Lynm = Rand(Lynm1, Lynm2,…, Lynm r ), where the specific value of Lynm is the preset r Lynm1, Lynm2, ..., Lynm r The layer normalized EPS smoothing term in a robust-lightweight Transformer model encoder in , where Lynm1 represents the encoding of the preset first layer normalized EPS smoothing term, Lynm2 represents the encoding of the preset second layer normalized EPS smoothing term, Lynm r Represents the encoding of the preset r-th layer normalized eps smoothing term; Numl=Randint(Numl min ,Numl max ), where Numl min With Numl max Indicates the minimum and maximum number of sub-encoders in the robust-lightweight Transformer model encoder. The specific value of Numl is Numl min With Numl max An integer between

[0030] The mutation factor S is set as a dynamic mutation factor that changes dynamically with the number of iterations. Its calculation formula is shown in formula (5):

[0031]

[0032] Where S max With S min Respectively represent the preset maximum and minimum variation factors;

[0033] In the coding crossover process, the crossover factor Cr changes with the number of iterations, and its change formula is shown in formula (6):

[0034]

[0035] Where, Cr max With Crmin Respectively represent the preset maximum and minimum cross factors;

[0036] Finally, according to the above parameters, the range of population P is selected. gen All individuals p in i , 1≤i≤NP, initialize.

[0037] Furthermore, the S3 specifically includes the following sub-steps:

[0038] S31: Evaluate the population P according to the optimization goal of the robust-lightweight Transformer model gen The model parameters of all individuals in the optimization target obj1(p i ), and F1 score, where the F1 score is calculated according to formula (7);

[0039]

[0040] S32: Determine the population P gen Is the F1 score of the individual greater than the adversarial training threshold Gt? If not, interrupt the individual adversarial training and save the pre-evaluated number of model parameters and F1 score, and set the adversarial attack success rate to 100%; if so, enter step S4 for subsequent adversarial training and optimization search.

[0041] Furthermore, the step S4 specifically includes the following sub-steps:

[0042] S41: Generate population P according to step S3 gen The individual p in which the F1 score is greater than the adversarial training threshold Gt i The replacement model is used to generate the adversarial attack training dataset D based on the replacement model and PGD adversarial attack. adv_tr And the adversarial attack test dataset D adv_te ; Among them, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te The specific generation process is as follows: First, from the training data set D tr With the test data set D te Randomly select m categories of benign samples from the M categories of benign sample data to construct the initial adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te , where m categories are randomly selected, the maximum is m max categories, the minimum is m min categories, 1≤m min ≤m≤m max ≤M; Secondly, based on the above replacement model, the adversarial attack training dataset D adv_trAnd the adversarial attack test dataset D adv_te All samples in are subjected to adversarial attacks one by one to generate corresponding adversarial samples;

[0043] S42: Based on the training data set D tr Compared with the above adversarial attack training dataset D adv_tr For individual p i The corresponding robust-lightweight Transformer model is trained against it. After the training is completed, the te Re-evaluate the F1 score of individual benign sample detection, that is, optimize the target obj2(p i ), and based on the adversarial attack test dataset D adv_te Evaluate the success rate of the adversarial attack, that is, optimize the target obj3(p i ).

[0044] Furthermore, in step S41, the adversarial sample is generated by the following steps:

[0045] S41.1: From the adversarial attack training dataset D adv_tr Select the original sample Sb adv_tr,tx , where Sb adv_tr,tx Denotes the adversarial attack training dataset D adv_tr The txth original sample in , 1≤tx≤Tr, Tr represents the adversarial attack training dataset D adv_tr ; and initialize the preset PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ, and the maximum number of iterations ζ of PGD adversarial attack;

[0046] S41.2: According to formula (8), randomly initialize the adversarial sample and mark the generated initialized adversarial sample as Sadv:

[0047] Sadv=Sb+Uniform(-ε,ε) (8)

[0048] Where Sb represents the original sample; Uniform(-ε,ε) represents uniformly sampling a random noise with the same shape as the original sample in the interval [-ε,ε]; adv_tr When generating the initial adversarial sample, Sb=Sb adv_tr,tx , 1≤tx≤Tr; in the adversarial attack test dataset D adv_te When generating the initial adversarial sample, Sb=Sb adv_te,ty , 1≤ty≤Te, where ty represents the adversarial attack test dataset D adv_te The tyth original sample in, Te represents the adversarial attack test dataset D adv_te The number of all samples in ;

[0049] S41.3: According to formula (9), the range of the adversarial sample Sadv is constrained:

[0050]

[0051] S41.4: Mark the current iteration number of the PGD adversarial attack as iter. For each iteration from iter = 1 to iteration number ζ, perform iterative optimization on the adversarial example Sadv according to formula (10):

[0052]

[0053] Where iter represents the current iteration number of PGD attack, Sadv iter and Sadv iter+1 denote the iter-th and (iter+1)-th adversarial samples respectively; Ran denotes the set of random perturbation constraints, Sb+Ran denotes the perturbation range of all adversarial samples after the original sample Sb is added with the perturbation ε; Π Sb+Ran According to formula (9), the generated adversarial sample Sadv iter Perform range constraints; Lable represents the true label of the sample; L(θ, Sadv iter , Lable) represents the loss function, θ is the network parameter of the replacement model; sign represents the gradient calculation symbol;

[0054] S41.5: The adversarial attack training dataset D adv_tr The original sample Sb adv_tr,tx Replace it with the adversarial sample Sadv generated after reaching the maximum number of PGD iterations ζ;

[0055] S41.6: Repeat the adversarial sample generation process described in steps S41.1 to S41.5 until the adversarial attack training dataset D adv_tr All original samples Sb adv_tr,tx , 1≤tx≤Tr, are all replaced by the corresponding tx-th adversarial sample;

[0056] S41.7: From the adversarial attack test dataset D adv_te Select the original sample Sb adv_te,ty , among which Sb adv_te,ty Denotes the adversarial attack training dataset D adv_tr The ty-th original sample in , 1≤ty≤Te, Te represents the adversarial attack test dataset D adv_te; and initialize the preset PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ and the maximum number of iterations ζ of PGD adversarial attack; repeat the adversarial sample generation process described in steps S41.1 to S41.5 above until the adversarial attack test dataset D adv_te All original samples Sb adv_te,ty , 1≤ty≤Te, are all replaced by the corresponding ty-th adversarial examples.

[0057] Furthermore, the step S5 specifically includes the following sub-steps:

[0058] S51: First, for the population P gen Each individual p i Assign two key parameters: the dominant individual p i The number of individuals ind_num p and by individual p i Dominant individual set Slu p ; Secondly, set rto = 1, ind_num p = 0 individuals are included in the frontier Ft rto In; Again, on the frontier Ft rto All individuals in Slu p Traverse and set the ind_num of each individual p Subtract 1; then, set rto = rto + 1, and then ind_num p = 0 is included in the frontier Ft rto Finally, repeat the above steps until the population P gen All individuals p in i All are included in a certain frontier Ft rto middle;

[0059] S52: After completing the non-dominated sorting, the crowding degree of each individual in the population is calculated. The specific steps of the crowding degree calculation are: first, take all individuals in a single frontier and sort them from small to large according to the fitness value of a certain optimization goal; second, take the largest fitness value as the fit o,max , the minimum fitness value is taken as fit o,min , and fit the two extreme values o,max and fit o,min The crowding distance between them is set to infinity, denoted as inf; again, the crowding distance between each individual and the nearest neighbor is calculated on the optimization target, and the two extreme values ​​of the crowding distance on the corresponding optimization target are fit o,max and fit o,minMin-max normalization is performed between the two sets of parameters, where the min-max normalization calculation formula is shown in formula (12) and the crowding calculation formula is shown in formula (13); then, all optimization objectives are traversed and the standardized crowding distances of all objectives are added together; finally, the next Pareto front is entered until the crowding distances of all individuals are calculated;

[0060]

[0061] Where, Represents the normalized fitness value of the i-th individual on the optimization target o; represents the fitness value of the i-th individual on the optimization target o, fit o,max and fit o,min Respectively represent the maximum and minimum values ​​of the congestion distance on the corresponding optimization target; dt i represents the crowding distance of the i-th individual; t represents the total number of optimization targets; Represents the fitness value of the i+1th individual on the optimization target o; Represents the fitness value of the i-1th individual on the optimization target o;

[0062] S53: Offspring population Q gen The specific steps of generating are: randomly select population P gen There are three groups of individuals with two individuals in each group. A binary tournament selection is performed on the undominated individuals with a large crowding degree in each group, and the selected individuals are used as parents, denoted as s1, s2 and s3. Among them, the undominated individuals are preferred. If the number of individuals dominated is the same, the individuals with a large crowding degree are considered. The mutation operation is performed on the parents s1, s2 and s3 as the basic individuals. The mutation generation method of the offspring individuals q1 and q2 is shown in formula (14) and formula (15):

[0063] q 1,d =s 1,d +S×(s 2,d -s 3,d )(14)

[0064] q 2,d =s 2,d +S×(s 3,d -s 1,d )(15)

[0065] Where S represents the variation factor that changes with the number of iterations, d represents the dth dimension of the individual code, 1≤d≤D; q 1,d and q 2,d Represents the dth dimension of offspring individuals q1 and q2 respectively; s 1,d 、s 2,d and s3,d They represent the dth dimension of the parent individuals s1, s2, and s3 respectively. After the generation of offspring individual mutations is completed, the generated offspring individuals are subjected to a crossover operation. The specific operation is shown in formulas (16) and (17):

[0066]

[0067] Where rand 1,d and rand 2,d Represents a random decimal generated between [0,1], s 1,d and s 2,d Represent the dth dimension of parent individuals s1 and s2 respectively, and Cr represents the crossover factor that changes with the number of iterations; after completing the crossover operation, let q1 = q′1, q2 = q′2; in the process of mutation and crossover, if the generated new code exceeds the preset range, the value exceeding the preset range of the code will be truncated, and the value exceeding the upper limit of the range will be truncated to the maximum value preset for the dth dimension code, and the value exceeding the lower limit of the range will be truncated to the minimum value preset for the dth dimension code, so that the generated offspring individual code is always within the preset range; then repeat the above operation until the offspring population Q gen The population size reaches NP; finally, record the current number of iterations gen.

[0068] Furthermore, in step S6, the external archive solution set Ach gen The update specifically includes the following steps:

[0069] S61: First, the parent population P gen and the offspring population Q gen Merge into a new population, where there are 2×NP individuals in the new population; secondly, gen With Q gen Perform the non-dominated sorting and crowding calculation described in step S6 on all individuals in the parent population, and filter the first NP individuals to obtain a new parent population P. gen ; Again, the current parent population P gen All individuals in the Pareto front are saved to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update the archive using the update method;

[0070] S62: External archive solution Ach gen The update method is as follows: First, in the current population P gen All individuals in the Pareto front are saved to Ach gen After the middle, to Ach gen Perform non-dominated sorting and crowding calculation on all individuals in Ach gen To update, if Achgen The number of individuals in the archive exceeds the preset maximum number of external archive individuals A max , then delete Ach gen Individuals with more dominated solutions and lower crowding degree go directly to Ach gen The number of individuals in the archive is less than the maximum number of individuals in the external archive A max .

[0071] Furthermore, the S8 specifically includes:

[0072] Unpack external archive Ach gen All non-dominated individuals in the Pareto front solution are taken as the Pareto front solution, and the optimal compromise individual is selected from the Pareto front solution. The model architecture and parameters encoded by the individual are used as the architecture and parameters of the optimal robust-lightweight Transformer model, thereby screening out a robust-lightweight Transformer model that takes into account the F1 score of multi-classification detection of benign samples, the success rate of adversarial attacks, and the minimization of the number of model parameters. It is then used for online deployment of power grid intrusion detection to achieve adversarial defense of the smart grid intrusion detection system.

[0073] A second aspect of an embodiment of the present invention provides a power grid countermeasure defense system based on robust and lightweight Transformer model optimization, which is equipped with the above-mentioned power grid countermeasure defense method based on robust and lightweight Transformer model optimization. The system includes:

[0074] Data preprocessing module, used to construct a training dataset D for robust-lightweight Transformer model optimization based on smart grid cyber-physical fusion system data tr With the test data set D te ;

[0075] The population generation module is used to set multiple first parameters, construct the optimization target of the robust-lightweight Transformer model, and integer encode the model architecture and hyperparameters to be optimized according to the preset first parameters to generate the initial population P gen ;

[0076] Adversarial training judgment module, used to evaluate the population P gen The model parameter amount and F1 score of all individuals in the training set are calculated, and it is determined whether the F1 score is greater than the adversarial training threshold. If not, the individual adversarial training is interrupted and the pre-evaluated model parameter amount and F1 score are saved, and the adversarial attack success rate is set to 100%. If so, the adversarial training and evaluation module 40 is entered for adversarial training and testing.

[0077] Adversarial training and evaluation module is used to transform individual p iThe characterized architecture and hyperparameters are decoded into the Transformer model as a replacement model for adversarial attack, and the adversarial attack training dataset D is generated based on the replacement model and projected gradient descent attack. adv_tr And the adversarial attack test dataset D adv_te , then based on the training data set D tr With adversarial attack training dataset D adv_tr For individual p i After the training is completed, the adversarial training is performed based on the test data set D te Evaluate the F1 score of individual benign sample detection based on the adversarial attack test dataset D adv_te Evaluate the success rate of adversarial attacks;

[0078] The offspring population generation module is used to generate P gen Perform non-dominated sorting and crowding calculation on all individuals in to generate the offspring population Q gen , and record the number of iterations gen;

[0079] External archive update module for population P gen With Q gen Perform non-dominated sorting and crowding calculation on all individuals in to obtain the new parent population P. gen , save all individuals in the current Pareto front to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update the archive using the update method;

[0080] The external archive saving module is used to save the external archive solution set Ach when the current iteration number reaches the maximum iteration number Gen gen A robust-lightweight Transformer model represented by all individuals in ;

[0081] The online deployment module is used to screen robust and lightweight Transformer models that take into account the F1 score of benign samples in multi-classification detection, the success rate of adversarial attacks, and the minimization of the number of model parameters, and use them for online deployment of power grid intrusion detection.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] (1) The present invention proposes an adversarial training method based on Projected Gradient Descent (PGD) adversarial attack, which uses a replacement model to generate adversarial samples, conducts adversarial attacks on the robust-lightweight Transformer model, and conducts adversarial training on the robust-lightweight Transformer model based on the generated adversarial samples, so as to achieve the purpose of improving the adversarial robustness of the Transformer model. The defense method described in the present invention is driven by actual power grid data and can effectively retain the classification accuracy of the robust-lightweight Transformer model for benign samples. While improving the adversarial robustness, it takes into account the accuracy of model lightweighting and smart grid multi-classification intrusion detection.

[0084] (2) The present invention takes the number of attention heads, activation function type, number of layers in the Encoder, and hyperparameters such as learning rate, batch size and dropout rate in the Transformer-Encoder as decision variables, and adopts an integer coding mechanism for encoding. The minimization of the model parameter amount (Para-num), the 1-F1 score (1-F1-Score) of benign samples in multi-classification detection and the adversarial attack success rate (AASR) are taken as optimization objectives. A binary tournament selection, mutation and crossover mechanism is designed. Elite individuals are screened through non-dominated sorting and congestion calculation. An external archiving mechanism for elite individuals is introduced to save elite individuals. Through offline multi-objective automatic optimization technology, a robust-lightweight Transformer model that takes into account model lightweight, high performance of multi-classification intrusion detection and strong adversarial robustness can be obtained. The obtained robust-lightweight Transformer model is deployed online in the smart grid intrusion detection system, thereby realizing effective defense of the smart grid intrusion detection system against adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a flow chart of the power grid countermeasure defense method based on robust-lightweight Transformer model optimization of the present invention;

[0086] Figure 2 is an example diagram of individual encoding and decoding of a typical robust-lightweight Transformer model generated in step S2 of the present invention into a robust-lightweight Transformer model;

[0087] Figure 3 This is an example diagram of adversarial sample generation in step S4 of the multi-category non-targeted adversarial attack example of the present invention;

[0088] Figure 4This is an example diagram of the binary tournament selection, mutation, and crossover operations of the robust-lightweight Transformer model individuals in step S5 of the present invention;

[0089] Figure 5 The Pareto front graph obtained in the single-category adversarial attack after iterative optimization search in the present invention;

[0090] Figure 6 The Pareto front graph obtained in the multi-category adversarial attack after iterative optimization search in the present invention;

[0091] Figure 7 This is a comparison chart of the performance indicators of the optimal compromise robust-lightweight Transformer model before and after adversarial training, which was screened in the single-category adversarial attack Pareto front diagram after iterative optimization search in the present invention;

[0092] Figure 8 This is a comparison chart of the performance indicators of the optimal compromise robust-lightweight Transformer model before and after adversarial training, which was screened in the multi-category adversarial attack Pareto front diagram after iterative optimization search in the present invention;

[0093] Figure 9 This is a structural diagram of the power grid countermeasure defense system based on robust-lightweight Transformer model optimization of the present invention. DETAILED DESCRIPTION

[0094] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0095] See also Figure 1 The power grid countermeasure defense method based on robust-lightweight Transformer model optimization of the present invention specifically includes the following steps:

[0096] S1: Building a training dataset D for robust-lightweight Transformer model optimization based on smart grid cyber-physical fusion system data tr With the test data set D te .

[0097] It should be noted that, in this embodiment, the training data set D tr With the test data set D teThe distribution ratio is 8:2. Of course, you can also divide each data set according to other ratios according to the actual needs of model training and testing, such as 7:3. After completing the division of each data set, according to the training data set D tr The data distribution of the training data set D tr With the test data set D te Perform Z-Score standardization processing, where the calculation formula of Z-Score standardization is shown as follows:

[0098]

[0099] Where x′ α,β represents the value of the βth feature of the αth data sample after normalization, x α,β represents the value of the βth feature of the αth data sample before standardization, μ represents the mean of the βth feature, and σ represents the standard deviation of the βth feature. tr And the test dataset D te Optimal architecture search and hyperparameter optimization for robust-lightweight Transformer models.

[0100] In this embodiment, two adversarial attack defense examples are performed, namely a single-category non-targeted adversarial attack defense example and a multi-category non-targeted adversarial attack defense example. In addition, the total number of samples in the smart grid cyber-physical fusion system dataset is 4618, that is, there are 4618 data samples in total, of which 3694 samples are in the training set and 924 samples are in the test set. The total number of sample features is 128, that is, each data sample has 128 features. The samples in the dataset exist in 37 actual smart grid scenarios, that is, M = 37.

[0101] S2: Set multiple first parameters, construct the optimization target of the robust-lightweight Transformer model, and integer encode the model architecture and hyperparameters to be optimized according to the preset first parameters to generate the initial population P gen ={p i}, i=1,2,…,NP, where p i represents the i-th individual, and NP represents the population size.

[0102] Furthermore, the first parameters include the population size NP, the maximum number of iterations Gen, the maximum number of external archived individuals A max , cross factor Cr, mutation factor F, adversarial training threshold Gt, PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ and PGD adversarial attack iteration number ζ; Among them, the cross factor Cr includes the maximum and minimum cross factors Cr max With Cr min, the variation factor F includes the maximum and minimum variation factors F max With F min .

[0103] In this embodiment, NP=20, Gen=20, A max =20、Cr max =0.9、Cr min =0.2, F max =0.9, F min =0.5, Gt=0.8, ε=0.2, δ=0.002 and ζ=30; it should be noted that the specific values ​​of the above first parameters can be set according to actual conditions.

[0104] Furthermore, the optimization objective of the robust-lightweight Transformer model is shown in formula (2):

[0105]

[0106] Where p represents the robust-lightweight Transformer model encoding individual to be optimized, the optimization target obj1(p) is the number of parameters (Para-num) of the robust-lightweight Transformer model, the optimization target obj2(p) is the 1-F1 score (1-F1-Score), and the optimization target obj3(p) is the PGD adversarial attack success rate (AASR). The optimization target obj1(p) is used to measure the number of parameters of the searched robust-lightweight Transformer model; the optimization target obj2(p) refers to the test dataset D constructed using the smart grid cyber-physical fusion data during the robust-lightweight Transformer model search process. te , the 1-F1 score indicator for multi-classification intrusion detection performance evaluation is calculated as shown in formula (3):

[0107]

[0108] Where TP represents the number of samples predicted as true normal in benign samples, TN represents the number of samples predicted as true abnormal in benign samples, FP represents the number of samples predicted as false normal in benign samples, and FN represents the number of samples predicted as false abnormal in benign samples. Benign samples refer to samples that have not been generated by adversarial samples. The optimization target obj3(p) refers to the F1 score of the robust-lightweight Transformer model for benign sample classification detection that is higher than the adversarial training threshold Gt, based on the training dataset D tr With the test data set Dte Some benign samples in the robust-lightweight Transformer replacement model generate the PGD adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te Then, based on the training data set D tr With adversarial attack training dataset D adv_tr Train a robust-lightweight Transformer model and use the adversarial attack test dataset D adv_te The PGD adversarial attack success rate is obtained when the robust-lightweight Transformer model is subjected to adversarial attacks. The parameters of the robust-lightweight Transformer replacement model are exactly the same as those of the robust-lightweight Transformer model. The specific calculation formula of the PGD adversarial attack success rate is shown in formula (4):

[0109]

[0110] In the formula, the total number of samples refers to the adversarial attack test dataset D adv_te The total number of samples in the attack dataset D is the number of successful attack samples. adv_te The total number of samples misclassified by the adversarial attack samples in .

[0111] In this embodiment, the adversarial attack samples generated by the single-category non-targeted adversarial attack defense example come from the first category of samples, namely, the short-circuit fault data samples occurring at 10% to 19% positions in bus L1; the multi-category non-targeted adversarial attack defense example has a total of three types of samples used to generate adversarial samples, namely, remote tripping command samples for intelligent electronic devices R1, R2, and R3.

[0112] Furthermore, the model architecture to be optimized and the hyperparameters are integer-encoded according to the preset first parameter to generate the initial population P gen , specifically including:

[0113] Randomly generate a population P containing NP individuals gen , that is, P gen ={p1,p2,…,p NP}, 0≤gen≤Gen, where gen represents the current number of iterations, and the specific expression of each individual in the population is: p i ={p i,1 ,p i,2 ,…,p i,j ,…,p i,D}, where 1≤i≤NP, 1≤j≤D; p i,j represents the jth dimension of the i-th individual; D represents the dimension of the individual encoding; individual pi The specific code is p i ={Ops,Bts,Lrt,Nhd,Fnet,Dpt,Ativ,Lynm,Numl}, where Ops represents the optimizer in the Transformer model training process; Bts represents the data batch size in the Transformer model training process; Lrt represents the learning rate in the Transformer model training process; Nhd represents the number of attention heads in the robust-lightweight Transformer model encoder; Fnet represents the dimension of the feedforward neural network in the robust-lightweight Transformer model encoder; Dpt represents the dropout rate in the Transformer model training process; Ativ represents the activation function type in the robust-lightweight Transformer model encoder; Lynm represents the layer normalization eps value in the robust-lightweight Transformer model encoder, where the eps value is a smoothing term to prevent the denominator from being zero; Numl represents the number of sub-encoders in the robust-lightweight Transformer model encoder.

[0114] like Figure 2 As shown, in the above real number discrete coding, the encoding method of each variable is specifically as follows:

[0115] Ops = Randint(1,6), where Randint represents a random integer selection function. The specific values ​​of Ops are: Ops = 1 represents the adaptive moment estimation optimizer (Adam), Ops = 2 represents the stochastic gradient descent optimizer (SGD), Ops = 3 represents the root mean square propagation optimizer (RMSProp), Ops = 4 represents the adaptive gradient-based optimizer (Adadelta), Ops = 5 represents the asynchronous stochastic gradient descent optimizer (ASGD), and Ops = 6 represents the corrected adaptive moment estimation optimizer (RAdam); Bts = Rand(Bts1, Bts2, …, Bts k ), where Rand is a random selection function, and the specific value of Bts is a preset k number of Bts1, Bts2,…, Bts k The data batch size encoding during the training process of a Transformer model in , where Bts1 represents the encoding of the preset first batch size, Bts2 represents the encoding of the preset second batch size, and Bts k Indicates the encoding of the preset k-th batch size; Lrt=Rand(Lrt1, Lrt2,…, Lrt l ), where the specific value of Lrt is a preset l Lrt1, Lrt2, ..., Lrt lThe learning rate encoding during the training process of a Transformer model in , where Lrt1 represents the encoding of the preset first learning rate, Lrt2 represents the encoding of the preset second learning rate, and Lrt l Indicates the encoding of the preset lth learning rate; Nhd=Rand(Nhd1,Nhd2,…,Nhd t ), where the specific value of Nhd is a preset t number of Nhd1, Nhd2, ..., Nhd t The number of attention heads of a robust-lightweight Transformer model encoder in , where Nhd1 represents the encoding of the preset first number of attention heads of the encoder, Nhd2 represents the encoding of the preset second number of attention heads of the encoder, and Nhd t Represents the encoding of the preset number of attention heads of the t-th encoder; Fnet = Randint(Fnet min ,Fnet max ), where Fnet min With Fnet max Indicates the minimum and maximum dimensions of the robust-lightweight Transformer model encoder feedforward neural network. The specific value of Fnet is Fnet min With Fnet max An integer between; Dpt=Rand(Dpt1,Dpt2,…,Dpt e ), where the specific value of Dpt is the preset e Dpt1, Dpt2, ..., Dpt e A certain drop rate in, where Dpt1 represents the encoding of the preset first drop rate, Dpt2 represents the encoding of the preset second drop rate, and Dpt e Indicates the encoding of the preset e-th drop rate; Ativ = Randint(1,2), where the specific value of Ativ is: Ativ = 1 indicates that the type of activation function in the robust-lightweight Transformer model encoder is the linear rectification function (relu), and Ativ = 2 indicates that the type of activation function in the robust-lightweight Transformer model encoder is the Gaussian error linear unit (gelu); Lynm = Rand(Lynm1, Lynm2,…, Lynm r ), where the specific value of Lynm is the preset r Lynm1, Lynm2, ..., Lynm r The layer normalized EPS smoothing term in a robust-lightweight Transformer model encoder in , where Lynm1 represents the encoding of the preset first layer normalized EPS smoothing term, Lynm2 represents the encoding of the preset second layer normalized EPS smoothing term, Lynm rRepresents the encoding of the preset r-th layer normalized eps smoothing term; Numl=Randint(Numl min ,Numl max ), where Numl min With Numl max Indicates the minimum and maximum number of sub-encoders in the robust-lightweight Transformer model encoder. The specific value of Numl is Numl min With Numl max An integer between .

[0116] The mutation factor S is set as a dynamic mutation factor that changes dynamically with the number of iterations. Its calculation formula is shown in formula (5):

[0117]

[0118] Where S max With S min Represents the preset maximum and minimum variation factors respectively.

[0119] In the coding crossover process, the crossover factor Cr changes with the number of iterations, and its change formula is shown in formula (6):

[0120]

[0121] Where, Cr max With Cr min Represents the preset maximum and minimum crossover factors respectively.

[0122] Finally, according to the above parameters, the range of population P is selected. gen All individuals p in i , 1≤i≤NP, initialize.

[0123] In this embodiment, k is preset to 6, then the values ​​of Bts1, Bts2, ..., Bts6 are 1, 2, 3, 4, 5, 6, indicating that the batch sizes are 32, 64, 128, 256, 384 and 512 respectively; l is preset to 7, then the values ​​of Lrt1, Lrt2, ..., Lrt7 are 1, 2, 3, 4, 5, 6, 7, indicating that the actual learning rates are 0.0001, 0.001, 0.01, 0.1, 0.2, 0.3 and 0.4 respectively; t is preset to 5, then Nhd1, Nhd2, ..., Nhd5 indicate that the number of attention heads is 1, 2, 4, 8 and 16 respectively; the minimum dimension Fnet of the encoder feedforward neural network of the robust-lightweight Transformer model is preset min =1, maximum dimension Fnet max =2048, the specific value of Fnet is Fnet min With Fnetmax An integer between 1 and 2; if e=7, then the values ​​of Dpt1, Dpt2, …, Dpt7 are 1, 2, 3, 4, 5, 6, 7, which respectively indicate that the specific values ​​of the discard rate are 0, 0.01, 0.1, 0.2, 0.3, 0.4, and 0.5; if r=6, then the values ​​of Lynm1, Lynm2, …, Lynm6 are 1, 2, 3, 4, 5, 6, which respectively indicate that the specific value of the eps smoothing term in layer normalization is 1×10 -6 , 1×10 -5 , 1×10 -4 , 1×10 -3 , 1×10 -2 and 1×10 -1 ; Preset the minimum number of sub-encoders in the robust-lightweight Transformer model encoder Numl min =1 and the maximum number Numl max =8, then the specific value of Numl is an integer from 1 to 8.

[0124] For example, in this embodiment, Figure 2 As shown, the representative initial robust-lightweight Transformer model individuals randomly generated according to the preset range of the above parameters are: p i ={4,2,6,2,1,3,2,1,1}.

[0125] S3: Evaluate population P gen The model parameters and F1 scores of all individuals in the dataset are calculated, and it is determined whether the F1 score is greater than the adversarial training threshold. If not, the individual adversarial training is interrupted and the pre-evaluated model parameters and F1 score are saved, and the adversarial attack success rate is set to 100%. If so, step S4 is entered for adversarial training and testing. The specific steps include the following:

[0126] S31: Evaluate the population P according to the optimization goal of the robust-lightweight Transformer model gen The model parameters of all individuals in the optimization target obj1(p i ), and F1 score, where the F1 score is calculated according to formula (7);

[0127]

[0128] S32: Determine the population P gen Is the F1 score of the individual greater than the adversarial training threshold Gt? If not, interrupt the individual adversarial training and save the pre-evaluated number of model parameters and F1 score, and set the adversarial attack success rate to 100%; if so, enter step S4 for subsequent adversarial training and optimization search.

[0129] S4: Individual p i The characterized architecture and hyperparameters are decoded into the Transformer model as a replacement model for adversarial attack, and an adversarial attack training dataset D is generated based on the replacement model and Projected Gradient Descent (PGD) attack. adv_tr And the adversarial attack test dataset D adv_te ,like Figure 3 As shown, based on the training data set D tr With adversarial attack training dataset D adv_tr For individual p i After the training is completed, the adversarial training is performed based on the test data set D te Evaluate the F1 score of individual benign sample detection based on the adversarial attack test dataset D adv_te Evaluate the success rate of the adversarial attack. This includes the following sub-steps:

[0130] S41: Generate population P according to step S3 gen The individual p in which the F1 score is greater than the adversarial training threshold Gt i The replacement model is used to generate the adversarial attack training dataset D based on the replacement model and PGD adversarial attack. adv_tr And the adversarial attack test dataset D adv_te ; Among them, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te The specific generation process is as follows: First, from the training data set D tr With the test data set D te Randomly select m categories of benign samples from the M categories of benign sample data to construct the initial adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te , where m categories are randomly selected, the maximum is m max categories, the minimum is m min categories, 1≤m min ≤m≤m max ≤M; Secondly, based on the above replacement model, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te All samples in the are subjected to adversarial attacks one by one to generate corresponding adversarial samples. The calculation method for generating adversarial samples is:

[0131] S41.1: From the adversarial attack training dataset D adv_tr Select the original sample Sb adv_tr,tx , where Sb adv_tr,tx Denotes the adversarial attack training dataset D adv_trThe txth original sample in , 1≤tx≤Tr, Tr represents the adversarial attack training dataset D adv_tr ; and initialize the preset PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ, and the maximum number of iterations ζ of PGD adversarial attack.

[0132] S41.2: According to formula (8), randomly initialize the adversarial sample and mark the generated initialized adversarial sample as Sadv:

[0133] Sadv=Sb+Uniform(-ε,ε) (8)

[0134] Where Sb represents the original sample; Uniform(-ε,ε) represents uniformly sampling a random noise with the same shape as the original sample in the interval [-ε,ε]; adv_tr When generating the initial adversarial sample, Sb=Sb adv_tr,tx , 1≤tx≤Tr; in the adversarial attack test dataset D adv_te When generating the initial adversarial sample, Sb=Sb adv_te,ty , 1≤ty≤Te, where ty represents the adversarial attack test dataset D adv_te The tyth original sample in, Te represents the adversarial attack test dataset D adv_te The number of all samples in .

[0135] S41.3: According to formula (9), the range of the adversarial sample Sadv is constrained:

[0136]

[0137] S41.4: Mark the current iteration number of the PGD adversarial attack as iter. For each iteration from iter = 1 to iteration number ζ, perform iterative optimization on the adversarial example Sadv according to formula (10):

[0138]

[0139] Where iter represents the current iteration number of PGD attack, Sadv iter and Sadv iter+1 denote the iter-th and (iter+1)-th adversarial samples respectively; Ran denotes the set of random perturbation constraints, Sb+Ran denotes the perturbation range of all adversarial samples after the original sample Sb is added with the perturbation ε; Π Sb+Ran According to formula (9), the generated adversarial sample Sadv iter Perform range constraints; Lable represents the true label of the sample; L(θ, Sadv iter, Lable) represents the loss function, θ is the network parameter of the replacement model; sign represents the gradient calculation symbol, and its calculation formula is shown in formula (11):

[0140]

[0141] Where z represents the independent variable of the sign function.

[0142] S41.5: The adversarial attack training dataset D adv_tr The original sample Sb adv_tr,tx Replace it with the adversarial sample Sadv generated after reaching the maximum number of PGD iterations ζ.

[0143] S41.6: Repeat the adversarial sample generation process described in steps S41.1 to S41.5 until the adversarial attack training dataset D adv_tr All original samples Sb adv_tr,tx , 1≤tx≤Tr, are all replaced by the corresponding tx-th adversarial examples.

[0144] S41.7: From the adversarial attack test dataset D adv_te Select the original sample Sb adv_te,ty , among which Sb adv_te,ty Denotes the adversarial attack training dataset D adv_tr The ty-th original sample in , 1≤ty≤Te, Te represents the adversarial attack test dataset D adv_te ; and initialize the preset PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ and the maximum number of iterations ζ of PGD adversarial attack; repeat the adversarial sample generation process described in steps S41.1 to S41.5 above until the adversarial attack test dataset D adv_te All original samples Sb adv_te,ty , 1≤ty≤Te, are all replaced by the corresponding ty-th adversarial examples.

[0145] S42: Based on the training data set D tr Compared with the above adversarial attack training dataset D adv_tr For individual p i The corresponding robust-lightweight Transformer model is trained against the test dataset D te Re-evaluate the F1 score of individual benign sample detection, that is, optimize the target obj2(p i ), and based on the adversarial attack test dataset D adv_te Evaluate the success rate of the adversarial attack, that is, optimize the target obj3(p i ).

[0146] For example, in this embodiment, there are M=37 categories in the data of the smart grid cyber-physical fusion system. Two adversarial attack defense examples are carried out, namely, a single-category non-targeted adversarial attack defense example and a multi-category non-targeted adversarial attack defense example, i.e., adversarial attack and defense examples when m=1 and m=3. In the single-category non-targeted adversarial attack defense example, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te The adversarial samples come from the first category of samples, that is, the data samples of short circuit faults at 10% to 19% of the busbar L1. In the single-category non-target adversarial attack defense example, that is, the adversarial attack training dataset D when m = 1 adv_tr There are 138 samples in total, namely Tr = 138, and the adversarial attack test dataset D adv_te There are 35 samples in total, i.e. Te = 35; in the multi-category non-targeted adversarial attack defense example, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te The adversarial samples come from three types of samples of remote tripping instructions for intelligent electronic devices R1, R2 and R3. In the multi-category non-target adversarial attack defense example, the adversarial attack training dataset D when m=3 is used. adv_tr There are 184 samples in total, namely Tr = 184, and the adversarial attack test dataset D adv_te There are 46 samples in total, i.e., Te = 46. In addition, the specific process of the above adversarial training is as follows:

[0147] (1) The training dataset D tr With adversarial attack training dataset D adv_tr Combine them to form the dataset D for adversarial training train , where D in the single-category non-targeted adversarial attack defense example train There are 3832 data samples in total, D in the multi-category non-targeted adversarial attack defense example train There are 3878 data samples in total; and according to individual p i Set the dataset D corresponding to the Bts code value train The batch size of .

[0148] (2) Use batches of training data to train the current individual p iThe robust-lightweight Transformer model represented is iteratively trained. During training, the 128-dimensional data features are split into Nhd attention heads for internal processing, that is, each attention head processes (128 / Nhd) features, and then the attention output is added to the original data and layer normalization operations are performed, that is, residual connection and layer normalization operations. The data dimension is then expanded and activated through the feedforward network Fnet and the activation function Ativ. After completing the above operations, a second residual connection and the final layer normalization are performed.

[0149] (3) The data after the completion layer normalization is processed through all the preset Numl sub-encoder layers, and the processed data is activated by the Tanh activation function so that its eigenvalue is in the range of (-1, 1). Finally, all the output data is flattened and input into the final fully connected layer for classification detection; repeat the above adversarial training operation until the model is trained.

[0150] Then, after the model completes the iterative training, the current robust-lightweight Transformer model is evaluated using the test dataset D te Calculate and save key detection performance indicators, namely intrusion detection accuracy, recall, precision and F1 score. In addition, use the adversarial attack test dataset D adv_te Evaluate the adversarial robustness of the robust-lightweight Transformer model, that is, the success rate of adversarial attacks on adversarial samples; finally, save the optimization objective function of the robust-lightweight Transformer model after adversarial training, that is, the number of model parameters, the 1-F1 score of benign sample intrusion detection, and the success rate of adversarial attacks.

[0151] S5: P gen Perform non-dominated sorting and crowding calculation on all individuals in to generate the offspring population Q gen , and record the number of iterations gen. The specific steps include the following:

[0152] S51: First, for the population P gen Each individual p i Assign two key parameters: the dominant individual p i The number of individuals ind_num p and by individual p i Dominant individual set Slu p ; Secondly, set rto = 1, ind_num p = 0 individuals are included in the frontier Ft rto In; Again, on the frontier Ft rto All individuals in Slu p Traverse and set the ind_num of each individual pSubtract 1; then, set rto = rto + 1, and then ind_num p = 0 is included in the frontier Ft rto Finally, repeat the above steps until the population P gen All individuals p in i All are included in a certain frontier Ft rto middle.

[0153] S52: After completing the non-dominated sorting, the crowding degree of each individual in the population is calculated. The specific steps of the crowding degree calculation are: first, take all individuals in a single frontier and sort them from small to large according to the fitness value of a certain optimization goal; second, take the largest fitness value as the fit o,max , the minimum fitness value is taken as fit o,min , and fit the two extreme values o,max and fit o,min The crowding distance between them is set to infinity, denoted as inf; again, the crowding distance between each individual and the nearest neighbor is calculated on the optimization target, and the two extreme values ​​of the crowding distance on the corresponding optimization target are fit o,max and fit o,min Min-max normalization is performed between the two sets of parameters, where the min-max normalization calculation formula is shown in formula (12) and the crowding calculation formula is shown in formula (13); then, all optimization objectives are traversed and the standardized crowding distances of all objectives are added together; finally, the next Pareto front is entered until the crowding distances of all individuals are calculated;

[0154]

[0155] Where, Represents the normalized fitness value of the i-th individual on the optimization target o; represents the fitness value of the i-th individual on the optimization target o, fit o,max and fit o,min Respectively represent the maximum and minimum values ​​of the congestion distance on the corresponding optimization target; dt i represents the crowding distance of the i-th individual; t represents the total number of optimization targets; Represents the fitness value of the i+1th individual on the optimization target o; Represents the fitness value of the i-1th individual on the optimization target o.

[0156] S53: Offspring population Q gen The specific steps of generating are: randomly select population P genThere are three groups of individuals with two individuals in each group. A binary tournament selection is performed on the undominated individuals with a large crowding degree in each group, and the selected individuals are used as parents, denoted as s1, s2 and s3. Among them, the undominated individuals are preferred. If the number of individuals dominated is the same, the individuals with a large crowding degree are considered. The mutation operation is performed on the parents s1, s2 and s3 as the basic individuals. The mutation generation method of the offspring individuals q1 and q2 is shown in formula (14) and formula (15):

[0157] q 1,d =s 1,d +S×(s 2,d -s 3,d )(14)

[0158] q 2,d =s 2,d +S×(s 3,d -s 1,d )(15)

[0159] Where S represents the variation factor that changes with the number of iterations, d represents the dth dimension of the individual code, 1≤d≤D; q 1,d and q 2,d Represents the dth dimension of offspring individuals q1 and q2 respectively; s 1,d 、s 2,d and s 3,d They represent the dth dimension of the parent individuals s1, s2, and s3 respectively. After the generation of offspring individual mutations is completed, the generated offspring individuals are subjected to a crossover operation. The specific operation is shown in formulas (16) and (17):

[0160]

[0161] Where rand 1,d and rand 2,d Represents a random decimal generated between [0,1], s 1,d and s 2,d Represent the dth dimension of parent individuals s1 and s2 respectively, and Cr represents the crossover factor that changes with the number of iterations; after completing the crossover operation, let q1 = q′1, q2 = q′2; in the process of mutation and crossover, if the generated new code exceeds the preset range, the value exceeding the preset range of the code will be truncated, and the value exceeding the upper limit of the range will be truncated to the maximum value preset for the dth dimension code, and the value exceeding the lower limit of the range will be truncated to the minimum value preset for the dth dimension code, so that the generated offspring individual code is always within the preset range; then repeat the above operation until the offspring population Q gen The population size reaches NP; finally, record the current number of iterations gen.

[0162] For example, in this embodiment, according to the three groups of individuals randomly selected in step S5 above and in combination with the accompanying drawings, Figure 4 The binary tournament selection, mutation and crossover operations are explained as follows: First, three groups of individuals are selected from the population, namely, {5, 4, 3, 1, 512, 3, 1, 2, 4} and {2, 2, 4, 3, 256, 3, 2, 2, 3} in the first group, {6, 2, 3, 2, 300, 3, 1, 1, 5} and {1, 5, 5, 2, 128, 1, 1, 4} in the second group, and {3, 4, 2, 1, 624, 3, 2, 2, 1} and {1, 2, 3, 1, 468, 2, 1, 2, 4} in the third group.}; In addition, the total number of dominated individuals of each individual is: 0 and 0 for the first group, 1 and 0 for the second group, and 1 and 2 for the third group; the crowding distance of each individual is: 0.5 and 0.2 for the first group, 0.7 and 0.1 for the second group, and 0.4 and 0.8 for the third group; Therefore, the parents s1, s2 and s3 selected from the three groups of individuals in a binary tournament are: s1 = {5, 4, 3, 1, 512, 3, 1, 2, 4}; s2 = {1, 5, 5, 2, 128, 1, 1, 1,4}; s3 = {1,2,3,1,468,2,1,2,4}; Secondly, according to formula (13) and formula (14), the parent individual mutation operation is performed to generate the offspring. When performing the mutation, the uniform mutation idea is adopted. The corresponding random number is generated according to the current mutation dimension. If the random number is greater than the current mutation factor, the mutation is performed until all dimensions of the individual have completed the mutation operation. In addition, the offspring individuals q1 and q2 obtained after the mutation are: q1 = {1,2,5,2,256,1,2,1,4} and q2 = {3,2,2, 1,436,3,1,2,2}; finally, the generated offspring individuals are crossover-operated based on formula (15) and formula (16), and the offspring individuals q1, q2 are compared with the parent individuals s1, s2 based on the randomly generated random number and the current crossover factor. If the current random number is greater than the crossover factor, crossover is performed until the crossover operations of all dimensions of the individual are completed. The final offspring individuals are: q1 = {5, 2, 3, 2, 256, 3, 2, 2, 4} and q2 = {3, 2, 2, 1, 436, 1, 1, 2, 2}.

[0163] S6: For population P gen With Q gen Perform non-dominated sorting and crowding calculation on all individuals in to obtain the new parent population P. gen , save all individuals in the current Pareto front to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update archives using the update method.

[0164] Further, in step S6, the external archive solution Ach gen The update specifically includes the following steps:

[0165] S61: First, the parent population P gen and the offspring population Q gen Merge into a new population, where there are 2×NP individuals in the new population; secondly, gen With Q gen Perform the non-dominated sorting and crowding calculation described in step S6 on all individuals in the parent population, and filter the first NP individuals to obtain a new parent population P. gen ; Again, the current parent population P gen All individuals in the Pareto front are saved to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update archives using the update method.

[0166] S62: External archive solution Ach gen The update method is as follows: First, in the current population P gen All individuals in the Pareto front are saved to Ach gen After the middle, to Ach gen Perform non-dominated sorting and crowding calculation on all individuals in Ach gen To update, if Ach gen The number of individuals in the archive exceeds the preset maximum number of external archive individuals A max , then delete Ach gen Individuals with more dominated solutions and lower crowding degree go directly to Ach gen The number of individuals in the archive is less than the maximum number of individuals in the external archive A max .

[0167] S7: Determine whether the current number of iterations has reached the maximum number of iterations Gen. If so, save the external archive solution set Ach gen If not, set the current number of iterations gen = gen + 1, and return to step S3 to continue searching and evaluating the robust-lightweight Transformer model.

[0168] For example, in this embodiment, after 20 iterations, the external archive solution set Ach obtained in the single-category non-targeted adversarial attack defense example is gen There are 4 individuals in total, namely: p Ach,1 ={1,1,1,3,163,1,2,2,3};p Ach,2={3,1,2,2,1,2,2,3,1};p Ach,3 ={3,3,3,1,63,2,1,4,1};p Ach,4 ={1,1,1,4,117,1,2,3,3}; In addition, the fitness values ​​of the four individuals are: [330510,0.0364,0.0000]; [71718,0.0490,0.0000]; [87652,0.0405,0.0000]; [295044,0.0392,0.0000]; The external archive solution set Ach obtained in the multi-class non-targeted adversarial attack defense example gen There are 10 individuals in total, namely: p Ach,1 ={1,2,1,3,2048,6,2,5,7};p Ach,2 ={5,4,7,2,1,4,2,1,1};p Ach,3 ={6,4,3,2,16,4,2,1,1};p Ach,4 ={1,3,1,5,1677,5,2,6,1};p Ach,5 ={1,3,1,5,1740,5,2,6,3};p Ach,6 ={1,1,1,5,1648,6,2,6,8};p Ach,7 ={1,1,1,4,2048,7,2,3,5};p Ach,8 ={3,6,3,4,1,3,1,1,1};p Ach,9 ={4,3,4,3,1216,6,2,6,8};p Ach,10 ={6,5,7,5,1251,3,2,1,1}; In addition, the fitness values ​​of the 10 individuals are: [4155941,0.0588,0.5179]; [71718,0.0839,0.5893]; [75573,0.0470,0.8214]; [502450,0.0752,0.5179]; [1546377,0.0587,0.5536]; [39265 65,0.0486,0.6071]; [2969893,0.0583,0.5893]; [71718,0.0702,0.6071]; [3038373,0.0507,0.5714]; [392968,0.0547,0.7679]; After iterative optimization search, the Pareto frontiers obtained in the single-category non-targeted adversarial attack defense example and the multi-category non-targeted adversarial attack defense example are as follows: Figure 5 and Figure 6 shown.

[0169] S8: Screen a robust-lightweight Transformer model that takes into account the F1 score of benign samples in multi-classification detection, the success rate of adversarial attacks, and the minimum number of model parameters, and use it for online deployment of power grid intrusion detection.

[0170] Specifically, the external archive is unpacked Ach gen All non-dominated individuals in the Pareto front solution are taken as the Pareto front solution, and the optimal compromise individual is selected from the Pareto front solution. The model architecture and parameters encoded by the individual are used as the architecture and parameters of the optimal robust-lightweight Transformer model, thereby screening out a robust-lightweight Transformer model that takes into account the F1 score of multi-classification detection of benign samples, the success rate of adversarial attacks, and the minimization of the number of model parameters. It is then used for online deployment of power grid intrusion detection to achieve adversarial defense of the smart grid intrusion detection system.

[0171] Furthermore, the performance indicators of intrusion detection also include accuracy, recall and precision, and their calculation formulas are expressed as follows:

[0172]

[0173] In this embodiment, the external archive solution set Ach is obtained from the single-category non-targeted adversarial attack defense example. gen The optimal compromise individual obtained by screening is p Ach,2 ={3,1,2,2,1,2,2,3,1}, and its intrusion detection performance indicators are as follows Figure 7 As shown; from the external archive solution set Ach of the multi-class non-targeted adversarial attack defense example gen The optimal compromise individual obtained by screening is p Ach,2 ={5,4,7,2,1,4,2,1,1}, and its intrusion detection performance indicators are as follows Figure 8 As shown in the figure, it can be seen from the compromised individual performance indicators that the robust-lightweight Transformer model searched in this embodiment significantly reduces the success rate of adversarial attacks while ensuring a low number of model parameters and a high detection success rate, effectively enhancing the adversarial robustness of the model.

[0174] In summary, through the technical solutions described in steps S1 to S8 above, the present invention comprehensively considers the three indicators of model lightweight, intrusion detection performance and robustness of the power grid intrusion defense system, designs a multi-objective optimization method of binary tournament selection, mutation and crossover operations, and elite individual external archiving mechanism, and rationally screens elite individuals through non-dominated sorting and congestion calculation, effectively realizing the multi-objective automatic optimization design of a robust-lightweight Transformer model for power grid intrusion detection application scenarios that takes into account model lightweight, intrusion detection accuracy, and robustness against attacks. The present invention not only further improves the automatic design level of the robust power grid intrusion detection system, but also takes into account the three important indicators of the model parameter quantity, intrusion detection performance and robust performance of the model, which is more conducive to the online deployment and detection of the robust power grid intrusion detection model, and further expands the engineering implementation and promotion application of multi-objective automatic deep learning technology in the field of smart grid information security. By combining formula (1) to formula (20) with steps S1 to S8 in the present invention, the robust-lightweight Transformer model finally obtained has excellent comprehensive performance in terms of model parameter quantity, various performance indicators and robustness indicators.

[0175] It is worth mentioning that the present invention also provides a power grid countermeasure defense system based on robust-lightweight Transformer model optimization, which is used to implement the power grid countermeasure defense method based on robust-lightweight Transformer model optimization in the above embodiment. Figure 9 As shown, the system includes a data preprocessing module 10, a population generation module 20, an adversarial training judgment module 30, an adversarial training and evaluation module 40, a child population generation module 50, an external archive update module 60, an external archive preservation module 70 and an online deployment module 80.

[0176] In this embodiment, the data preprocessing module 10 is used to construct a training data set D for robust-lightweight Transformer model optimization based on the smart grid cyber-physical fusion system data. tr With the test data set D te .

[0177] In this embodiment, the population generation module 20 is used to set a plurality of first parameters, construct the optimization target of the robust-lightweight Transformer model, and integer encode the model architecture to be optimized and the hyperparameters according to the preset first parameters to generate the initial population P gen .

[0178] In this embodiment, the adversarial training judgment module 30 is used to evaluate the population P genThe model parameter quantities and F1 scores of all individuals in the training set are calculated, and it is determined whether the F1 score is greater than the adversarial training threshold. If not, the individual adversarial training is interrupted and the pre-evaluated model parameter quantities and F1 scores are saved, and the adversarial attack success rate is set to 100%; if so, the adversarial training and evaluation module 40 is entered for adversarial training and testing.

[0179] In this embodiment, the adversarial training and evaluation module 40 is used to i The characterized architecture and hyperparameters are decoded into the Transformer model as a replacement model for adversarial attack, and the adversarial attack training dataset D is generated based on the replacement model and projected gradient descent attack. adv_tr And the adversarial attack test dataset D adv_te , then based on the training data set D tr With adversarial attack training dataset D adv_tr For individual p i After the training is completed, the adversarial training is performed based on the test data set D te Evaluate the F1 score of individual benign sample detection based on the adversarial attack test dataset D adv_te Evaluate the success rate of adversarial attacks.

[0180] In this embodiment, the offspring population generation module 50 is used to generate the P gen Perform non-dominated sorting and crowding calculation on all individuals in to generate the offspring population Q gen , and record the number of iterations gen.

[0181] In this embodiment, the external archive update module 60 is used to update the population P gen With Q gen Perform non-dominated sorting and crowding calculation on all individuals in to obtain the new parent population P. gen , save all individuals in the current Pareto front to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update archives using the update method.

[0182] In this embodiment, the external archive saving module 70 is used to save the external archive solution set Ach when the current iteration number reaches the maximum iteration number Gen. gen A robust-lightweight Transformer model represented by all individuals in .

[0183] In this embodiment, the online deployment module 80 is used to screen a robust-lightweight Transformer model that takes into account the F1 score of benign samples in multi-classification detection, the success rate of adversarial attacks, and the minimization of the number of model parameters, and uses it for online deployment of power grid intrusion detection.

[0184] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc. Various media that can store program codes.

[0185] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A power grid adversarial defense method based on robust-lightweight Transformer model optimization, characterized by: The following steps are involved: S1: Building a training dataset D for robust-lightweight Transformer model optimization based on smart grid cyber-physical fusion system data tr With the test data set D te ; S2: Set multiple first parameters, construct the optimization target of the robust-lightweight Transformer model, and integer encode the model architecture and hyperparameters to be optimized according to the preset first parameters to generate the initial population P gen ={p i }, i=1,2,…,NP, where p i represents the i-th individual, NP represents the population size; S3: Evaluate population P gen The model parameter quantity and F1 score of all individuals in the test are evaluated, and whether the F1 score is greater than the adversarial training threshold is determined. If not, the individual adversarial training is interrupted and the pre-evaluated model parameter quantity and F1 score are saved, and the adversarial attack success rate is set to 100%. If so, step S4 is entered for adversarial training and testing; S4: Individual p i The characterized architecture and hyperparameters are decoded into the Transformer model as a replacement model for adversarial attack, and the adversarial attack training dataset D is generated based on the replacement model and projected gradient descent attack. adv_tr And the adversarial attack test dataset D adv_te , then based on the training data set D tr With adversarial attack training dataset D adv_tr For individual p i After the training is completed, the adversarial training is performed based on the test data set D te Evaluate the F1 score of individual benign sample detection based on the adversarial attack test dataset D adv_te Evaluate the success rate of adversarial attacks; S5: P gen Perform non-dominated sorting and crowding calculation on all individuals in to generate the offspring population Q gen , and record the number of iterations gen; S6: For population P gen With Q gen Perform non-dominated sorting and crowding calculation on all individuals in to obtain the new parent population P. gen , save all individuals in the current Pareto front to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update the archive using the update method; S7: Determine whether the current number of iterations has reached the maximum number of iterations Gen. If so, save the external archive solution set Ach gen If not, set the current iteration number gen = gen + 1 and return to step S3 to continue searching and evaluating the robust-lightweight Transformer model; S8: Screen a robust-lightweight Transformer model that takes into account the F1 score of benign samples in multi-classification detection, the success rate of adversarial attacks, and the minimum number of model parameters, and use it for online deployment of power grid intrusion detection.

2. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1 is characterized in that: The step S1 specifically includes: The smart grid cyber-physical fusion system data is divided into a training dataset D for robust-lightweight Transformer model search. tr With the test data set D te ; and according to the training data set D tr The data distribution of the training data set D tr With the test data set D te Perform Z-Score standardization processing, where the calculation process of Z-Score standardization is shown as follows: Where x′ α,β represents the value of the βth feature of the αth data sample after normalization, x α,β represents the value of the βth feature of the αth data sample before standardization, μ represents the mean of the βth feature, and σ represents the standard deviation of the βth feature; then the training data set D after Z-Score standardization is completed tr And the test dataset D te Optimal architecture search and hyperparameter optimization for robust-lightweight Transformer models.

3. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1 is characterized in that: The first parameters include the population size NP, the maximum number of iterations Gen, the maximum number of external archived individuals A max , cross factor Cr, mutation factor F, adversarial training threshold Gt, PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ and PGD adversarial attack iteration number ζ; Among them, the cross factor Cr includes the maximum and minimum cross factors Cr max With Cr min , the variation factor F includes the maximum and minimum variation factors F max With F min ; The optimization goal of the robust-lightweight Transformer model is: Where p represents the encoding individual of the robust-lightweight Transformer model to be optimized, the optimization target obj1(p) is the number of parameters of the robust-lightweight Transformer model Para-num, the optimization target obj2(p) is the multi-classification benign sample intrusion detection performance index 1-F1 score F1-Score, and the optimization target obj3(p) is the PGD adversarial attack success rate AASR; among them, the optimization target obj1(p) is used to measure the number of parameters of the searched robust-lightweight Transformer model; the optimization target obj2(p) refers to the test dataset D constructed using the smart grid cyber-physical fusion data during the robust-lightweight Transformer model search process. te , the 1-F1 score indicator for multi-classification intrusion detection performance evaluation is calculated as shown in formula (3): Where TP represents the number of samples predicted as true normal in benign samples, TN represents the number of samples predicted as true abnormal in benign samples, FP represents the number of samples predicted as false normal in benign samples, and FN represents the number of samples predicted as false abnormal in benign samples. Benign samples refer to samples that have not been generated by adversarial samples. The optimization target obj3(p) refers to the F1 score of the robust-lightweight Transformer model for benign sample classification detection that is higher than the adversarial training threshold Gt, based on the training dataset D tr With the test data set D te Some benign samples in the robust-lightweight Transformer replacement model generate the PGD adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te Then, based on the training data set D tr With adversarial attack training dataset D adv_tr Train a robust-lightweight Transformer model and use the adversarial attack test dataset D adv_te The PGD adversarial attack success rate obtained when the robust-lightweight Transformer model is subjected to adversarial attacks. The parameters of the robust-lightweight Transformer replacement model are exactly the same as those of the robust-lightweight Transformer model. The specific calculation formula of the PGD adversarial attack success rate AASR is shown in formula (4): In the formula, the total number of samples refers to the adversarial attack test dataset D adv_te The total number of samples in the attack dataset D is the number of successful attack samples. adv_te The total number of samples misclassified by the adversarial attack samples in ; The model architecture to be optimized and the hyperparameters are integer-encoded according to the preset first parameter to generate the initial population P gen , specifically including: Randomly generate a population P containing NP individuals gen , that is, P gen ={p1,p2,…,p NP }, 0≤gen≤Gen, where gen represents the current number of iterations, and the specific expression of each individual in the population is: p i ={p i,1 ,p i,2 ,…,p i,j ,…,p i,D }, where 1≤i≤NP, 1≤j≤D; p i,j represents the jth dimension of the i-th individual; D represents the dimension of the individual encoding; individual p i The specific code is p i ={Ops,Bts,Lrt,Nhd,Fnet,Dpt,Ativ,Lynm,Numl}, where Ops represents the optimizer during Transformer model training; Bts represents the data batch size during Transformer model training; Lrt represents the learning rate during Transformer model training; Nhd represents the number of attention heads in the robust-lightweight Transformer model encoder; Fnet represents the dimension of the feedforward neural network in the robust-lightweight Transformer model encoder; Dpt represents the dropout rate during Transformer model training; Ativ represents the activation function type in the robust-lightweight Transformer model encoder; Lynm represents the layer normalization eps value in the robust-lightweight Transformer model encoder, where the eps value is a smoothing term to prevent the denominator from being zero; Numl represents the number of sub-encoders in the robust-lightweight Transformer model encoder; In the above real number discrete coding, the encoding method of each variable is as follows: Ops = Randint(1,6), where Randint represents a random integer selection function. The specific values ​​of Ops are: Ops = 1 represents an adaptive moment estimation optimizer, Ops = 2 represents a stochastic gradient descent optimizer, Ops = 3 represents a root mean square propagation optimizer, Ops = 4 represents an adaptive gradient-based optimizer, Ops = 5 represents an asynchronous stochastic gradient descent-based optimizer, and Ops = 6 represents a corrected adaptive moment estimation optimizer; Bts = Rand(Bts1, Bts2, …, Bts k ), where Rand is a random selection function, and the specific value of Bts is a preset k number of Bts1, Bts2,…, Bts k The data batch size encoding during the training process of a Transformer model in , where Bts1 represents the encoding of the preset first batch size, Bts2 represents the encoding of the preset second batch size, and Bts k Indicates the encoding of the preset k-th batch size; Lrt=Rand(Lrt1, Lrt2,…, Lrt l ), where the specific value of Lrt is a preset l Lrt1, Lrt2, ..., Lrt l The learning rate encoding during the training process of a Transformer model in , where Lrt1 represents the encoding of the preset first learning rate, Lrt2 represents the encoding of the preset second learning rate, and Lrt l Indicates the encoding of the preset lth learning rate; Nhd=Rand(Nhd1,Nhd2,…,Nhd t ), where the specific value of Nhd is a preset t number of Nhd1, Nhd2, ..., Nhd t The number of attention heads of a robust-lightweight Transformer model encoder in , where Nhd1 represents the encoding of the preset first number of attention heads of the encoder, Nhd2 represents the encoding of the preset second number of attention heads of the encoder, and Nhd t Represents the encoding of the preset number of attention heads of the t-th encoder; Fnet = Randint(Fnet min ,Fnet max ), where Fnet min With Fnet max Indicates the minimum and maximum dimensions of the robust-lightweight Transformer model encoder feedforward neural network. The specific value of Fnet is Fnet min With Fnet max An integer between; Dpt=Rand(Dpt1,Dpt2,…,Dpt e ), where the specific value of Dpt is the preset e Dpt1, Dpt2, ..., Dpt e A certain drop rate in, where Dpt1 represents the encoding of the preset first drop rate, Dpt2 represents the encoding of the preset second drop rate, and Dpt e Indicates the encoding of the preset e-th drop rate; Ativ = Randint(1,2), where the specific value of Ativ is: Ativ = 1 indicates that the type of activation function in the robust-lightweight Transformer model encoder is a linear rectification function, and Ativ = 2 indicates that the type of activation function in the robust-lightweight Transformer model encoder is a Gaussian error linear unit; Lynm = Rand(Lynm1, Lynm2,…, Lynm r ), where the specific value of Lynm is the preset r Lynm1, Lynm2, ..., Lynm r The layer normalized EPS smoothing term in a robust-lightweight Transformer model encoder in , where Lynm1 represents the encoding of the preset first layer normalized EPS smoothing term, Lynm2 represents the encoding of the preset second layer normalized EPS smoothing term, Lynm r Represents the encoding of the preset r-th layer normalized eps smoothing term; Numl=Randint(Numl min ,Numl max ), where Numl min With Numl max Indicates the minimum and maximum number of sub-encoders in the robust-lightweight Transformer model encoder. The specific value of Numl is Numl min With Numl max An integer between The mutation factor S is set as a dynamic mutation factor that changes dynamically with the number of iterations. Its calculation formula is shown in formula (5): Where S max With S min Respectively represent the preset maximum and minimum variation factors; In the coding crossover process, the crossover factor Cr changes with the number of iterations, and its change formula is shown in formula (6): Where, Cr max With Cr min Respectively represent the preset maximum and minimum cross factors; Finally, according to the above parameters, the range of population P is selected. gen All individuals p in i , 1≤i≤NP, initialize.

4. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1 is characterized in that: The S3 specifically includes the following sub-steps: S31: Evaluate the population P according to the optimization goal of the robust-lightweight Transformer model gen The model parameters of all individuals in the optimization target obj1(p i ), and F1 score, where the F1 score is calculated according to formula (7); S32: Determine the population P gen Is the F1 score of the individual greater than the adversarial training threshold Gt? If not, interrupt the individual adversarial training and save the pre-evaluated number of model parameters and F1 score, and set the adversarial attack success rate to 100%; if so, enter step S4 for subsequent adversarial training and optimization search.

5. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1 is characterized in that: The step S4 specifically includes the following sub-steps: S41: Generate population P according to step S3 gen The individual p in which the F1 score is greater than the adversarial training threshold Gt i The replacement model is used to generate the adversarial attack training dataset D based on the replacement model and PGD adversarial attack. adv_tr And the adversarial attack test dataset D adv_te ; Among them, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te The specific generation process is as follows: First, from the training data set D tr With the test data set D te Randomly select m categories of benign samples from the M categories of benign sample data to construct the initial adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te , where m categories are randomly selected, the maximum is m max categories, the minimum is m min categories, 1≤m min ≤m≤m max ≤M; Secondly, based on the above replacement model, the adversarial attack training dataset D adv_tr And the adversarial attack test dataset D adv_te All samples in are subjected to adversarial attacks one by one to generate corresponding adversarial samples; S42: Based on the training data set D tr Compared with the above adversarial attack training dataset D adv_tr For individual p i The corresponding robust-lightweight Transformer model is trained against the test dataset D te Re-evaluate the F1 score of individual benign sample detection, that is, optimize the target obj2(p i ), and based on the adversarial attack test dataset D adv_te Evaluate the success rate of the adversarial attack, that is, optimize the target obj3(p i ).

6. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 5 is characterized in that: In step S41, the adversarial sample is generated by the following steps: S41.1: From the adversarial attack training dataset D adv_tr Select the original sample Sb adv_tr,tx , where Sb adv_tr,tx Denotes the adversarial attack training dataset D adv_tr The txth original sample in , 1≤tx≤Tr, Tr represents the adversarial attack training dataset D adv_tr ; and initialize the preset PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ, and the maximum number of iterations ζ of PGD adversarial attack; S41.2: According to formula (8), randomly initialize the adversarial sample and mark the generated initialized adversarial sample as Sadv: Sadv=Sb+Uniform(-ε,ε) (8) Where Sb represents the original sample; Uniform(-ε,ε) represents uniformly sampling a random noise with the same shape as the original sample in the interval [-ε,ε]; adv_tr When generating the initial adversarial sample, Sb=Sb adv_tr,tx , 1≤tx≤Tr; in the adversarial attack test dataset D adv_te When generating the initial adversarial sample, Sb=Sb adv_te,ty , 1≤ty≤Te, where ty represents the adversarial attack test dataset D adv_te The tyth original sample in, Te represents the adversarial attack test dataset D adv_te The number of all samples in ; S41.3: According to formula (9), the range of the adversarial sample Sadv is constrained: S41.4: Mark the current iteration number of the PGD adversarial attack as iter. For each iteration from iter = 1 to iteration number ζ, perform iterative optimization on the adversarial example Sadv according to formula (10): Where iter represents the current iteration number of PGD attack, Sadv iter and Sadv iter+1 denote the iter-th and (iter+1)-th adversarial samples respectively; Ran denotes the set of random perturbation constraints, Sb+Ran denotes the perturbation range of all adversarial samples after the original sample Sb is added with the perturbation ε; Π Sb+Ran According to formula (9), the generated adversarial sample Sadv iter Perform range constraints; Lable represents the true label of the sample; L(θ, Sadv iter , Lable) represents the loss function, θ is the network parameter of the replacement model; sign represents the gradient calculation symbol; S41.5: The adversarial attack training dataset D adv_tr The original sample Sb adv_tr,tx Replace it with the adversarial sample Sadv generated after reaching the maximum number of PGD iterations ζ; S41.6: Repeat the adversarial sample generation process described in steps S41.1 to S41.5 until the adversarial attack training dataset D adv_tr All original samples Sb adv_tr,tx , 1≤tx≤Tr, are all replaced by the corresponding tx-th adversarial sample; S41.7: From the adversarial attack test dataset D adv_te Select the original sample Sb adv_te,ty , among which Sb adv_te,ty Denotes the adversarial attack training dataset D adv_tr The ty-th original sample in , 1≤ty≤Te, Te represents the adversarial attack test dataset D adv_te ; and initialize the preset PGD adversarial attack perturbation strength ε, PGD adversarial attack step size δ and the maximum number of iterations ζ of PGD adversarial attack; repeat the adversarial sample generation process described in steps S41.1 to S41.5 above until the adversarial attack test dataset D adv_te All original samples Sb adv_te,ty , 1≤ty≤Te, are all replaced by the corresponding ty-th adversarial examples.

7. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1 is characterized in that: The step S5 specifically includes the following sub-steps: S51: First, for the population P gen Each individual p i Assign two key parameters: the dominant individual p i The number of individuals ind_num p and by individual p i Dominant individual set Slu p ; Secondly, set rto = 1, ind_num p = 0 individuals are included in the frontier Ft rto In; Again, on the frontier Ft rto All individuals in Slu p Traverse and set the ind_num of each individual p Subtract 1; then, set rto = rto + 1, and then ind_num p = 0 is included in the frontier Ft rto Finally, repeat the above steps until the population P gen All individuals p in i All are included in a certain frontier Ft rto middle; S52: After completing the non-dominated sorting, the crowding degree of each individual in the population is calculated. The specific steps of the crowding degree calculation are: first, take all individuals in a single frontier and sort them from small to large according to the fitness value of a certain optimization goal; second, take the largest fitness value as the fit o,max , the minimum fitness value is taken as fit o,min , and fit the two extreme values o,max and fit o,min The crowding distance between them is set to infinity, denoted as inf; again, the crowding distance between each individual and the nearest neighbor is calculated on the optimization target, and the two extreme values ​​of the crowding distance on the corresponding optimization target are fit o,max and fit o,min Min-max normalization is performed between the two sets of parameters, where the min-max normalization calculation formula is shown in formula (12) and the crowding calculation formula is shown in formula (13); then, all optimization objectives are traversed and the standardized crowding distances of all objectives are added together; finally, the next Pareto front is entered until the crowding distances of all individuals are calculated; Where, Represents the normalized fitness value of the i-th individual on the optimization target o; represents the fitness value of the i-th individual on the optimization target o, fit o,max and fit o,min Respectively represent the maximum and minimum values ​​of the congestion distance on the corresponding optimization target; dt i represents the crowding distance of the i-th individual; t represents the total number of optimization targets; Represents the fitness value of the i+1th individual on the optimization target o; Represents the fitness value of the i-1th individual on the optimization target o; S53: Offspring population Q gen The generation steps are as follows: randomly select a population P gen There are three groups of individuals with two individuals in each group. A binary tournament selection is performed on the undominated individuals with a large crowding degree in each group, and the selected individuals are used as parents, denoted as s1, s2 and s3. Among them, the undominated individuals are preferred. If the number of individuals dominated is the same, the individuals with a large crowding degree are considered. The mutation operation is performed on the parents s1, s2 and s3 as the basic individuals. The mutation generation method of the offspring individuals q1 and q2 is shown in formula (14) and formula (15): q 1,d =s 1,d +S×(s 2,d -s 3,d ) (14) q 2,d =s 2,d +S×(s 3,d -s 1,d ) (15) Where S represents the variation factor that changes with the number of iterations, d represents the dth dimension of the individual code, 1≤d≤D; q 1,d and q 2,d Represents the dth dimension of offspring individuals q1 and q2 respectively; s 1,d 、s 2,d and s 3,d They represent the dth dimension of the parent individuals s1, s2, and s3 respectively. After the generation of offspring individual mutations is completed, the generated offspring individuals are subjected to a crossover operation. The specific operation is shown in formulas (16) and (17): Where rand 1,d and rand 2,d Represents a random decimal generated between [0,1], s 1,d and s 2,d Represent the dth dimension of parent individuals s1 and s2 respectively, and Cr represents the crossover factor that changes with the number of iterations; after completing the crossover operation, let q1 = q′1, q2 = q′2; in the process of mutation and crossover, if the generated new code exceeds the preset range, the value exceeding the preset range of the code will be truncated, and the value exceeding the upper limit of the range will be truncated to the maximum value preset for the dth dimension code, and the value exceeding the lower limit of the range will be truncated to the minimum value preset for the dth dimension code, so that the generated offspring individual code is always within the preset range; then repeat the above operation until the offspring population Q gen The population size reaches NP; finally, record the current number of iterations gen.

8. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1 is characterized in that: In step S6, the external archive solution Ach gen The update specifically includes the following steps: S61: First, the parent population P gen and the offspring population Q gen Merge into a new population, where there are 2×NP individuals in the new population; secondly, gen With Q gen Perform the non-dominated sorting and crowding calculation described in step S6 on all individuals in the parent population, and filter the first NP individuals to obtain a new parent population P. gen ; Again, the current parent population P gen All individuals in the Pareto front are saved to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update the archive using the update method; S62: External archive solution Ach gen The update method is as follows: First, in the current population P gen All individuals in the Pareto front are saved to Ach gen After the middle, to Ach gen Perform non-dominated sorting and crowding calculation on all individuals in Ach gen To update, if Ach gen The number of individuals in the archive exceeds the preset maximum number of external archive individuals A max , then delete Ach gen Individuals with more dominated solutions and lower crowding degree go directly to Ach gen The number of individuals in the archive is less than the maximum number of individuals in the external archive A max .

9. The power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to claim 1, characterized in that: The S8 specifically includes: Unpack external archive Ach gen All non-dominated individuals in the Pareto front solution are taken as the Pareto front solution, and the optimal compromise individual is selected from the Pareto front solution. The model architecture and parameters encoded by the individual are used as the architecture and parameters of the optimal robust-lightweight Transformer model, thereby screening out a robust-lightweight Transformer model that takes into account the F1 score of multi-classification detection of benign samples, the success rate of adversarial attacks, and the minimization of the number of model parameters. It is then used for online deployment of power grid intrusion detection to achieve adversarial defense of the smart grid intrusion detection system.

10. A power grid countermeasure defense system based on robust-lightweight Transformer model optimization, incorporating the power grid countermeasure defense method based on robust-lightweight Transformer model optimization according to any one of claims 1 to 9, characterized in that: The system comprises: Data preprocessing module, used to construct a training dataset D for robust-lightweight Transformer model optimization based on smart grid cyber-physical fusion system data tr With the test data set D te ; The population generation module is used to set multiple first parameters, construct the optimization target of the robust-lightweight Transformer model, and integer encode the model architecture and hyperparameters to be optimized according to the preset first parameters to generate the initial population P gen ; Adversarial training judgment module, used to evaluate the population P gen The model parameter amount and F1 score of all individuals in the training set are calculated, and it is determined whether the F1 score is greater than the adversarial training threshold. If not, the individual adversarial training is interrupted and the pre-evaluated model parameter amount and F1 score are saved, and the adversarial attack success rate is set to 100%. If so, the adversarial training and evaluation module 40 is entered for adversarial training and testing. Adversarial training and evaluation module is used to transform individual p i The characterized architecture and hyperparameters are decoded into the Transformer model as a replacement model for adversarial attack, and the adversarial attack training dataset D is generated based on the replacement model and projected gradient descent attack. adv_tr And the adversarial attack test dataset D adv_te , then based on the training data set D tr With adversarial attack training dataset D adv_tr For individual p i After the training is completed, the adversarial training is performed based on the test data set D te Evaluate the F1 score of individual benign sample detection based on the adversarial attack test dataset D adv_te Evaluate the success rate of adversarial attacks; The offspring population generation module is used to generate P gen Perform non-dominated sorting and crowding calculation on all individuals in to generate the offspring population Q gen , and record the number of iterations gen; External archive update module for population P gen With Q gen Perform non-dominated sorting and crowding calculation on all individuals in to obtain the new parent population P. gen , save all individuals in the current Pareto front to the external archive solution set Ach gen In the proposed external archive solution Ach gen Update the archive using the update method; The external archive saving module is used to save the external archive solution set Ach when the current iteration number reaches the maximum iteration number Gen gen A robust-lightweight Transformer model represented by all individuals in ; The online deployment module is used to screen robust and lightweight Transformer models that take into account the F1 score of benign samples in multi-classification detection, the success rate of adversarial attacks, and the minimization of the number of model parameters, and use them for online deployment of power grid intrusion detection.

Images